The influence of stakeholder involvement in evaluation studies on the use of evaluation information: A longitudinal study

Cummings, Richard J. (1997) The influence of stakeholder involvement in evaluation studies on the use of evaluation information: A longitudinal study. PhD thesis, Murdoch University.

Abstract

The level of use of the information provided by an evaluation study has been shown to be influenced by the involvement of the stakeholders in the study, their commitment to the study and their attitudes to using the information. However, the research supporting these findings generally has had significant limitations, investigating only one factor at a time and employing retrospective and simulation research designs. Few longitudinal studies have been conducted which could examine these factors conjointly as well as their relationship to the interaction of stakeholders and evaluation consultants over the period of an ongoing evaluation study.

The purpose of the research reported in this dissertation was to investigate over the period of an evaluation study, changes in the commitment, attitudes and involvement of stakeholders and their use of the information provided by these studies. A theoretical model, the Stakeholder Involvement Model, was developed to account for changes in these variables and their relationships. This model draws on the theory of reasoned action of Fishbein and Ajzen (1980), Janis and Mann's (1977) conflict model of decision making, Polanyi's (1958) theory of personal knowledge, Patton's (1978) concept of the personal factor, Hammond's (1983) concept of personal knowledge, and Vlahov's (1989) model of evaluation utilization. A broad, multifaceted construct of use of evaluation information and expanded constructs of stakeholder commitment and involvement are incorporated into the model.

A longitudinal design was used to test the Stakeholder Involvement Model and to compare it to three alternative models. The data came from structured interviews conducted on two occasions with a total of 83 stakeholders who were associated with one of ten evaluation studies of a variety of social programmes. The analysis was conducted using structural equation modelling, which is well suited to testing longitudinal models.

The study found that the level of commitment of stakeholders and their involvement in an evaluation study varied considerably and that they made use of the evaluation information in a wide range of ways. Furthermore, stakeholder involvement was found to be positively related to both commitment and attitudes towards using the information from the study, and also exerted the greatest influence on the use of the information. In the comparative analysis, none of the models in their original form was found to fit the data adequately, but when modified using exploratory analysis three of the models fitted the data very well. Of these, the Stakeholder Involvement Model accounted for the greatest amount of variance in reported stakeholder use of information. These findings highlight the importance of stakeholder involvement in programme evaluation, but the exploratory nature of the analysis indicates that further research using an independent sample is required for confirmation of the findings.

The major conclusions of the study are that stakeholders do make substantial use of evaluation information and that their level of use is most influenced by their involvement in the evaluation study. These conclusions have important implications for the practice of programme evaluation, particularly for the nature and frequency of the relationship between evaluators and stakeholders and its influence on the eventual use of the results of the evaluation study. The thesis concludes with a discussion of this and other implications.

Item Type:	Thesis (PhD)
Murdoch Affiliation(s):	School of Education
Notes:	Note to the author: If you would like to make your thesis openly available on Murdoch University Library's Research Repository, please contact: repository@murdoch.edu.au. Thank you.
Supervisor(s):	Straton, Ralph
URI:	http://researchrepository.murdoch.edu.au/id/eprint/50244

Item Control Page