Catalog Home Page

Implications and applications of modern test theory in the context of outcomes based education

Andrich, D. (2002) Implications and applications of modern test theory in the context of outcomes based education. Studies in Educational Evaluation, 28 (2). pp. 103-121.

Link to Published Version: http://dx.doi.org/10.1016/S0191-491X(02)00015-9
*Subscription may be required

Abstract

This article is based on the articulation of student outcome statements within a general movement of outcomes based education which is placed into a hierarchical component analysis at different levels of scale (Andrich, 2002). Two important ideas emerge from this conceptualisation. First, qualitative differences among strands of a construct can be merged to make a higher order construct; second, a succession of quantitative differences along a construct become qualitative differences. This article shows that the probabilistic Rasch model of modern test theory is compatible with such a hierarchical component analysis. The model can be used to locate items empirically on a continuum of achievement to produce evidence as to where there might be problems in operating tasks, clues as to where to focus in understanding students' problems with concepts, and clues for constructing and improving marking keys for both dichotomously and polytomously scored items. Two issues of model fit between the data and the model were elaborated briefly. First, it was indicated that there were two paradigmatic approaches to this issue in modern test theory: in one, the case for a chosen model is that the model describes the data, and if one model does not describe the data well enough according to the researcher's criteria, then a model with more parameters which does describe the data is chosen; in the other, criteria for the invariance of comparisons of persons and items are articulated into a probabilistic model for relevant kinds of data, and if the data do not accord with the model, then the reasons for this misfit are studied, not only from a technical perspective on the items, but also for their possible substantive implications. Second, it was stressed that in carrying out test of fit between the data and the model, the level of scale and precision at which this is done needs to be kept in the foreground. Finally and most importantly the article highlights the important insight from this model that students will show increasing competence at a particular level by a judicious progression along the continuum to more challenging work rather than by merely repeating work at the particular level. Appreciating the probabilistic implication of this model can also be helpful in recognizing that the probabilistic relationship between the hypothesized location of the person and the items reflects an imperfect relationship between any manifest performance and the latent competence.

Publication Type: Journal Article
Publisher: Elsevier
Copyright: © 2002 Elsevier Science Ltd
URI: http://researchrepository.murdoch.edu.au/id/eprint/33138
Item Control Page Item Control Page