Development of a Descriptive Fit Statistic for the Rasch Model

The Rasch model (Rasch 1960/1980) has become a standard measurement model for the analysis and validation of educational and psychological tests and for the purpose of scaling examinees (Embretson, 2000; Hambleton, 1991; Rupp, 2006). This is due to the appealing properties of the model which are generally referred to as objective measurement (Karabatsos, 2000). These properties include parameter separability, existence of a common interval scale for both persons and items, unidimensionality of measurement, and existence of a sufficient statistic to estimate person and item parameters independently of each other (Baghaei, 2009; Fischer, 2006). Nevertheless, these features are not achieved automatically upon subjecting raw data to the Rasch model. The degree to which these essential properties of measurement are attained depends on the fit of data to the Rasch model. The attractive properties of the Rasch model and the assumptions of the model--that is, unidimensionality, parallel item characteristic curves, and stochastic independence--allow for elegant methods of testing the model. Over the years, numerous global and local statistical tests have been proposed to check the conformity of data to the Rasch model principles (see Maydeu-Olivares, 2013). Most of the fit tests available for the Rasch model rely on the principles of statistical hypothesis testing based on the null hypothesis in which the Rasch model holds, and an alternative hypothesis in which the Rasch model does not hold. The most widely used methods for testing the Rasch model are Andersen's likelihood ratio test (LR test) (Andersen, 1973), Martin-Lof's test (Martin-Lof, 1973), [Q.sub. 1] test (Van den Wollenberg, 1982), and [R.sub. 1] test (Glas, 1988). However, statistical tests for the Rasch model have certain disadvantages that affect their utility and reliability as model checks. The most serious problem of statistical significance testing is the sensitivity to sample size and the number of items. For instance, statistical tests might lead to statistically significant results even under practically negligible deviation from the Rasch model when sample size is large (high statistical power) or statistically nonsignificant results even under large deviations from the Rasch model when samples are small and thus have low statistical power (Gustafsson, 1980). Thus, sample size determination given type-I-risk [alpha], type-II-risk [beta], and model deviation [delta] for testing the Rasch model is needed (Kubinger, 2009). Nevertheless, only a few methods for sample size determination under restrictive assumptions are available (Draxler, 2010; Kubinger, 2011), while most of the statistical tests for the Rasch model do not permit sample size determination. Lastly, statistical tests for the Rasch model test perfect fit while an actual data set never fits a mathematical model perfectly. Due to the above-mentioned limitations of statistical testing, descriptive fit statistics are used to provide a measure for the conformity of data to the Rasch model, which can be either global or local. Local fit statistics, such as the residual-based infit and outfit values (Wright, 1979), provide a measure for the conformity of portions of the data on both items and persons to the Rasch model specifications. Global descriptive fit statistics, i.e., measures for the overall conformity of data to the Rasch model, appear to be rarely used in practice. Consequently, the goal of the present study is to develop global descriptive fit statistics for checking whether the Rasch model holds. Such measures are commonly used in structural equation modeling literature (see West, 2012) but are not used in IRT circles. A number of statistics are proposed and evaluated under different conditions of varying test length and sample size using a simulation study. The proposed statistics are based on the rationale of two statistical tests for the Rasch model, which are reviewed below. …