Estimating Parameters in the Generalized Graded Unfolding Model: Sensitivity to the Prior Distribution Assumption and the Number of Quadrature Points Used.

The generalized graded unfolding model (J. Roberts, J. Donoghue, and J. Laughlin, 1998, 1999) is an item response theory model designed to unfold polytomous responses. The model is based on a proximity relation that postulates higher levels of expected agreement with a given statement to the extent that a respondent is located close to the statement on a unidimensional latent continuum. J. Roberts and others (1998) have examined the recovery of item and person parameters from the generalized graded unfolding model. Item and person parameters were estimated. This study used simulation methods to assess the sensitivity of model parameters to the prior distribution used in these estimation procedures. It also examined the effects of the number of quadrature points used in the numerical integration process and the calibration sample on the accuracy of the resulting parameter estimates. The results show that item parameter estimated derived from the marginal maximum likelihood procedure were fairly robust to discrepancies between the prior and true distributions of person parameters. Consequently, the person parameter estimates derived from the expected a posteriori procedure were generally robust as well, except for those individuals with the most extreme response patterns. The results also indicate that 20 quadrature points are adequate for the accurate recovery of model parameters. These findings will help establish the utility of the generalized graded unfolding model in applied measurement situations and will promote the use of item response theory in the attitude/preference measurement domain. (Contains 2 tables, 14 figures and 19 references.) (Author/SLD) ******************************************************************************** * Reproductions supplied by EDRS are the best that can be made * * from the original document. * ******************************************************************************** Estimating Parameters in the Generalized Graded Unfolding Model: Sensitivity to the Prior Distribution Assumption and the Number of Quadrature Points Used James S. Roberts Medical University of South Carolina John R. Donoghue Educational Testing Service James E. Laughlin University of South Carolina Paper Presented at the Annual Meeting of the National Council on Measurement in Education, April 22, 1999, Montreal, PQ, Canada Visit Our Web Site Devoted to Item Response Theory Models for Unfolding: www.musc. edu/cdap/roberts U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) rtrtl-h-i;document has been reproduced as received from the person or organization originating it. CI Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. PERMISSION TO REPRODUCE AND I DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) j 2 BEST COPY AVMLABLE Abstract The generalized graded unfolding model (Roberts, Donoghue and Laughlin, 1998, 1999) is an item response theory model designed to unfold polytomous responses. It is appropriate for data obtained in situations where subjects respond to a series of statements using either a binary or graded scale of agreement (e.g., situations where Thurstone or Likert attitude measurement procedures are employed). The model is based on a proximity relation which postulates higher levels of expected agreement with a given statement to the extent that a respondent is located close to the statement on a unidimensional latent continuum. Roberts et al. (1998) have examined the recovery of item and person parameters from the generalized graded unfolding model. Item parameters were estimated with a marginal maximum likelihood technique in which person parameters where numerically integrated out of the likelihood function based on a prior distribution. Person parameters were estimated using an expected a posteriori technique which also required numerical integration over a prior distribution of person parameters. Roberts et al. (1998) examined recovery of model parameters when the prior distribution of person parameters perfectly matched the true distribution. In contrast, this study used simulation methods to assess the sensitivity of model parameter estimates to the prior distribution used in these estimation procedures. It also examined the effects of the number of quadrature points used in the numerical integration process and the calibration sample size on the accuracy of the resulting parameter estimates. The results showed that item parameter estimates derived from the marginal maximum likelihood procedure were fairly robust to discrepancies between the prior and true distributions of person parameters. Consequently, the person parameter estimates derived from the EAP procedure were generally robust as well, except for those individuals with the most extreme response patterns. The results also indicated that 20 quadrature points were adequate for the accurate recovery of model parameters. These findings will help establish the utility of the generalized graded unfolding model in applied measurement situations and will promote the use of item response theory in the attitude/preference measurement domain.The generalized graded unfolding model (Roberts, Donoghue and Laughlin, 1998, 1999) is an item response theory model designed to unfold polytomous responses. It is appropriate for data obtained in situations where subjects respond to a series of statements using either a binary or graded scale of agreement (e.g., situations where Thurstone or Likert attitude measurement procedures are employed). The model is based on a proximity relation which postulates higher levels of expected agreement with a given statement to the extent that a respondent is located close to the statement on a unidimensional latent continuum. Roberts et al. (1998) have examined the recovery of item and person parameters from the generalized graded unfolding model. Item parameters were estimated with a marginal maximum likelihood technique in which person parameters where numerically integrated out of the likelihood function based on a prior distribution. Person parameters were estimated using an expected a posteriori technique which also required numerical integration over a prior distribution of person parameters. Roberts et al. (1998) examined recovery of model parameters when the prior distribution of person parameters perfectly matched the true distribution. In contrast, this study used simulation methods to assess the sensitivity of model parameter estimates to the prior distribution used in these estimation procedures. It also examined the effects of the number of quadrature points used in the numerical integration process and the calibration sample size on the accuracy of the resulting parameter estimates. The results showed that item parameter estimates derived from the marginal maximum likelihood procedure were fairly robust to discrepancies between the prior and true distributions of person parameters. Consequently, the person parameter estimates derived from the EAP procedure were generally robust as well, except for those individuals with the most extreme response patterns. The results also indicated that 20 quadrature points were adequate for the accurate recovery of model parameters. These findings will help establish the utility of the generalized graded unfolding model in applied measurement situations and will promote the use of item response theory in the attitude/preference measurement domain.