Item Parameter Drift in IRT-Based Licensure Examinations.

The purpose of this study was to investigate whether systematic, non-zero differences between pairs of item bank b-v& ues have occurred in the recent history of two licensure examinations. Licensing examinations were studied for two related health care professions (Program 1 and Program 2). A series of analysis of covariance models was fit to the data in order to investigate the magnitude of changes in item bank b-values and the relationship of any changes to variables indexing factors that have been documented to affect the stability of item parameters. For Program 1, one 300-item form was used, and for Program 2, two 203-item forms were used to obtain a larger number of items. Analyses were first performed to evaluate the relationship, if any, between the dependent variable and the covariates. For both examinations, b-value differences across pairs of item administrations were not influenced by the changing position of the item in different forms. There were systematic changes as a function of time elapsed from a baseline, as discussed in the paper. Item parameter or scale drift was noted for one examination, while bank or pool drift was found for both. Factors that moderated bank drift, and differences between actual form cutscores and cutscores adjusted for documented bank and scale drift ranged from one to five points for the two examinations. Nine tables and five figures present details of the analyses. (SLD) *********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. *********************************************************************** U.S. DEPARTMENT OF EDUCATION Or lice of Educational Research and improvement EDUCATIONAL RESOURCES IWORMATION CENTER (ERIC) This document haS been reproduced as received horn the person or Organization originating o Minor changes have been made to improve reproductudn quality Points of view or opinionS staled in this docu mvnt do not necessarily represent official OERI position or policy "PERMIFSION TO REPRODUCE THIS MATERIAL HAS BEEN GRANTED BY R63Ecr S>lieE,5 TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC). " ITEM PARAMETER DRIFT IN IRT-BASED LICENSURE EXAMINATIONS Robert C. Sykes Ryoko Ito CTB Macmillan/McGraw-Hill This paper was presented in April, 1993 at the Annual Meeting of the National Council on Measurement in Education Atlanta, GA BEST COPY AVAELASLE 2 Introduction The successful equating of test forms is essential for the validity of test scores produced by testing programs that administer multiple forms. For criterion-referenced tests (CRTs), equating is required to adjust forms for differences in difficulty and subsequently to ensure that performance is evaluated relative to a standard which is fixed over time. A criterion level of performance, often defined on a pool or bank scale, must be translated to a raw cutscore for each administered form in a manner that is not influenced by changes in the ability of candidates taking each form. For one-parameter Item Response Theory (IRT) or Rasch-based examinations such as the licensure examinations investigated in this paper, equating of test forms may be accomplished by calibrating an anchor set of scored items on a large representative sample and setting the mean of the calibrated item difficulties or b-values equal to the mean bank b-value for the anchor items. Because all of the anchor items in a form have been previously administered at least once before, although not necessarily in the same previous form, bank b-values exist for all of these items. The resulting, post-equated, current bank bvalues may then be used to generate a raw-to-IRT(theta)-score correspondence table which determines the number of items that must be answered correctly in order to attain a performance standard on the equated form. This passing standard is, in theory, independent of the set of items constituting the