Computerized Multistage Testing (2016)

Test Design, Item Pool, and Maintenance Overview of Computerized Multistage Tests Duanli Yan, Charles Lewis, and Alina A. von Davier Linear Tests and Computerized Adaptive Tests (CATs) Multistage Tests (MSTs) MST Designs for Different Purposes Implementation Schemes Designing MST Content Balance and Assembly Exposure Control Routing Scoring, Linking, and Equating Reliability, Validity, Fairness, and Test Security Current and Future Applications Logistic Challenges Summary Multistage Test Designs: Moving Research Results into Practice April L. Zenisky and Ronald K. Hambleton The MST Design Structure The State of Research: MST Development and Design Considerations Conclusions and Next Steps Item Pool Design and Maintenance for Multistage Testing Bernard P. Veldkamp Designing an Item Pool Blueprint Applications in Item Writing Maintenance Discussion Mixed-Format Tests: Issues and Methods Jiseon Kim and Barbara G. Dodd Literature Review on Design Components in Mixed-Format MST Comparing Other Testing Approaches Issues and Future Research Suggestions for Mixed-Format MST Conclusion Design and Implementation of Large-Scale Multistage Testing Systems Richard Luecht MST Design and Implementation Considerations Conclusions: A Research Agenda Test Assembly Overview of Test Assembly Methods in Multistage Testing Yi Zheng, Chun Wang, Michael J. Culbertson, and Hua-Hua Chang MST Framework MST Assembly Design Automated Assembly for MST Setting Difficulty Anchors and Information Targets for Modules On-the-Fly MST (OMST) Assembly Paradigm MST, CAT, and Other Designs-Which Way to Go? Using a Universal Shadow-Test Assembler with Multistage Testing Wim J. van der Linden and Qi Diao Solving Shadow-Test Assembly Problems Basic Design Parameters Different Testing Formats Relative Efficiency of Formats Empirical Study Concluding Comments Multistage Testing by Shaping Modules on the Fly Kyung (Chris) T. Han and Fanmin Guo MST by Shaping MST-S versus MST-R versus CAT Discussion and Conclusion Optimizing the Test Assembly and Routing for Multistage Testing Angela Verschoor and Theo Eggen Optimizing MST Assembly: A Nonexhaustive Search Limited Item Pools, Two- and Three-Parameter Models Discussion Routing, Scoring, and Equating IRT-Based Multistage Testing Alexander Weissman Introduction Motivation for Tailored Testing Routing Rules Scoring and Classification Methodologies Final Comments A Tree-Based Approach for Multistage Testing Duanli Yan, Charles Lewis, and Alina A. von Davier Regression Trees Tree-Based Computerized Adaptive Tests Tree-Based Multistage Testing Algorithm An Application Discussion Limitations and Future Research Multistage Testing for Categorical Decisions Robert Smith and Charles Lewis Computer-Mastery Methods Information Targeted at Cut versus at Ability Influence of Multiple Cut Scores Factors That Can Reduce Optimal Solutions Example Based on Smith and Lewis (1995) Adaptive Mastery Multistage Testing Using a Multidimensional IRT Model C.A.W. Glas Introduction Definition of the Decision Problem Computation of Expected Loss and Risk Using Backward Induction Selection of Items and Testlets Simulation Studies Conclusions and Further Research Multistage Testing Using Diagnostic Models Matthias von Davier and Ying (Alison) Cheng The DINA Model and the General Diagnostic Model Experience with CD-CATs CD-MSTs Discussion Considerations on Parameter Estimation, Scoring, and Linking in Multistage Testing Shelby J. Haberman and Alina A. von Davier Notation The Item Response Model The Test Score Approaches to Parameter Estimation Conclusions Test Reliability, Validity, Fairness, and Security Reliability of Multistage Tests Using Item Response Theory Peter W. van Rijn Test Reliability Application: IRT Reliability for MST in NAEP Conclusion Multistage Test Reliability Estimated via Classical Test Theory Samuel A. Livingston and Sooyeon Kim The Estimation Procedure Testing the Accuracy of the Estimation Procedure How Accurate Were the Estimates? Evaluating Validity, Fairness, and Differential Item Functioning in Multistage Testing Rebecca Zwick and Brent Bridgeman Content Balancing Opportunities for Item Review and Answer Changing Skipping Strategies MST Routing Algorithms The Digital Divide Comparability of Computer Platforms Accommodations for Students with Disabilities and English Language Learners Differential Item Functioning Analysis in MSTs Application of the Empirical Bayes DIF Approach to Simulated MST Data Summary Test Security and Quality Control for Multistage Tests Yi-Hsuan Lee, Charles Lewis, and Alina A. von Davier An Overview of a Three-Component Procedure Tools to Evaluate Test Security and Quality Control A Simulation Study Using CUSUM Statistics to Monitor Item Performance Discussion Applications in Large-Scale Assessments Multistage Test Design and Scoring with Small Samples Duanli Yan, Charles Lewis, and Alina A. von Davier Small Data Sample Item Pool Various MST Module Designs Routing and Scoring Comparisons of the Six MST Designs Discussion Limitations and Future Research The Multistage Test Implementation of the GRE Revised General Test Frederic Robin, Manfred Steffen, and Longjuan Liang From CAT to MST MST Design Implementation Monitoring Summary The Multistage Testing Approach to the AICPA Uniform Certified Public Accounting Examinations Krista J. Breithaupt, Oliver Y. Zhang, and Donovan R. Hare Research on Multistage Testing Item Bank Development for MST Content Security Monitoring for MST Inventory Exposure Planning for MST Discussion Transitioning a K-12 Assessment from Linear to Multistage Tests Carolyn Wentzel, Christine M. Mills, and Kevin C. Meara Administering CTP Items Online Creating a New MST Scale Using IRT Multistage-Adaptive Test Development Score Reporting Summary A Multistage Testing Approach to Group-Score Assessments Andreas Oranje, John Mazzeo, Xueli Xu, and Edward Kulick Targeted Testing Goals of the Study Methods Results Discussion Controlling Multistage Testing Exposure Rates in International Large-Scale Assessments Haiwen Chen, Kentaro Yamamoto, and Matthias von Davier Item Exposure Rate Control for Multistage Adaptive Assessments Method: How to Compute and Adjust the Item Exposure Rates Data Results Conclusion Software Tools for Multistage Testing Simulations Kyung (Chris) T. Han and Michal Kosinski MSTGen R Conclusions Closing Remarks Past and Future of Multistage Testing in Educational Reform Isaac I. Bejar Future of MST A Model-Based Three-Stage Design Item Generation and Automated Scoring and Broadly Accessible Test Content Summary and Conclusions Bibliography