Measuring 21st Century Reading Comprehension through Scenario-Based Assessments
Author: Shore, Jane R.
Date: 2017-00-00
Type: Reports - Evaluative
Subjects: Reading Comprehension,Reading Skills,21st Century Skills,Vignettes,Psychometrics,Student Evaluation,Evaluation Methods,Formative Evaluation,Reading Tests,Scaffolding (Teaching Technique),Language Arts,Test Validity

In the United States, educational thought leaders have called for higher expectations (Gordon Commission, 2010), more rigorous college and career readiness standards for K-12 education (National Governors Association Center for Best Practices &amp; Council of Chief State School Officers, 2010), and new constructs such as 21st century skills including collaborative problem solving in digital environments (Partnership for 21st Century Skills, Coiro, Castek, &amp; Henry, 2013). They have also challenged assessment developers to address how best to provide information that is useful for instruction for various learners (Gordon Commission, 2013; Purpura &amp; Turner, 2014; Watkins &amp; Lindahl, 2010). Expanding the scope and variety of constructs (e.g., including elements like collaborative learning and digital literacy) will ensure measurement keeps pace with how people function and interact in various everyday reading activities. This chapter describes a new assessment design approach called scenario-based assessment (SBA) and explains how it can be used to measure the reading ability of school-aged children in the current context of high standards. SBA combines a cluster of techniques for delivering a set of tasks and items that provide a platform for measuring the kinds of demanding reading skills, while simultaneously affording the potential to increase the instructional relevance of the assessment. SBAs in reading typically include a range of principles and techniques that distinguish them from other types of assessments: (1) they provide an authentic purpose for reading, (2) they place reading in context for completing a set of interrelated activities that may move from more guided to independent performance, (3) items tend to require the integration and evaluation of a wide range of diverse sources and, (4) in many cases, items provide scaffolds (e.g., a graphic organizer for an analysis of text structures) and guidelines (e.g., tips for summary writing) to help better understand and model the target performance in the assessment (O'Reilly &amp; Sabatini, 2013). Some SBAs also include items that model the social aspects of literacy and learning, such as engaging with peers or a teacher to clarify understanding in reading, reviewing and evaluating peer writing. Using these principles, SBAs may broaden the range of interactions, perspectives, and information a test taker is exposed to on a topic. Ultimately, the key aims of scenario-based reading assessments are to measure 21st century reading ability while simultaneously supporting reading development and instructional usefulness. In this chapter, we delineate two types of SBAs in reading, the Global, Integrated Scenario-Based Assessment (GISA) and English Language Formative Assessment (ELFA). These two assessments were part of two separate research projects. GISA was developed with a primary focus on benchmark or summative applications, across kindergarten through 12th grade. ELFA, on the other hand, was developed as a classroom-based, formative assessment of reading comprehension at the middle-school grade level. The GISA framework and design relied on computer delivery and principles from cognitive science, whereas ELFA was paper-based for its easier integration in daily instruction. Employing the SBA approach to developing reading assessments, both projects also aimed to build their SBAs to be feasible and practical, while maintaining adequate psychometric properties. Consequently, we also briefly describe some empirical evidence collected to date in support of these aims. We conclude this chapter with some considerations in designing SBA assessments based on the lessons we have learned from GISA and ELFA. [This manuscript is an early draft of a paper published in: M. K. Wolf, &amp; Y. G. Butler (Eds.), &quot;English Language Proficiency Assessments for Young Learners,&quot; (pp. 234-252). New York, NY: Routledge.]