The Impact of the Measures of Academic Progress (MAP) Program on Student Reading Achievement. Final Report. NCEE 2013-4000
Author: Cordray, David
Date: 2012-12-00
Type: Reports - Research
Subjects: Academic Achievement,Student Characteristics,Reading Achievement,Learner Engagement,Program Effectiveness,Individualized Instruction,Standardized Tests,Control Groups,Elementary Education,Fidelity,Reliability,Benchmarking,Teacher Education,Experimental Groups,Teacher Surveys,Observation,Grade 4,Grade 5,Elementary School Students,Research Methodology,Recruitment,Response Rates (Questionnaires),Statistical Significance

During the past decade, the use of standardized benchmark measures to differentiate and individualize instruction for students received renewed attention from educators. Although teachers may use their own assessments (tests, quizzes, homework, problem sets) for monitoring learning, it is challenging for them to equate performance on classroom measures with likely performance on external measures, such as statewide tests or nationally normed standardized tests. One of the most widely used commercially available systems incorporating benchmark assessment and training in differentiated instruction is the Northwest Evaluation Association's (NWEA) Measures of Academic Progress (MAP) program. The MAP program includes: (1) computer-adaptive assessments administered to students three or four times a year; and (2) teacher training and access to MAP resources on how to use data from these assessments to differentiate instruction. MAP tests and training are currently in use in nearly 20 percent of K-12 school districts nationwide and more than a third of districts in the Midwest. Although the technical merits and popularity of MAP assessments have been widely referenced in practitioner-oriented journals and teacher magazines, few studies have investigated the effects of MAP or other benchmark assessment programs on student outcomes. This study was designed to address questions from Midwestern states and districts about the extent to which benchmark assessment may affect teachers' differentiated instructional practices and student achievement. Thirty-two elementary schools in five districts in Illinois participated in a two-year randomized controlled trial to assess the effectiveness of the MAP program. Half the schools were randomly assigned to implement the MAP program in grade 4, and the other half were randomly assigned to implement MAP in grade 5. Schools assigned to grade 4 treatment served as the grade 5 control condition, and schools assigned to grade 5 treatment served as the grade 4 control. The results of the study indicate that the MAP program was implemented with moderate fidelity but that MAP teachers were not more likely than control group teachers to have applied differentiated instructional practices in their classes. Overall, the MAP program did not have a statistically significant impact on students' reading achievement in either grade 4 or grade 5. Appended are: (1) School and Student Characteristics; (2) Impact Estimation and Impact Estimates; (3) Results of Sensitivity Analyses; (4) Missing Data Imputation Procedures; (5) Response Rates on Surveys, Logs, and Classroom Observations; (6) MAP Observation Protocol; (7) MAP Instructional Logs; (8) MAP Teacher Survey for MAP Teachers; (9) MAP Student Engagement Survey; (10) MAP School Leader Survey; (11) MAP Recruitment Process; (12) Assessment of Control Group Contamination and of Integrity of Year 2 Intervention--Control Contrast; (13) Implementation Fidelity and Achieved Relative Strength; and (14) The Achieved Relative Strength Index. (Contains 6 figures, 56 tables, 1 box, and 98 footnotes.)