DOI: 10.3886/icpsr36387.v1
title: Impacts and Implementation of the i3-Funded Scale-Up of Success for All
title: Version 1

description: The Impacts and Implementation of the i3-Funded Scale-Up of Success for All study examined the implementation and impacts of a whole-school reform model, Success for All (SFA), which seeks to increase reading proficiency among students in elementary schools serving low-income children, as this model was scaled up under an Investing in Innovation (i3) grant from the United States Department of Education. The impact evaluation used a cluster random assignment design in which 37 schools in five school districts were selected at random either to implement SFA (the program group schools) or to put in place their business-as-usual reading program (the control group schools). This collection contains Parts 1 to 7 for public use; and Part 8 for restricted use only. In Part 1: Primary Student Sample data, children who began kindergarten in sampled schools and remained in them through second grade were assessed using standardized assessments of phonics skills, reading fluency, and comprehension. In Part 2: Auxiliary Sample, additional analyses examined third-year impacts on reading skills for all second graders, whether or not they remained continuously enrolled in the study schools, and for students in grades 3 through 5. The analysis also examined impacts for subgroups defined by ethnicity, early literacy skills measured upon entry into kindergarten, and other variables. Part 3: School Achievement Snapshot Data contain implementation data for analysis using an instrument created by SFA's developers that measures the extent to which program elements were put in place at each program group at the end of each year. This instrument was utilized to assess fidelity to the program model. The implementation analysis was also informed by principal (Part 5: Principal Survey) and teacher surveys (Part 4: Teacher Survey) in all schools, as well by logs (Part 6: Teacher Log) that asked teachers to document their instructional practices in early reading classes. These data, collected in all three study years, make it possible to measure implementation over the study period and to compare program group schools and control group schools on a variety of dimensions. Part 7: Scale-Up Sample Data provided data on the schools that adopted SFA (and those which were approached, but did not adopt), and schools that adopted before the Investing in Innovations grant. Part 8 is the restricted version of Part 7, which retains geographic location variables.
description: MDRC employed a cluster random assignment research design to determine whether Success for All was as effective in its larger-scale version as it had been in earlier tests of the program. In this design, 37 schools located in five districts were assigned at random: 19 to receive the program and the remaining 18 to constitute a "business-as-usual" control group. The evaluation compared the reading outcomes of kindergartners in both sets of schools in the first year of the study, and of first-graders and second-graders in the next two years, using age-appropriate assessments that were administered individually to each child. During the second and third years of the study, students in third through fifth grades were also assessed, this time in a group setting. Student scores on state standardized tests served as additional measures of reading skills. The impact analysis also measured the program's effects on student attendance and grade promotion. Impacts were unlikely to occur unless schools were able to implement the program with reasonable fidelity. MDRC collected and analyzed data from a variety of sources -- teacher logs, principal and teacher surveys, and rating sheets completed by the coaches -- to examine the implementation experiences of the schools participating in the study. The analysis also compared the characteristics and implementation levels of the study schools with those of other schools that put Success for All in place as part of the i3 replication.
description: Part 2, Auxiliary Student Sample Data, contains three weight variables: SCH26_GR3_WT, SCH26_GR4_WT, and SCH26_GR5_WT. In general, ICPSR provides unweighted data, but Part 3, School Achievement Snapshot Data, contains weighted variables of scores. Users are highly encouraged to refer to the User Guide for more information on weights.
description: ICPSR data undergo a confidentiality review and are altered when necessary to limit the risk of disclosure. ICPSR also routinely creates ready-to-go data files along with setups in the major statistical software formats as well as standard codebooks to accompany the data. In addition to these procedures, ICPSR performed the following processing steps for this data collection: Checked for undocumented or out-of-range codes..
description: Presence of Common Scales: Woodcock-Johnson Letter Word Identifcation, Woodcock-Johnson Word Attack, Woodcock-Johnson Passage Comprehension, Test of Word Reading Efficiency, Gates-MacGinitie Reading Test-Vocabulary, Gates-MacGinitie Reading Test-Comprehension
description: Response Rates: The response rate varied by instrument ranging from 85% to 100%
description: Datasets: 
 
 DS0: Study-Level Files 
 DS1: Primary Student Sample Data 
 DS2: Auxiliary Student Sample Data 
 DS3: School Achievement Snapshot Data 
 DS4: Teacher Survey Data 
 DS5: Principal Survey Data 
 DS6: Teacher Log Data 
 DS7: Scale-Up Sample Data For Public Use 
 DS8: Scale-Up Sample Data For Restricted Use
description: record abstracts, cognitive assessment test, paper and pencil interview (PAPI), on-site questionnaire 
Part 8 is for restricted use only; it contains the same data as Part 7, with geographic location variables included: longitude and latitude.
 For more information about the Success for All study, please refer to the MDRC Web site.
description: 37 schools including their teachers, principles and students in five school districts in four states that agreed to participate in the study. Smallest Geographic Unit: District (anonymized)
description: 37 schools within 5 pre-specified school districts in 4 states agreed to participate in the study. The schools within each district were randomly assigned to program or control conditions, which produced 19 program group schools and 18 control group schools.