A Summary of the BURST[R]: Reading Efficacy Trial
Author: Rowan, Brian
Date: 2019-03-00
Type: Reports - Research
Subjects: Reading Programs,Reading Instruction,Randomized Controlled Trials,Reading Achievement,Elementary School Students,Primary Education,Emergent Literacy,Supplementary Education,Reading Difficulties,Reading Fluency,Reading Tests,Experimental Groups,Control Groups,Outcomes of Education,Instructional Effectiveness,Program Implementation

This report summarizes the results of a cluster-randomized field trial that estimated the effect of &quot;BURST[R]: Reading&quot; on primary grades students' early literacy achievement. BURST is a widely adopted supplemental reading program designed for use with students struggling to acquire early literacy skills and is meant to provide supplemental instruction to these students outside the regular reading program. The program uses an &quot;assess, group, instruct&quot; format in which schools identify struggling readers using the &quot;DIBELS Next&quot; assessment, then use a proprietary algorithm to place identified students into reasonably homogenous skill groups on the basis of DIBELS results, and then provide targeted instruction to these groups using BURST curriculum and lesson materials. Over the four-year period AY 2013-2014 to AY 2016-2017, the University of Michigan (in cooperation with Amplify, Inc.) carried out a study in 52 high-poverty schools serving grades K-3 located in 9 states in different geographic areas of the United States during the period AY 2013-2014 to AY 2016-2017. The study randomly assigned 27 schools to the BURST treatment group and 25 to a control group that was provided free access to the &quot;DIBELS Next&quot; assessment for use in a regular universal screening process. More than 29,000 students enrolled in grades K-3 at treatment and control schools participated in the study contributing about 1.8 observations per student. Data analysis showed no evidence of differential attrition in the study groups, there was strong evidence of baseline equivalence of the treatment and control samples in the study, and cross-over from one experimental condition to the other was minimal and similar across treatment and control groups. The study found that schools assigned to the BURST treatment group offered BURST instruction to both struggling and non-struggling readers and that the average struggling reader received about 40 hours of BURST instruction in a given a year such that (over a four year period) the average struggling reader in a BURST school could be expected to accumulate between 120 and 140 hours of BURST instruction. Overall rates of provision of BURST instruction in study schools was found to be similar to rates of provision of BURST instruction in schools with similar demographic characteristics that had purchased and were using the BURST program outside the efficacy trial in AY2016-2017 but less than the amount of instruction recommended by the program developer. Using the Star Early Literacy assessment as the primary outcome, and after adjusting this outcome for several pre-treatment covariates using a Peters-Belson type strategy, the study estimated sample average treatment effects on students' early literacy learning using permutation tests that took into account the various forms of clustering in the experimental design and that controlled statistical significance tests for family wise error rates due to multiple comparisons. The results of these statistical tests showed that the BURST program did not have statistically significant effects on the early reading achievement of all students who attended BURST schools, did not have statistically significant positive effects on the early reading achievement of struggling readers who attended BURST schools, did not have statistically significant positive effects on the early reading achievement of students who attended BURST schools for three or more years consecutively, and did not have statistically significant positive effects on the early reading achievement of students who attended BURST schools that had a predicted probability greater than 1% of complying (versus not complying) with treatment assignment. There was only slight evidence of school-to-school variability in program effects, with schools implementing BURST instruction at higher rates tending to have slightly larger positive effects on students' early reading achievement than schools implementing BURST instruction at lower rates. These differences were very small, however, and as such, were not assessed for statistical significance. In all, the study's results are best summarized as follows: In a sample of 52 schools located in school districts that are smaller than the average U.S. school district, in communities that are more disadvantaged than average, and serving higher percentages of lower achieving students than average, the added benefit of using BURST for supplemental reading instruction under routine conditions of implementation was found to be negligible compared to engaging in universal screening with DIBELS and conducting supplemental reading instruction under business as usual conditions.