Does a Summer Reading Program Based on Lexiles Affect Reading Comprehension? Final Report. NCEE 2012-4006
Author: Wilkins, Chuck
Date: 2012-03-00
Type: Numerical/Quantitative Data
Subjects: Reading Comprehension,Independent Reading,Reading Programs,Reading Achievement,Economically Disadvantaged,Parent Participation,Student Interests,Measures (Individuals),Reading,Summer Programs,Grade 3,Effect Size,Surveys,Family Literacy,Control Groups,Outcomes of Treatment,Pretests Posttests,Intervals

This report presents estimates from a large-scale, multi-district RCT (randomized controlled trial) on the effectiveness of a summer reading program on improving student reading comprehension for economically disadvantaged grade 3 students reading below the 50th percentile nationally. This study focused on the summer between grades 3 and 4 for three reasons: (1) independent reading demands increase dramatically in grades 3 and 4 (Chall 1983; National Research Council 1998); (2) the grade 3 to grade 4 transition was not a focus of previous studies; and (3) Texas state assessment data are available for the first time for students beginning in grade 3, and those data were used to control for baseline differences in this study. Each student in the treatment group was sent a single shipment of eight books matched to his or her reading level and interest area during the first part of the summer (June/July 2009), followed by a reminder postcard each week for six weeks. Eight books were chosen because this was the number used in two of the summer reading programs shown to have statistically significant positive effects on reading comprehension for specific subgroups (Kim 2006) or for the entire sample (Kim and White 2008). Seven previous studies examined summer reading programs, and five found a statistically significant improvement in reading achievement following implementation of a reading program (Allington et al. 2010; Butler 2010; Crowell and Klein 1981; Kim 2006; Kim and White 2008). Of the five studies that used an RCT design, three found a statistically significant effect on reading achievement (Allington et al. 2010; Kim 2006; Kim and White 2008). The current study's confirmatory finding did not replicate the findings from these studies. Two of the five RCT studies found that students sent books over the summer reported reading more books than did students who were not sent books (Kim 2007; Kim and Guryan 2010); an exploratory analysis in the current study found similar results. The summer reading program examined in this study did not include teacher support, instructional components, or parent involvement, which several previous studies had included to varying degrees--four RCTs (Kim 2006, 2007; Kim and Guryan 2010; Kim and White 2008) and one quasi-experiment (Butler 2010). These other components could potentially account for differences in observed effects across studies. Also, the program examined in the current study spanned a single summer, whereas the program examined in Allington et al. (2010) spanned three summers. Further, the current study sample consisted of economically disadvantaged students reading below the 50th percentile nationally, while the samples in the studies with statistically significant results consisted of students with economically diverse backgrounds (Kim 2006, 2007; Kim and Guryan 2010; Kim and White 2008) and were not composed exclusively of students reading below the 50th percentile nationally (Allington et al. 2010; Butler 2010; Crowell and Klein 1981; Kim 2006, 2007; Kim and Guryan 2010; Kim and White 2008). One possible inference to draw from this study, and the more recent work of Kim and colleagues (Kim and Guryan 2010; Kim and White 2008), is that some of the components that Kim and his colleagues added--in particular, personalized teacher encouragement of each student to read the books during the summer and brief, small group lessons on strategies for reading--may be essential components to success. Although such additions may be costly and time intensive for the teaching staff, many teachers find this type of activity a rewarding part of their jobs. Future scale-up research could continue to examine the issue of varied types of teacher and parent support components that Kim included (Kim 2006, 2007; Kim and Guryan 2010; Kim and White 2008). Allington (2010) found that when students were provided books over a period of three summers, even without any additional support components, student reading significantly improved. Therefore, it may be that teacher and parent support components are necessary for a summer reading program to be effective during a single summer, but may be less important if students participate in summer reading programs over a longer time period. Appended are: (1) Description of the Lexile Framework[R] for Reading; (2) Findings from previous studies of summer reading programs; (3) Student interest survey, explanatory letter, postcard, and summer reading survey; (4) Power analysis; (5) Texas Assessment of Knowledge and Skills-Lexile linking study; (6) Recruitment and study sample details; (7) Participating district profiles; (8) Description of the grade 3 Texas Assessment of Knowledge and Skills-Reading; (9) Random assignment; (10) Missing Data; (11) Summer reading survey results; (12) Models used for primary, sensitivity, and exploratory analyses; and (13) Tables of analytic output. (Contains 44 tables, 3 figures and 48 footnotes.)