Impacts of Title I Supplemental Educational Services on Student Achievement. NCEE 2012-4053
Author: Deke, John
Date: 2012-05-00
Type: Numerical/Quantitative Data
Subjects: Supplementary Education,Academic Achievement,Outcomes of Education,Elementary School Students,Middle School Students,School Districts,Tutoring,Reading Instruction,Mathematics Instruction,Low Income Groups,Economically Disadvantaged,Eligibility,Federal Legislation,Educational Legislation,Program Effectiveness,Scores

This report presents the findings of an evaluation sponsored by the Institute of Education Sciences (IES) at the U.S. Department of Education (ED) and conducted by Mathematica Policy Research (Mathematica) that uses a regression discontinuity (RD) design to assess the potential benefits of offering SES in districts that have unmet need. Specifically, the study focuses on six school districts in which more eligible students applied for SES than could be served with available funds (i.e., oversubscribed districts), and which therefore allocated scarce SES spaces by giving priority to lower-achieving students among the eligible applicants. The answers to the key questions addressed in the study are as follows: (1) &quot;What is the average impact of offering SES to eligible applicants who are on the cusp of having access to services, in school districts where services are oversubscribed?&quot; Across the six oversubscribed districts, the authors find no evidence of impacts of offering SES to students near the cut point for an offer. For students in these oversubscribed districts in grades 3-8 at the cusp of receiving an offer of services, they find no statistically significant impact of &quot;offering&quot; SES on student achievement in reading or in mathematics. The point estimate of the average impact on reading is -0.03 standard deviations, and that for mathematics is 0.05 standard deviations. Furthermore, there is no evidence of potential benefits for at-risk subgroups of students. Similarly, they find no statistically significant impact of &quot;participating&quot; in SES on student achievement in reading or mathematics. The estimated impact of participating (which involved an average of 21 hours of services) is -0.10 standard deviations for reading and 0.11 standard deviations for math (again estimated for students in grades 3-8 near the cutoff for an offer, in these oversubscribed districts); (2) &quot;What are the characteristics of SES provided to students in oversubscribed districts?&quot; Across districts participating in this study, services averaged 21.2 hours per student for the school year (standard deviation of 8.8), with over a third of the students (36 percent) receiving tutoring in both reading and math, 55 percent receiving tutoring in only reading, and 9 percent receiving tutoring in only math. For students receiving reading services, the mean was 17.2 hours of tutoring (standard deviation of 9.2). For students receiving math services, the mean was 12.5 hours of tutoring (standard deviation of 8.2). In the study districts, most providers (70 percent) were for-profit firms. On average, 60 percent of providers' instructional staff were regular schoolteachers working in the local district. Most providers (64 percent) offered services at the schools of their students. Providers reported that group sizes of 2-5 students were most frequently used, with most other sessions provided individually (in one-on-one sessions). On average, 44 percent of provider services were in groups of 2-5 students, 34 percent in one-on-one sessions, and 21 percent in groups of 6-10; and (3) &quot;Are the characteristics of Supplemental Educational Services, providers, or practices in host school districts correlated with the estimated impacts?&quot; There is variation across providers in the average number of hours of math and reading services received by students, with average hours in services focused on math ranging from 0 to 27 across providers and average hours in services focused on reading ranging from 0 to 43. However, the intensity of services is not significantly related to the estimated size of impacts on math or reading act (for these eligible students near the cutoff for having been offered SES). They also found no evidence that any other observed provider characteristics were significantly associated with stronger impacts. Appended are: (1) SES Assignment and Participation; (2) Estimation Methods; (3) Diagnostic Analyses; (4) Exploratory Analyses; (5) Assessing Robustness of Impact Estimates; (6) Impact Estimates and Graphical Analyses by Mini-Study; (7) SES Survey Data Collection Methods; and (8) Supplemental Materials to Chapter III: SES Provider Characteristics and Students' SES Experiences. (Contains 57 tables, 72 figures and 30 footnotes.) [For &quot;Impacts of Title I Supplemental Educational Services on Student Achievement. Executive Summary. NCEE 2012-4054,&quot; see ED532017.]