An Early Look at the Effects of Success Academy Charter Schools
Author: Unterman, Rebecca
Date: 2017-08-00
Type: Reports - Evaluative
Subjects: Charter Schools,Urban Schools,Success,Academic Achievement,Program Effectiveness,Elementary School Students,Selective Admission,Competitive Selection,Admission (School),Reading Achievement,Mathematics Achievement,Kindergarten,Grade 1,Grade 3,Grade 4,At Risk Students,Low Income Groups,Minority Group Students,Tests,Scores,Access to Education

Success Academy is a rapidly expanding charter school network in New York City, with schools located in the Bronx, Brooklyn, Manhattan, and Queens. In the 2016-2017 school year, Success Academy served roughly 14,000 students across 41 elementary, middle, and high schools, which at the time was about 13 percent of the students attending charter schools in the city and equivalent to about 1.2 percent of all New York City Department of Education (NYCDOE) students. In this paper, the Manpower Demonstration Research Corporation (MDRC) looks back to the beginning of Success Academy and uses naturally occurring lotteries for the seven Success Academy schools in operation during the 2010-2011 school year to estimate the difference in students' academic achievement caused by the opportunity to enroll in a Success Academy school. This work is part of an independent analysis that MDRC conducted for Success Academy's 2015 Investing in Innovation (i3) federal grant application. The educational outcomes of elementary students who received placements in their local Success Academy schools by being selected in lotteries are compared with the outcomes of students who entered these lotteries but were not selected for their local Success Academy schools. Because not all students selected in lotteries enrolled in Success Academy schools and some of those not selected through lotteries for their local (or closest) Success Academy school ended up enrolling in a different Success Academy school, MDRC also estimates the effects of enrolling in a Success Academy school (for the lottery sample). A statistical adjustment makes it possible to compare those who won their lotteries and did enroll in Success Academy schools with those who lost their lotteries and did not enroll in Success Academy schools. The preliminary analysis in this paper focuses on the 4,710 students (3,804 students entering kindergarten and 906 entering first grade) who applied to oversubscribed Success Academy schools in the spring of 2010 for the 2010-2011 school year and who therefore participated in lotteries. Only applicants from this round of admissions could be followed long enough to measure their academic achievement using New York State's third- and fourth-grade mathematics and reading exams at the time the analysis was conducted. While 4,710 students is a relatively large sample, because the number of applicants greatly exceeded the number of seats available for the 2010-2011 school year, the sample only includes about 700 students who were offered admission. The resulting low ratio of lottery &quot;winners&quot; to all applicants limits the statistical precision of the estimates presented. Within the constraints of this analysis, MDRC finds that for the subset of students who were selected through a lottery and then enrolled in a Success Academy school, enrollment had large, positive effects on students' math achievement in third grade, roughly equivalent in magnitude to between one and one and a half typical years of learning for this age group. Effects on reading did not reach conventional levels of statistical significance but are positive and promising. A larger sample size would be necessary to obtain more precise estimates. As explained in this report, the program's effect was also estimated for all lottery winners, not just those who enrolled, and these estimates were similar but of a smaller magnitude. The remainder of this paper explains these findings in more detail. Appendix A describes how the lotteries occur within the Success Academy admission process and how the analytic models are estimated. [Success Academy Charter Schools funded this work as part of its 2015 Investing in Innovation grant application.]