Schools to Watch: School Transformation Network, a U.S. Deparment of Education Investing in Innovation (i3) Development Grant. Final Evaluation Report
Author: University of Illinois, Center for Prevention Research and Development (CPRD)
Date: 2015-09-00
Type: Reports - Research
Subjects: Educational Change,Educational Improvement,Quasiexperimental Design,Program Effectiveness,Academic Achievement,Control Groups,Experimental Groups,Middle Schools,At Risk Students,Low Achievement,Program Evaluation,Formative Evaluation,Longitudinal Studies,Intervention,Mathematics Achievement,Reading Achievement,Language Arts,Scores,Comparative Analysis,Grade 8,Scoring Rubrics,Surveys,Focus Groups,School Culture,Educational Environment,Cooperation,Teamwork,Leadership Responsibility,Change Strategies,Teaching Methods,Coaching (Performance),Best Practices

The Schools to Watch: School Transformation Network Project is a whole school reform model designed to improve the educational practices, experiences, and outcomes of low-performing middle-grades schools. Developed by the National Forum to Accelerate Middle-Grades Reform, the four-year project was funded in 2010 by a U.S. Department of Education Investing in Innovation (i3) development grant. The purpose of the study was to examine the impact of the project on intermediate outcomes such as culture, collaboration, and instructional practices as well as the long term outcome of student achievement. The study employed a quasi-experimental design where two student cohorts were tracked over four years at 34 schools (17 intervention and 17 comparison) in three states. The intervention schools were comprised of persistently low-performing middle-grades schools serving high need students. Comparison schools were selected using key demographics to match to intervention schools. Several process and measurement tools for assessing implementation and intermediate outcomes were used, including surveys, the STW criteria rating rubric, coach's logs, and focus groups. The long term outcome data for the impact study included student English and math achievement scores on annual standardized state assessments. To examine achievement scores between intervention and comparison students, a series of 2-level models (students within schools) were run to assess 8th grade achievement (i.e., after students received all three years of the intervention). Results showed that i3 STW Project schools improved their culture and climate, collaboration practices, leadership practices, STW criteria implementation, and classroom instructional practices. There was no overall intervention effect on either English or math student achievement, however, significant results were found for the highest implemented schools, those project schools that achieved STW designation during the project. The results of the study provide unique insight into the reform process for i3 STW Project schools as well as other middle-grades schools that are struggling to improve. The multiple supports provided by the project combined with the guiding vision of the STW criteria and rubric supported these high need schools to improve contextual factors (i.e., culture, collaboration, leadership, teaching and learning practices), and for a subsample of schools, student achievement. Districts and schools embarking on reform need to focus on collaborative leadership, have a guiding vision, use a continuous improvement model for instructional improvements, and value networking with other schools to gain knowledge. Two appendices are included: (1) Psychometric Properties of the Self-Study Survey Constructs; and (2) Fidelity Matrix for National Forum's STW School Transformation Network Project.