Annual Evaluation Report for the Pennsylvania Dyslexia Screening and Early Literacy Intervention Pilot Program Pilot Year 2, 2016-17 School Year
Author: Kuchle, Laura
Date: 2018-01-00
Type: Reports - Research
Subjects: Dyslexia,Screening Tests,Emergent Literacy,Early Intervention,Pilot Projects,Kindergarten,Grade 1,Reading Instruction,Program Effectiveness,Elementary School Students,Reading Fluency,Reading Tests,Comparative Analysis,Regression (Statistics),Public Schools,Quasiexperimental Design,Scores,Reading Skills,Intelligence Tests,Vocabulary,Verbal Ability,Program Implementation,Fidelity,Institutional Characteristics,Student Characteristics,Faculty Development,Statistical Analysis

The 3-year Pennsylvania Dyslexia Screening and Early Literacy Intervention Pilot Program (Pilot) began in 2015-16 (Year 1) with the kindergarten class of 2015-16 (Cohort 1). In 2016-17 (Year 2), the Pilot was implemented with Cohort 1 students, now in first grade, and a second cohort of kindergarteners (Cohort 2). A third cohort will be added in 2017-18. The Pilot provides two levels of support: (1) a classroom program, which supplements core instruction for all students, with an increased focus on phonemic awareness and multisensory structured language (MSL), and (2) an MSL intervention to provide extra instruction for students identified as needing more support based on early literacy screening in the winter of kindergarten. Both levels of support are meant to affect special education referrals and students' literacy skills, measured by the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) Next benchmark assessments (only DIBELS data are available at this time). This report presents key findings from Year 2. The effectiveness of the classroom program was evaluated using a school-level matched design, in which the performance of students in the 21 Pilot schools was compared with the performance of students in 21 matched comparison (Comparison) schools identified through Mahalanobis distance matching. These analyses suggest that both Pilot cohorts (21 schools) outperformed the Comparison sample on some spring 2017 (Year 2) measures (nonsense word fluency for both cohorts and letter naming fluency for Cohort 2). This may be because of improved implementation in Year 2, and is particularly encouraging given the Comparison sample's participation in another literacy initiative, which may result in an underestimation of Pilot program effects compared with typical schools (which may not use universal screening to inform core instruction and identify students to receive supplemental intervention). The effectiveness of the MSL intervention was assessed using a regression discontinuity (RD) design, in which Pilot students eligible for the intervention were compared with similarly performing students in the same schools who were not eligible for the intervention. Although these analyses yielded no positive effects, exploratory analyses suggest that the intervention may have contributed to improved performance for Cohort 1 Pilot intervention students compared to similar Comparison students. Exploratory analyses also found an association between intervention hours and outcomes, suggesting increased dosage might yield stronger intervention effects (most schools did not meet target hours). These findings suggest that the combination of enhanced core instruction and supplemental MSL intervention improve some student outcomes in school settings, warranting further study. The Pilot's final evaluation report will cover the third year of Pilot implementation, allowing comparisons across three cohorts and expanding the Pilot to second grade (Cohort 1), and consider other variables such as special education status. This report includes numerous exhibits to explain implementation, samples, and findings, and includes the following appendices: (1) Study Design; (2) Matching to Establish Comparison Sample; (3) Supplementary Implementation Information; (4) Comparisons of Analysis Samples; (5) Sample Parent Notification and Opt-Out Template Provided by PDE to Pilot Districts; (6) Technical Information.