Impact Evaluation of Mathematics &quot;i-Ready&quot; for Striving Learners Using 2018-19 Data. Final Report. No. 048
Author: Randel, Bruce
Date: 2020-12-10
Type: Reports - Research
Subjects: Program Effectiveness,Grade 2,Grade 3,Grade 4,Grade 5,Elementary School Students,Program Implementation,Intervention,Fidelity,African American Students,Hispanic American Students,Racial Differences,Ethnicity,Achievement Gains,Educational Technology,Technology Uses in Education,Public Schools,Student Needs,Student Evaluation,Computer Assisted Testing,Mathematics Instruction,Learning Problems,Mathematics Achievement

Curriculum Associates' &quot;i-Ready® Personalized Instruction&quot; (&quot;i-Ready&quot;) is a supplemental, online personalized instruction program available for mathematics and reading. Prior research has indicated &quot;i-Ready&quot; has a positive impact on student achievement for students overall (e.g., Swain, Randel, &amp; Norman Dvorak, 2020). The present study furthers that work by examining the impacts of &quot;i-Ready&quot; for striving learners specifically, to provide schools and districts with more targeted information on its effectiveness for these struggling students. The Human Resources Research Organization (HumRRO), in collaboration with Century Analytics, implemented a quasi-experimental design (QED) using academic year 2018-19 &quot;i-Ready&quot; data to evaluate the impact of &quot;i-Ready&quot; mathematics Instruction on student mathematics achievement for striving learners in grades 2-5 on a nationally normed cognitive assessment. Two populations of striving learners were examined at each grade -- those who tested two or more grade levels below their current grade in mathematics at baseline and a subset of these students who fell at the bottom 20th percentile of mathematics achievement. The percentiles were based on mathematics achievement measured by the &quot;i-Ready® Diagnostic&quot; (Diagnostic) at baseline. It was hypothesized student achievement, as measured by the Diagnostic, would be higher for striving learners using &quot;i-Ready&quot; for mathematics over comparison groups of students who did not use this instruction. Exploratory analyses examined whether the findings were consistent for Black or African American striving learners and those of Hispanic origin. Matching was conducted at each grade level to meet two needs: 1) identify a set of comparison schools demographically similar to our &quot;i-Ready&quot; schools, and 2) identify a set of academically equivalent comparison students within the matched comparison schools. Students who received &quot;i-Ready&quot; and students in the comparison group took the mathematics version of the Diagnostic assessment. To estimate impacts, hierarchical-linear modeling (HLM) was conducted separately for each grade level with students at level 1 and schools at level 2. This process was conducted for the full sample of striving learners and again for the subsample of students at the bottom 20th percentile. Results suggest both the striving learners and students at the bottom 20th percentile using &quot;i-Ready&quot; with fidelity in the treatment schools performed statistically significantly better on mathematics than students in the comparison schools who did not use this instruction. The effect sizes for striving learners and the subset of the bottom 20th percentile students at grades 3 and 5 fell within the range which recent research characterizes as modest for an education intervention (Kraft, 2019). The effect sizes for striving learners and the bottom 20th percentile students at grades 2 and 4 fell above this range. These findings provide support that &quot;i-Ready&quot; for mathematics used with fidelity in schools can lead to higher mathematics achievement for striving learners. Exploratory analyses found that these impacts were consistent for the Black or African American striving learners and for the striving learners of Hispanic origin at grades 2, 3, and 5. A positive Hispanic origin by treatment group interaction was present at grade 4, indicating &quot;i-Ready&quot; had greater impacts on mathematics achievement of striving learners of Hispanic origin as compared to striving learners not of Hispanic origin who used &quot;i-Ready.&quot;