Evaluation of &quot;System 44.&quot; Final Report [2012]
Author: Beam, Margaret
Date: 2012-09-00
Type: Reports - Research
Subjects: Reading Programs,Reading Difficulties,Phonics,Decoding (Reading),Educational Technology,Reading Instruction,Elementary School Students,Middle School Students,Grade 4,Grade 5,Grade 6,Grade 7,Grade 8,Program Effectiveness,Computer Assisted Instruction,Program Implementation,Context Effect,Teacher Attitudes,Accuracy,Reading Fluency,Reading Comprehension,English Language Learners,Students with Disabilities,Low Income Students,Minority Group Students,Achievement Tests,Reading Tests,Elementary School Teachers,Middle School Teachers,Urban Schools,Learning Disabilities,Gender Differences

The purpose of this evaluation of Scholastic's &quot;System 44&quot; conducted by RMC Research was to expand the existing research on students with learning disabilities by conducting a randomized study of struggling readers with approximately half of the sample comprised of students with learning disabilities. Specifically, this evaluation examined the impact of &quot;System 44&quot; on the reading outcomes of struggling readers and on a subsample of students with learning disabilities in Grades 4-8. The evaluation of the implementation and impact of &quot;System 44&quot; involved 12 elementary schools and 4 middle and K-8 schools in a district in Michigan. Scholastic's &quot;System 44&quot; is a foundational reading program intended for older struggling readers who have not mastered basic phonics and decoding skills. Combining researched-based phonics instruction with adaptive technology, &quot;System 44&quot; is designed to improve students' word reading accuracy, fluency, and comprehension. The &quot;System 44&quot; program delivers research-based instruction through an adaptive computer component; teacher-led small-group instruction; and individual student practice involving high-interest, leveled materials. Thus students who have not responded to classroom reading instruction may benefit from the more intensive and specific decoding instruction provided through &quot;System 44.&quot; The evaluators selected the target sample based on student performance on the fall 2011 Michigan Educational Assessment Program (MEAP) and spring 2011 AIMSweb assessment. The Scholastic Reading Inventory (SRI) was used to screen students for &quot;System 44&quot; eligibility. The district administered the SRI to all students in the target sample. Those students who scored below 600 Lexiles on the SRI were administered the Scholastic Phonics Inventory (SPI). All students who scored in the Beginning or Developing reader categories on the SPI were randomly assigned (stratified by school and grade level) to either the &quot;System 44&quot; treatment group or the control group. RMC Research hired and trained 4 local testers to individually administer a battery of standardized reading tests to all treatment and control group students. The testers administered the tests in October 2011 to establish baseline scores and again in May 2012 to attain follow-up scores. The tests included the following: (1) Test of Silent Reading Efficiency and Comprehension (TOSREC); (2) Comprehensive Test of Phonological Processing (CTOPP) Elision subtest; (3) Test of Word Reading Efficiency (TOWRE) Sight Word Efficiency subtest; and (4) Test of Word Reading Efficiency (TOWRE) Phonetic Decoding Efficiency subtest. The evaluation of &quot;System 44&quot; revealed significant impacts on several tests for both the overall sample and the learning disabled sample. Additional findings revealed that impacts were stronger on several tests for middle school students than for elementary school students, particularly on SPI Nonsense Word Accuracy, TOSREC, and SRI. Although significant impacts were attained by the end of Year 1, the majority of students in the study did not complete the &quot;System 44&quot; program. Data collected through teacher surveys, classroom visits, and interviews provided information on teachers' implementation of &quot;System 44&quot; in the classroom, and software usage data were used to examine differences in students with varying program exit and topic completion patterns. [For the November 2011 report, see ED613693.]