Evaluation of &quot;System 44.&quot; Final Report [2011]
Author: Beam, Margaret
Date: 2011-11-00
Type: Reports - Research
Subjects: Reading Programs,Reading Difficulties,Phonics,Decoding (Reading),Educational Technology,Reading Instruction,Elementary School Students,Middle School Students,Grade 4,Grade 5,Grade 6,Grade 7,Grade 8,Program Effectiveness,Computer Assisted Instruction,Program Implementation,Suburban Schools,Context Effect,Teacher Attitudes,Accuracy,Reading Fluency,Reading Comprehension,English Language Learners,Students with Disabilities,Low Income Students,Minority Group Students,Achievement Tests,Reading Tests,Elementary School Teachers,Middle School Teachers

Scholastic's &quot;System 44&quot; is a foundational reading program intended for older struggling readers who have not mastered basic phonics and decoding skills. Combining researched-based phonics instruction with adaptive technology, &quot;System 44&quot; is designed to improve students' word reading accuracy, fluency, and comprehension. The &quot;System 44&quot; program delivers research-based instruction through an adaptive computer component; teacher-led small group instruction; and individual student practice involving high-interest, leveled materials. Thus students who have not responded to classroom reading instruction may benefit from the more intensive and specific decoding instruction provided through &quot;System 44.&quot; Using a randomized design, this evaluation assessed the effectiveness of &quot;System 44&quot; in terms of improving the foundational reading skills of struggling readers in Grades 4-8 in a large suburban school district in southern California during the 2010-2011 school year. The evaluation of the implementation and impact of &quot;System 44&quot; involved 7 of the 11 elementary schools and all 4 middle schools in the district. A 2-step process was used to establish student eligibility for &quot;System 44.&quot; The Scholastic Reading Inventory (SRI) was used to screen students in Grades 4-8 who performed below the 50th percentile on the spring 2010 California Standards Test (CST) for &quot;System 44&quot; eligibility. Those students who scored below 600 Lexiles on the SRI were administered the Scholastic Phonics Inventory (SPI), a computer-based test used to identify students in need of additional phonics instruction. Students who scored in the Beginning or Developing reader categories on the SPI were randomly assigned (stratified by school and grade level) to either the &quot;System 44&quot; treatment group or the control group. Data collection activities for the &quot;System 44&quot; evaluation included student reading tests, teacher surveys, &quot;System 44&quot; classroom observations, a professional development observation, and staff interviews. RMC Research hired and trained 4 local testers to administer a battery of standardized reading tests to all treatment and control students. The testers administered the tests to each student separately over a 3-week period in September and October 2010 to establish baseline scores and again in May 2011 to attain follow-up scores. Listed in order of administration, the tests included the following: (1) Test of Silent Reading Efficiency and Comprehension (TOSREC); (2) Comprehensive Test of Phonological Processing (CTOPP) Elision subtest; (3) Woodcock-Johnson III Word Identification subtest; (4) Woodcock-Johnson III Word Attack subtest; (5) Test of Word Reading Efficiency (TOWRE) Sight Word Efficiency subtest; and (6) Test of Word Reading Efficiency (TOWRE) Phonetic Decoding Efficiency subtest. This report details the program impact findings and concludes with recommendations from the evaluation team.