**[Striving Readers]{.smallcaps}**

**[Implementation of the]{.smallcaps}**

**[Targeted and the Whole School Interventions]{.smallcaps}**

**[Summary of Year 1 (2006-07)]{.smallcaps}**

**[memphis City Schools]{.smallcaps}**

**[July 2008]{.smallcaps}**

**[Authors:]{.smallcaps}**

**[Debra Coffey, M.A, Research for Better Schools]{.smallcaps}**

**[Kelly Feighan, M.A., Research for Better Schools]{.smallcaps}**

This report was submitted to the U.S. Department of Education, Office of
Elementary and Secondary Education to fulfill requirements of the
Striving Readers grant, Marcia Kingman, Program Officer.

> **Table of Contents**
>
> **Description of the Memphis Striving Readers Project Targeted
> Intervention \...\...\...\...\...\...\.... 1**
>
> **Logic Model for the Targeted Intervention
> \...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...
> 2**
>
> **Design of the Year 1 Targeted Intervention Implementation Study
> \...\...\...\...\...\...\...\...\...\...\..... 4**
>
> **Rating the Levels of Implementation
> \...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...
> 4**
>
> **Professional Development Levels
> \...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...
> 4**
>
> **Levels of Implementation and Variability of Classroom Instruction
> \...\...\...\...\...\...\...\...\..... 6**
>
> **Conclusions Regarding Implementation of the Targeted Intervention
> \...\...\...\...\...\...\...\...\..... 8**
>
> **Description of the Memphis Striving Readers Project Whole-School
> Intervention\...\...\...\..... 9**
>
> **Logic Model for the Whole-School Intervention
> \...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...
> 9**
>
> **Design of the Year 1 Implementation Study for the Whole-School
> Intervention \...\...\...\... 10**
>
> **Data Sources Linked to Research Questions for the Whole-School
> Intervention \..... 11**
>
> **Summary of Levels of Implementation Attained for the Whole-School
> Intervention\..... 13**
>
> **Conclusions Regarding Implementation of the Whole-School
> Intervention \...\...\...\...\...\... 15**
>
> **References
> \...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...
> 15**
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page i
>
> **EXECUTIVE SUMMARY FOR THE MEMPHIS STRIVING READERS PROJECT YEAR 1
> IMPLEMENTATION STUDY**
>
> The United States Department of Education awarded the Memphis City
> Schools (MCS) district a five-year Striving Readers grant to help
> address the city's significant educational needs. MCS, which serves
> more than 116,000 students, is the 21st largest K12 district in the
> United States. Over 95 percent of the 196 MCS schools are Title I
> schools, and 71 percent of students qualify for free or reduced-price
> lunches (The Urban Child Institute, 2008). According to the Memphis
> City Schools web site, approximately 86 percent of MCS students are
> African American, 8 percent are white, and 6 percent are other races
> and/or ethnicities (second bullet,
>
> "MCS Quick Facts: Students," n.d.).
>
> According to Ippolito, Steele, and Samson (2008), "reading and writing
> proficiency are critical determinants of students' overall success in
> school" (p. 2). MCS statistics underscore this statement: In 2005, the
> latest year for which figures are available, 69.2 percent of MCS
> students graduated high school within four years (Hart, 2008).
> Seventy-one percent of middle school students (grades 6 through 8)
> scored below the 50th percentile on the Reading/Language Arts portion
> of the state's assessment, the Tennessee Comprehensive Assessment
> Program (TCAP)
>
> (Potts, Perkins, Heeren, Harris, & Feldman, 2008).
>
> The Memphis Striving Readers Project (MSRP) consists of a whole-school
> intervention and a targeted intervention, both of which are
> experimental designs that are being implemented in eight MCS middle
> schools. During its first year (2006--07), the project involved a
> total of 5,895 students in the whole-school program, while the
> targeted intervention served 707 students who were randomly assigned
> to the supplemental intervention.1 The whole-school intervention
> includes a literacy-focused professional development program for
> teachers in core content areas and coaching support for literacy
> activities in their classrooms (further details about the whole-
> school intervention are provided in the "Description of the
> Whole-School Intervention" section). The targeted intervention is a
> commercially available reading intervention program that is being
> provided to students who have demonstrated the strongest need for
> reading support, i.e., they scored in the bottom 25 percent on the
> reading portion of the TCAP.
>
> ***Description of the Memphis Striving Readers Project Targeted
> Intervention***
>
> The targeted intervention is a randomized control trial that tests the
> effectiveness of *READ*
>
> *180,* a commercially available reading intervention program from
> Scholastic. The program targets struggling readers in the fourth
> through twelfth grades. *READ 180* combines a software program,
> teacher-directed instruction using a textbook and similar resources,
> and independent or modeled reading (i.e., reading while listening to
> audiobooks) during daily 90-minute class
>
> 1 Data sources: For the targeted intervention enrollment, the
> "Enrollment file 06-14-07 SR Schools Only" data file provided by MCS.
> For the school populations, data were downloaded on July 8, 2008, from
> http://www.memphis-
> schools.k12.tn.us/admin/communications/directoryofschools.html
>
> periods. Teachers who teach *READ 180* are expected to attend two
> whole-day trainings
>
> (experienced teachers may attend only one new day of training) and are
> asked to attend six networking meetings and to complete six online
> training modules provided by Scholastic. In MCS*, READ 180* is a
> supplemental program---it is offered in addition to the regular
>
> English language arts courses that all students receive---and targets
> struggling readers, who are enrolled in *READ 180* classes for up to
> two years. For the Striving Readers study, these students are defined
> as those who scored in the bottom 25 percent of the TCAP. All students
> eligible for *READ 180* according to this standard were assigned
> randomly to the experimental condition
>
> (those who were enrolled in *READ 180*) or control condition (those
> who were enrolled only in the "business as usual" language arts and/or
> reading courses). During Year 1, 707 students
>
> (ranging from 58 to 100 at each school) were assigned to the
> experimental condition and were enrolled in *READ 180,* while 1,333
> students were assigned to the control condition*.*
>
> **Logic Model for the Targeted Intervention**
>
> The logic model for the targeted intervention, as published in
> Scholastic's *READ 180*
>
> *Enterprise Edition Research Protocol and Tools* (2007), appears as
> Figure 1. Additions or clarifications that have been added for this
> project appear in blue italic or blue underlined text. Because this
> logic model was created by Scholastic, it includes several outcomes
> (such as those related to behavior and attendance) that are not being
> or tracked or measured in the Memphis Striving Readers Project (MSRP).
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 2 of 15
>
> *Figure 1: Logic Model of Targeted Intervention*
>
> **The *READ 180* Enterprise Edition (EE) Logic Model**
>
> **Resources Ongoing Teaching/**
>
> **Learning Activities**

**Short-Term Outcomes Long-Term Outcomes**

> 1\) Teacher training and professional development
>
> 2\) Scholastic Professional Development for administrators and
> technical directors
>
> 3\) *rBook* Teacher's Edition and
>
> *rBook* student work text
>
> 4\) Networked computers with microphones and headsets, teacher
> workstation, and printer
>
> 5\) CD players with headphones
>
> 6\) TV with DVD player
>
> 7\) *READ 180* EE topic software, audiobooks, paperbacks, and Anchor
> Videos
>
> 8\) Classroom space adequate for
>
> *READ 180* instruction
>
> 9\) Scholastic Achievement
>
> Manager (SAM)

A daily 90-minute instructional block

20-minutes Whole-Group Instruction to start the class

Small-group rotations during which students are divided into 3 groups
and spend 20 minutes each rotating through

> •Small-Group Instruction
>
> •Modeled and Independent Reading

•Use of *READ 180* EE Topic Software Teachers regularly use *READ 180*
instructional strategies and materials contained in *READ 180* program
guides, [which include (but are not limited to)]{.underline}

[independent reading of leveled texts, use of]{.underline}

[graphic organizers, and specific teaching of]{.underline}

[vocabulary.]{.underline}

10 minutes of Whole-Group Wrap-Up to conclude the class

Enrollment of 15-18 students per class *\[or up to 21\]*

Enrollment lasts the entire school year *\[for two years\]*

Instruction follows *rBook* scope and sequence Teachers and
administrators regularly use diagnostic tests (SRI) and Scholastic
Management Suite for continuous assessment, placement, and monitoring

Improved classroom behavior and school attendance, and decreased
disciplinary incidents

Increased motivation and engage- ment in reading

Increased reading proficiency as reflected in SRI scores and other
indicators monitored by SAM

Improved state and local assessment results *\[at least 50% of the READ
180 program students will score "proficient" on TCAP reading, language
arts, and other subject area subtests, and those students will make a
mean gain on reading and other subject area subtests of at least 5--10
NCEs over control students\]*

Improved learning in all subject areas

> Contextual effects such as the characteristics of the school district,
> other instructional programs in use, and external events may also
> influence outcomes. The high-fidelity classroom model includes student
> involvement/engagement with classroom activities.
>
> Logic Model copyright © 2007 by Scholastic Inc.; [additions in blue
> and underlined were added for clarity by RBS]{.underline};
>
> *additions in blue italic are specific to the Memphis Striving Readers
> Project.*
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 3 of 175
>
> **Design of the Year 1 Targeted Intervention Implementation Study**
>
> Because this was considered an effectiveness study rather than an
> efficacy study, a plan to study implementation of the targeted
> intervention was not developed for Year 1. We present in Figure 2
> (page 5) the data that are available from Year 1 and indicate when
> additional data will be available for future years of the study. All
> data are and will be available at the teacher/ classroom level, but
> not at the individual class period level.
>
> **Rating the Levels of Implementation**
>
> As noted in Figure 2, the sources of data for rating the
> implementation fidelity of *READ 180* were teacher surveys, classroom
> observations, data generated by SAM, and documents related to
> professional development. Findings from all of these sources were
> translated to a 4-point scale ranging from 0 to 3. For all ratings,
> "adequate" is defined at 2 or above---the "good" or
>
> "excellent" level. The "Professional Development Scales" and "Levels
> of Implementation and Variability of Classroom Instruction" sections
> each include more detailed descriptions of the specific data sources
> used for those areas.
>
> Table 1 provides the scale and indicates in gray those that are
> considered "adequate." Table 2
>
> (page 8) provides the score for each area of measurement and the
> overall rating for each teacher/classroom.
>
> *Table 1: Scale for Levels of Teacher/Classroom Preparation and
> Implementation Fidelity*
>
> **Scale score Description**
>
> 0 Poor
>
> 1 Moderate
>
> 2 Good
>
> 3 Excellent
>
> ***Professional Development Levels***
>
> The professional development participation score was developed by
> adding together the points assigned to different types of professional
> development. There were four types of professional development: (1)
> attendance at each (of three) all-day session earned a 2,
>
> \(2\) attendance at each (of six) networking meeting earned a 1, and
> (3) each year of experience teaching *READ 180* earned a 2 (up to a
> maximum of 6). This resulted in a possible total of 18 points. (The
> Year 2 scale will add points based on whether teachers completed
> Scholastic's online training course.) Evaluators, in consultation with
> MCS staff members, determined that professional development scale
> scores greater than or equal to ten would be considered
>
> "excellent," those between seven and nine would be considered "good,"
> those between four and six would be "moderate," and scores three and
> below would be "poor." These ratings are included in the second column
> of Table 2 (page 8).
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 4 of 15
>
> *Figure 2: Year 1 Data Sources on Implementation Linked with Research
> Questions---Targeted*
>
> *Intervention*
>
> **[Research Questions]{.underline} Measures/Data Sources**
>
> Surveys SAM\* Observations Record review
>
> **[What was the level of implementation and variability of
> professional development for teachers in]{.underline}**
>
> **[Year 1?]{.underline}**
>
> Types/amount of professional development
>
> provided to teachers Y1 (Y2)
>
> Proportion of teachers at different levels of
>
> professional development Y1
>
> Proportion of teachers at adequate level of
>
> professional development Y1
>
> Types/amount of professional development
>
> provided to district leaders (Y2) (Y2)
>
> Proportion of leaders at different levels of
>
> professional development (Y2) (Y2)
>
> **[What was the level of implementation and variability of classroom
> instruction in Year 1?]{.underline}**
>
> Proportion of teachers with access to
>
> materials and resources, technology Y1 Y1 (Y2) (Y2) Y1
>
> Proportion of teachers who implemented the
>
> classroom instruction model at different levels
>
> Proportion of teachers who implemented the

Y1 Y1 Y1 (Y2) (Y2) Y1

> classroom model at adequate level Y1 Y1 Y1 (Y2) (Y2) Y1
>
> **[What did the counterfactual (for targeted intervention) look like
> in Year 1?]{.underline}**
>
> Experiences of the control students parallel
>
> to the interventions received by the treatment students2

(Y2) (Y2) (Y2)

> *\* SAM is the Scholastic Achievement Manager, a feature of the* READ
> 180 *program that automatically generates student-level data based on
> work students have done and assessments they have completed using*
> READ 180 *software.*
>
> 2 Because *READ 180* is being implemented as a *supplemental* program
> in MSRP, students in both the experimental and control groups
> experience the same English/language arts classes that they would have
> without MSRP; however, the experimental students *also* have the *READ
> 180* class. It is possible that some control students participated in
> ELA-related elective or exploratory classes (e.g., creative writing).
> This will be explored further in subsequent years; evaluators are
> working with MCS staff to determine the most appropriate relevant
> source of data.
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 5 of 15
>
> ***Levels of Implementation and Variability of Classroom
> Instruction***
>
> Implementation fidelity and variability were monitored through
> classroom observations, teacher surveys, and data generated by the
> Scholastic Achievement Manager (SAM), which tracks the progress of
> students and the use of *READ 180* tools by teachers. As this is an
> implementation report, not an impact report, the variables used
> focused on what was made available to or completed by students, not
> how well students completed tasks or assessments. For example, the
> Scholastic Reading Counts! (SRC) variable included is the number of
> quizzes that students took (which is a rough estimation of how many
> books a student read during the Independent Reading portion of the
> *READ 180* class) not how well they did on those quizzes.
>
> Evaluators first examined all included variables and created an
> equation for translating each survey, SAM, or observational variable
> to the 4-point scale. Second, an equation was created that encompassed
> the data from within each source of data (surveys, SAM, observations,
> and professional development). Finally, those scores were averaged to
> create the ratings that appear in Table 2. When data were missing,
> they were left out of the second (or within-source) and overall
> equations. For example, if the May observation did not happen, the
> February observation rating served as the average observation rating.
> If there was no survey linked with a specific teacher/classroom ID,
> the overall rating was calculated using only professional development,
> SAM, and observation data.
>
> Members of the evaluation team completed classroom observations during
> February and May of 2007. All 19 *READ 180* teachers were observed at
> least once; 11 teachers were observed during both February and May.
> The evaluation team completely re-created the observation protocol
> between February and May. If a teacher was observed twice, ratings
> from these two
>
> sets of observations were calculated separately and averaged; if a
> teacher was observed once, the rating is from just that observation.
> Observations focused on the extent to which teachers structured the
> class and the lessons observed according to the *READ 180* model and
> the extent to which classrooms had the resources and materials
> required for the program.
>
> The observation protocol used during February included a rubric that
> provided descriptions of different levels of implementation and asked
> observers to rate the environment and lessons presented. The ratings
> used for this report are these:
>
> Environment: schedule, i.e., to what extent the class followed the
> 90-minute model with 20 minutes of whole-group instruction, 20 minutes
> each of small-group instruction, computer use, and independent/guided
> reading, and a 10-minute wrap-up
>
> Environment: room arrangement, i.e., the extent to which the room and
> furniture are arranged appropriately for the *READ 180* program
> rotations
>
> Presentation, time, and content of whole-group instruction
>
> Presentation, time, and content of small-group instruction
>
> Use of the Scholastic *rBook* for instruction
>
> Use of instructional software and length of time software was used
>
> Engagement of students in independent reading
>
> Presentation, time, and content of whole-group wrap-up
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 6 of 15
>
> Items on the observation protocols used during May were more specific
> in the information required. For example, observers were asked to
> record the levels of engagement of students (or the extent to which
> the students were on task) during the different rotations of the
> class. The items used for the ratings in this report include the
> following:
>
> Time (in minutes) of different portions of class
>
> Presentation of whole-group instruction multiplied by student
> engagement in whole- group instruction
>
> Presentation of small-group instruction multiplied by student
> engagement in small- group instruction
>
> Use of individual computers multiplied by student engagement in
> computer rotation
>
> Engagement in independent reading
>
> Presentation and content of whole-group wrap-up multiplied by student
> engagement in whole-group wrap-up
>
> Number of students in class (was the number 21 or fewer, as specified
> by the model)
>
> Use of Scholastic books and materials
>
> Room space and arrangement of furniture
>
> Surveys were administered during the summer after Year 1; 14 teachers
> completed surveys
>
> (one additional teacher completed a survey but did not provide any way
> of identifying herself or her classroom). Survey questions focused on
> availability and use of specific products, equipment, and materials.
> (The survey also asked about the number of years a teacher had taught
> *READ 180* before Year 1; this was included in the professional
> development scale).
>
> Teacher use of Red Routines (Scholastic lesson plans for *READ 180*),
> SAM reports, and purposeful strategies for forming and re-forming
> small groups
>
> Month that CD player and *READ 180* teacher supplies were received
>
> Month that computers and software were received
>
> Frequency of availability of *working* computers, software, and other
> technology
>
> Month that *rBooks* were received
>
> SAM data were generated for all students in *READ 180*, so evaluators
> linked all students with their teachers and averaged together the data
> from all students taught by each teacher. The SAM variables used for
> this report included the following:
>
> Number of Scholastic Reading Inventory (SRI) assessments administered
>
> Number of *READ 180* computer software sessions
>
> Average daily number of minutes spent in these sessions
>
> Number of *rSkills* assessments
>
> Average number of minutes taken to complete them
>
> Number of SRC quizzes taken
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 7 of 15
>
> *Table 2: READ 180 Fidelity of Implementation Teacher/Classroom
> Ratings*
>
> Note: ID numbers were randomly generated and hold no meaning
>
> **Teacher/**
>
> **Classroom Prof. Observation(s) Ratings Survey SAM Overall**
>
> **ID Dev.Rating Feb. May Avg. Ratings Ratings Rating**
>
> 910 2 2 2 2.0 2.8 3 **2.45**
>
> 707 3 2 3 2.5 2.0 2 **2.38**
>
> 899 3 1 3 2.0 2.2 2 **2.30**
>
> 821 1 3 3 3.0 2.0 3 **2.25**
>
> 322 2 3 2 2.5 2.8 1 **2.08**
>
> 397 2 2 3 2.5 2.8 1 **2.08**
>
> 604 3 2 \* 2.0 2.2 1 **2.05**
>
> 848 2 3 1 2.0 2.0 2 **2.00**
>
> 628 2 1 \* 1.0 2.8 2 1.95
>
> 122 2 2 \* 2.0 2.0 1 1.75
>
> 221 2 2 1 1.5 1.8 1 1.58
>
> 694 2 1 \* 1.0 2.2 1 1.55
>
> 513 2 1 2 1.5 \* 1 1.50
>
> 727 0 \* 2 2.0 \* 2 1.33
>
> 242 0 1 \* 1.0 1.0 2 1.00
>
> 298 1 2 2 2.0 \* 0 1.00
>
> 516 2 1 \* 1.0 \* 0 1.00
>
> 380 1 2 0 1.0 1.6 0 0.90
>
> 895 0 2 \* 2.0 \* 0 0.67
>
> Number
>
> "Adequate" 13 12 9 12 11 8 **8**
>
> Percent
>
> "Adequate" 68.4 66.7 75.0 66.7 78.6 42.1 **42.1**
>
> *\* indicates that data were not gathered or not available*
>
> **Conclusions Regarding Implementation of the Targeted Intervention**
>
> An analysis of the *READ 180* program data gathered and obtained found
> wide variation in implementation across classrooms in the eight
> schools. Of the nineteen teachers, thirteen were rated adequate for
> professional development, and twelve teachers/classrooms were rated
> adequate based on classroom observations. More than three-quarters
> were rated adequate based on teacher surveys. Of the nineteen
> classrooms, eight were determined to reach the "adequate" level based
> on SAM data. Eight teachers/classrooms were rated adequate overall,
> and another
>
> two were very close to adequate, but the remaining eight were
> substantially below adequate. It is important to remember two points
> when considering these findings:
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 8 of 15
>
> \(1\) Some classrooms did not receive all the resources and materials
> necessary for fully implementing *READ 180* until December or January.
> This affected Year 1 findings but should not affect subsequent years.
>
> \(2\) There were only two rounds of observations completed during
> Year 1. In Year 2, three rounds of observations were completed by the
> evaluator, two were completed by MCS, and one was completed by
> Scholastic. This more comprehensive effort will result in more
> reliable data.

***Description of the Memphis Striving Readers Project Whole-School
Intervention*** The whole-school intervention, or Memphis Content
Literacy Academy (MCLA), was implemented during Year 1 in four middle
schools serving students in grades six through eight. The program was
designed to improve teachers' pedagogy and, ultimately, student academic
achievement through intensive professional development, on-site literacy
coaching assistance,

> and a leadership seminar. A team of university and school district
> staff envisioned implementing a rigorously designed research-based
> model that encouraged teachers to integrate literacy practices in the
> academic content areas English/language arts (ELA), mathematics,
> science, and social studies. They hypothesized that more effective and
> frequent integration of literacy strategies by teachers would lead to
> student performance improvements in reading and core academic content
> classes.
>
> **Logic Model for the Whole-School Intervention**
>
> The logic model below links the intervention's primary activities,
> such as professional development for teachers and on-site coaching
> assistance, with desired program outcomes, including improved student
> performance on the Tennessee Comprehensive Assessment Program
>
> (TCAP) and Iowa Test of Basic Skills (ITBS). As shown, MCLA trains
> teachers to develop and implement lessons that integrate literacy so
> that students will have exposure to and gain competence in using
> specific literacy strategies to increase their comprehension of
> content- related text. Specifically, MCLA designers hope that students
> will use certain research-based strategies that help them strengthen
> their fluency, vocabulary, and comprehension skills before, during,
> and after reading. Researchers hypothesize that continued use of the
> strategies among students will lead to improved student academic
> performance.
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 9 of 15
>
> *Figure 3: Logic Model of the Memphis Striving Readers Whole-School
> Intervention*
>
> **INPUTS ACTIVITIES/OUTPUTS OUTCOMES**
>
> ***Teacher professional development*---Memphis Content Literacy
> Academy**
>
> **(MCLA)**
>
> 30 weekly 3-hour sessions for a total of 180 hours over two years
> designed to train teachers to develop and implement eight classroom
> action plans each year Provide coaching on site for core content
> teachers
>
> ***Principal professional development*---Fellowship**
>
> 45-hour course, over a two-year period, trains principals to
>
> • Provide teachers with feedback from classroom walkthroughs
>
> • Explore schoolwide factors principals can
>
> influence to sustain MCLA
>
> • Incorporate literacy into the school improvement plans
>
> **MCLA Coach professional development**
>
> **MCLA Instructor professional development**

**Core content teachers plan and implement lessons integrating literacy
strategies within their content areas:**

• Use **assessments** of student knowledge and literacy abilities to
plan instruction to meet the needs of all learners

• Provide explicit and direct **instruction** and practice incorporating
appropriate literacy strategies matched to content learning objectives

• Use of set content area **standards** for their instruction plans and
identify literacy strategies students will use with relevant texts

• Use supplementary content-relevant reading materials to meet
individual student needs

• Design and use cooperative learning activities to provide students
extensive practice opportunities

• Collaborate with other core content teachers to

> work on integrating literacy strategies in core content lessons

**Students use literacy strategies when reading content- relevant texts
for core content classes,** specifically: students use before-, during-,
and after-reading strategies to understand and learn from grade- level
content- related texts. These strategies help them develop

• Fluency

• Vocabulary

• Comprehension Student assumes appropriate roles and responsi- bilities
during cooperative learning activities

**Improved student performance** on TCAP and ITBS

> **Schoolwide Factors**
>
> \(1\) Principal leadership, (2) school culture supportive of the use
> of literacy strategies in core content classes,
>
> \(3\) environment press by number/percentage of core content teachers
> that have been trained by MCLA and who are integrating literacy
> strategies in their content lesson
>
> **Design of the Year 1 Implementation Study for the Whole-School
> Intervention**
>
> The implementation evaluation of the MCLA program in Year 1 addressed
> three overarching research questions: (1) To what degree did the
> implemented MCLA treatment match the
>
> intended program standards and features? (2) What contextual district
> and school-level factors influenced the implementation of the MCLA
> program? (3) How did the professional development events, materials,
> or structures present in the control schools compare with what
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 10 of 15
>
> was present in the treatment schools? Specific research questions
> about the Year 1 whole-school intervention included:
>
> What types and amount of professional development were provided to
> teachers, building principals, literacy coaches, and MCLA instructors?
>
> What proportion of teachers received and participated at different
> levels of professional development (e.g., how many used program
> materials or completed the evening MCLA course?)
>
> What type of coaching was provided to teachers?
>
> What is the Year 1 MCLA classroom instructional model?
>
> In addition to answering these questions as part of the implementation
> study of the whole- school intervention, evaluators began in Year 1 to
> develop an Innovation Configuration Map
>
> (Hall & Hord, 2006) that described the observable variations of each
> program component as it was being implemented. This map was refined in
> Year 2, and has allowed evaluators to begin measuring the level of
> implementation in MCLA teachers' classrooms.
>
> ***Data Sources Linked to Research Questions for the Whole-School
> Intervention***
>
> Figure 4 summarizes the relevant sources of data used formally and
> informally in assessing Year 1 implementation of MCLA. An anonymous
> survey administered to teachers at the end of their first semester of
> MCLA provided results about the number of times they worked with
> coaches in their school, and whether or not they completed required
> MCLA assignments (lessons integrating literacy in the classroom).
> Participation in MCLA evening classes was measured through an analysis
> of attendance sign-in sheets collected at the end of the fall
> semester. A
>
> proxy measure for attendance was used at the end of the spring
> semester (whether or not teachers received a stipend for course
> completion) since not all attendance sheets were available. Other data
> sources used in the implementation report include coaching logs that
> list planned and completed coaching activities with specific teachers
> (including the type of professional development provided to coaches)
> and checkout logs of materials in the curriculum resource center (CRC)
> funded by the grant. Information collected from school principals
> about their attendance in MCLA professional development was culled
> largely through qualitative interviews rather than surveys; however, a
> few attendance sheets were obtained from the four principals showing
> their participation in the principal fellowship course.
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 11 of 15
>
> *Figure 4: Data Sources Linked to Research Questions---Whole-School
> Intervention (Year 1)*
>
> **Research questions Measures/Data Sources**
>
> **Surveys/Logs Classroom**
>
> **Observations Record Review**
>
> **What was the level of implementation and variability of MCLA
> professional development and support**
>
> **for teachers, coaches, and building principals in Year 1?**
>
> ***Professional development for teachers***
>
> Type/amount of PD provided to teachers
>
> Proportion of teachers at different levels of PD
>
> Proportion of teachers at adequate level
>
> x

(anonymous) x x x x

> x

(anonymous) x

> of PD x x
>
> Types/amount of coaching provided to
>
> teachers x x x
>
> Proportion of teachers at different levels
>
> of coaching x x x
>
> Proportion of teachers at adequate level
>
> of coaching x x
>
> ***Professional development for coaches/other relevant staff***
>
> Type/amount of PD provided to coaches x x
>
> Proportion of coaches at different levels
>
> of PD x
>
> Type/amount of PD provided to school
>
> principals x
>
> Proportion of school principals at
>
> different levels of PD x
>
> Type/amount of PD provided to district
>
> leaders n/a
>
> Proportion of district leaders at different
>
> levels of PD n/a
>
> **What was the level of implementation and variability of classroom
> instruction in Year 1?**
>
> Proportion of teachers with access to
>
> materials and resources x
>
> Proportion of teachers who implemented literacy strategies (CAPs)
>
> Proportion of teachers who implemented the model at adequate level
>
> x

(anonymous) x x

> x x x
>
> **What did the counterfactual look like in Year 1?**
>
> Proportion of teachers at control schools reporting literacy-related
> PD at follow-up x
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 12 of 15
>
> **Summary of Levels of Implementation Attained for the Whole-School
> Intervention**
>
> In Year 1, MCLA professional development instructors offered 24
> evening class sessions grouped by content area. These evening sessions
> plus several all-day session involving all participants, resulted in a
> total of about 85 hours of formal professional development sessions.
> Instructors also required teachers to implement eight classroom action
> plans that integrated specific literacy practices into the lesson
> plans the teachers used for their own classes. The action plans
> covered the following topics: vocabulary knowledge, student-generated
> questions, read-alouds, think-pair-share, semantic features, leveled
> text, oral retelling, and written retellings of expository text.
> Literacy coaches helped teachers complete the assignments through
> lesson modeling, debriefing conferences, observations, and other
> general support activities. Finally, in addition to coaching
> assistance, teachers were free to use materials and resources from an
> on-site curriculum resource center maintained by the literacy coaches.
>
> RBS tracked attendance at the fall 2006 MCLA evening classes to
> determine individual and schoolwide program participation in the four
> schools receiving the first two years of the intervention. Teachers
> who attended the classes had very high participation (the number of
> possible classes a teacher could attend varied slightly by content
> area). Overall, 43 of the 69
>
> (62.3%) teacher participants attended 80 percent or more of their
> classes, 19 (27.5%) attended between 70 and 80 percent of their
> classes, and 7 (10.1%) attended fewer than 70 percent of the classes.
> Ten (14.5%) individuals had perfect attendance.
>
> Teacher participation in MCLA across the four schools varied widely in
> Year 1. In fact, just over half (51%) of all eligible content area
> teachers in the four schools enrolled in MCLA for the first semester.
> RBS assigned an implementation rating to each school using a formula
> that takes into account teachers' course attendance and completion
> rates and includes the number of eligible teachers who opted not to
> participate in the program. All eligible content area teachers in the
> school were assigned one of four numerical ratings depending on how
> many MCLA professional development sessions they attended in fall 2006
> and whether or not they completed the course in spring 2007. Ratings
> are assigned as follows: teachers who attended 25 percent or fewer of
> the professional development sessions were given a "1," those
> attending between 26
>
> and 50 percent of the sessions offered were assigned a "2," teachers
> participating in between 51 and 75 percent of the sessions received a
> "3," and those who attended 76 to 100 percent of the sessions were
> given a rating of "4."
>
> Once individual ratings were tallied, RBS calculated the average score
> for a school and assigned it one of four corresponding schoolwide
> implementation ratings:
>
> 1 = minimal implementation
>
> 1.1 to 2 = low implementation
>
> 2.1 to 3 = medium implementation
>
> 3.1 to 4 = high implementation
>
> Table 2 summarizes these implementation ratings as well as the number
> and percentage of participants in the intervention by school.
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 13 of 15
>
> *Table 3: Schoolwide MCLA Participation and Implementation Rankings,
> Year 1*
>
> Number of content
>
> area teachers
>
> participating in
>
> MCLA
>
> Number of
>
> eligible content

area teachers in

> August
>
> Percent of

eligible teachers

> participating in
>
> MCLA
>
> School's
>
> implementation
>
> rating (includes

nonparticipants)

> School W 14 40 34.6 1.9 (minimal)
>
> School X 12 29 41.4 2.0 (low)
>
> School Y 16 19 84.2 3.3 (high)
>
> School Z 27 44 61.5 2.7 (medium)
>
> Results show considerable variability in the level of program
> implementation across the four schools: the level was high in one
> school, medium in one, low in one, and minimal in a fourth. These
> ratings reflect the variation in the proportions of teachers who
> participate in MCLA.
>
> Although not shown in the above table, an analysis of attendance also
> indicated that MCLA participation rates varied among teachers by
> content group. The largest group enrolled overall was ELA teachers
> (64.9% of eligible ELA teachers), followed by math (56.5%), science
>
> (54.5%), social studies (40.9%), reading (40.0%), and teachers who
> taught in more than one discipline (31.3%). The percent of classes
> attended during the fall semester was fairly consistent across school
> and content area, ranging from 80 to 88 percent.
>
> Participation in the principal fellowship course during Year 1 was
> uneven and information on specific attendance dates is unavailable.
> MCLA developers found it difficult to maintain principal attendance at
> these weekly meetings and estimated that of the four invitees, one was
> consistently present, one was there for two-thirds of the sessions,
> one individual attended approximately one-third of the time, and one
> principal participated only once or twice during the course of the
> semester. The developer has made significant changes to the structure
> of the fellowship in Year 2 based upon the Year 1 experience and has
> begun inviting assistant
>
> principals to attend the fellowship sessions.
>
> The Year 1 implementation report contains additional information about
> MCLA participants' use of program materials and experience with the
> on-site literacy coaches. Nine in ten participants (91%) completed the
> evening course in spring 2007 (six teachers dropped out), and
>
> 85 percent of participants worked with coaches during the school day
> at least ten times during the year, according weekly logs kept by the
> coaches during Year 1. The remaining ten teachers collaborated with
> coaches less frequently, according to the coaching logs. While it does
> not appear that exposure to coaches differed by grade level, eight of
> the ten teachers with fewer than ten exposures taught at the same
> school. Evaluators cannot determine whether the differential coaching
> dosage is due to the manner in which coaches maintained their logs or
> an actual difference in the level of coaching provided to teachers at
> that particular school.
>
> Lastly, approximately 70 percent (N = 48) of the 69 MCLA participants
> checked out resources or materials provided by the program at least
> once in Year 1. Data from the anonymous teacher survey corroborates
> that while a majority of teachers visited the resource center, very
> few teachers used the materials multiple times or for a prolonged
> period of time.
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 14 of 15
>
> Participants who used the materials issued high praise; however,
> encouraging teachers to use the center was a continued challenge
> according to interviews with MCLA program staff throughout Year 1.
>
> **Conclusions Regarding Implementation of the Whole-School
> Intervention**
>
> Analysts found substantial variation in the implementation of the
> Memphis Content Literacy Academy professional development program
> offered to teachers in four of the eight middle schools. This program,
> designed to help teachers integrate literacy strategies into their
> content area lessons, included evening courses for teachers, a
> principal component, and on-site literacy coaching. Its level of
> implementation ran from minimal at one school (whose principal showed
> little support for MCLA) to high at another school (whose principal
> remained actively committed to MCLA during its first year).
>
> ***References***
>
> Hall, G. E., & Hord, S.M. (2006). *Implementing Change: Patterns,
> Principles, and Potholes*
>
> *(Second Edition).* Boston, MA: Allyn & Bacon.
>
> Hart, T. (2008, June 10). Memphis school board realizes it's time for
> change. *The Commercial Appeal.* Retrieved July 14, 2008, from
> http://www.commercialappeal.com/news/2008/jun/10/guest-column-city-school-board-
> realizes-its-time/
>
> Ippolito, J., Steele, J.L., & Samson, J.F. (2008). Introduction: Why
> adolescent literacy matters now. *Harvard Educational Review,* 78(1),
> 1--5.
>
> MCS Quick Facts. (n.d.) Retrieved July 21, 2008, from
> http://www.memphis- schools.k12.tn.us/facts_about_mcs.asp
>
> Potts, R., Perkins, J.H., Heeren, E., Harris, R., & Feldman, J. (2008,
> May). *Will that work for us? Interpreting research from the Memphis
> Striving Readers Project (MSRP).* Paper presented
>
> at the International Reading Association Research Conference, Atlanta,
> GA.
>
> Urban Child Institute, The. (2008). *The State of children in Memphis
> and Shelby County.* Memphis, TN: Author. Retrieved July 21, 2008, from
> http://www.theurbanchildinstitute.org/
>
> Memphis Striving Readers Project Year 1 Implementation Executive
> Summary Page 15 of 15