MOBILIZING MOBILIZING

VOLUNTEER VOLUNTEER TUTORS

TO IMPROVE 

STUDENT LITERACY LITERACY

Implementation, Impacts, 

and Costs of the 

Reading Partners Program 

Robin Tepper Jacob 

Catherine Armstrong 

Jacklyn Altuna Willard 	 					March 2015 
_______________________________ 

MDRC BOARD OF DIRECTORS 

Mary Jo Bane 

Chair 

Thornton Bradshaw Professor of 

Public Policy and Management 

John F.

Gov

 Kennedy School of 

ernment 

Harvard University 

Robert Solow 

Chairman Emeritus 

Institute Professor Emeritus 

Massachusetts Institute of 

Technology 

Rudolph G. Penner 

Treasurer 

Senior Fellow and Arjay and 

Frances Miller Chair in 

Public Policy 

Urban Institute 

_______________________________ 

Robert E. Denham 

Partner 

Munger, Tolles & Olson LLP 

Ron Haskins 

Senior Fellow, Economic Studies 

Co-Director, Center on Children and 

Families 

Brookings Institution 

James H. Johnson, Jr. 

William Rand Kenan Jr. 

Distinguished Professor of 

Strategy and Entrepreneurship 

Director, Urban Investment 

Strategies Center 

University of North Carolina 

Lawrence F. Katz 

Elisabeth Allison Professor of 

Economics 

Harvard University 

Bridget Terry Long 

Professor of Education and 

Economics 

Graduate School of Education 

Harvard University 

Josh B. McGee 

Vice President of Public 

Accountability 

Laura and John Arnold Foundation 

Richard J. Murnane 

Thompson Professor of Education 

and Society 

Graduate School of Education 

Harvard University 

Jan Nicholson 

President 

The Grable Foundation 

John S. Reed 

Retired Chairman 

Citigroup 

Michael Roster 

Former General Counsel 

Stanford University 

Former Managing Partner 

Morrison & Foerster, Los Angeles 

Cecilia E. Rouse 

Dean, Woodrow Wilson School of 

Public and International Affairs 

Katzman-Ernst Professor in the 

Economics of Education 

Professor of Economics and 

Public Affairs 

Princeton University 

Isabel V. Sawhill 

Senior Fellow, Economic Studies 

Co-Director, Center on Children and 

Families 

Brookings Institution 

Gordon L. Berlin 

President, MDRC 
 

Mobilizing Volunteer Tutors

to Improve Student Literacy 

Implementation, Impacts, and Costs 

of the Reading Partners Program 

Robin Tepper Jacob

(Institute for Social Research, 

University of Michigan) 

Catherine Armstrong 

Jacklyn Altuna Willard

(MDRC) 

with 

A. Brooks Bowden
 

Yilin Pan
 

(Center for Benefit-Cost Studies of Education,
 

Teachers College, Columbia University)
 

March 2015 
 

 

 

 

 

Funders for the Reading Partners Evaluation 

This report is based on work supported by the Social Innovation Fund (SIF), a key White House 

initiative and program of the Corporation for National and Community Service (CNCS). The Social 

Innovation Fund combines public and private resources with the goal of increasing the impact of 

innovative, community-based solutions that have compelling evidence of improving the lives of 

people in low-income communities throughout the United States. 

The Edna McConnell Clark Foundation is leading a SIF project that includes support from CNCS and 

15 private co-investors: The Edna McConnell Clark Foundation, The Annie E. Casey Foundation, The 

Duke Endowment, The William and Flora Hewlett Foundation, The JPB Foundation, George Kaiser 

Family Foundation, The Kresge Foundation, Open Society Foundations, The Penzance Foundation, 

The Samberg Family Foundation, The Charles and Lynn Schusterman Family Foundation, The Starr 

Foundation, Tipping Point Community, The Wallace Foundation, and the Weingart Foundation. 

Dissemination of MDRC publications is supported by the following funders that help finance 

MDRC’s public policy outreach and expanding efforts to communicate the results and implications of 

our work to policymakers, practitioners, and others: The Annie E. Casey Foundation, The Harry and 

Jeanette Weinberg Foundation, Inc., The Kresge Foundation, Laura and John Arnold Foundation, 

Sandler Foundation, and The Starr Foundation. 

In addition, earnings from the MDRC Endowment help sustain our dissemination efforts. Contributors 

to the MDRC Endowment include Alcoa Foundation, The Ambrose Monell Foundation, AnheuserBusch Foundation, Bristol-Myers Squibb Foundation, Charles Stewart Mott Foundation, Ford 

Foundation, The George Gund Foundation, The Grable Foundation, The Lizabeth and Frank Newman 

Charitable Foundation, The New York Times Company Foundation, Jan Nicholson, Paul H. O’Neill 

Charitable Foundation, John S. Reed, Sandler Foundation, and The Stupski Family Fund, as well as 

other individual contributors. 

The findings and conclusions in this report do not necessarily represent the official positions or 

policies of the funders. 

For information about MDRC and copies of our publications, see our website: www.mdrc.org. 

Copyright © 2015 by MDRC®. All rights reserved. 
Overview 

This study reports on an evaluation of the Reading Partners program, which uses community volunteers to provide one-on-one tutoring to struggling readers in underresourced elementary schools. 

Established in 1999 in East Menlo Park, California, Reading Partners’ mission is to help children 

become lifelong readers by empowering communities to provide individualized instruction with 

measurable results. At each school, Reading Partners transforms a dedicated space into a “reading 

center,” places a full-time team member on site to manage day-to-day operations, and recruits a 

corps of 40 to 100 community volunteers to work one-on-one with students in pull-out sessions during the school day or after school in kindergarten through grade 5. (This evaluation included only 

students in grades 2 through 5.) 

In March 2011, Reading Partners received a three-year True North Fund investment of up to $3.5 

million in grants from the Edna McConnell Clark Foundation and the Social Innovation Fund, 

matched by $3.5 million from True North Fund co-investors, to further expand its early-intervention 

literacy program to elementary schools throughout the country and evaluate its effectiveness. This 

report is the second publication from that evaluation. A policy brief released by MDRC in June 2014 

reported the initial findings from the evaluation, which was conducted during the 2012-2013 school 

year in 19 schools in three states, with more than 1,100 students randomly assigned to the study’s 

program and control groups. 

This report builds on those initial findings by describing the Reading Partners program and its implementation in greater detail, exploring whether the program is more or less effective for particular 

subgroups of students, and assessing some of the potential explanations for the program’s success to 

date. In addition, this report includes an analysis of the cost of implementing the Reading Partners 

program in 6 of the 19 sites. 

Key Findings 

•	 Despite the myriad difficulties inherent in operating a program whose direct service providers 

are volunteers, Reading Partners was implemented in the schools with a relatively high degree 

of fidelity to the program model. On average, students in the study received approximately 1.5 

tutoring sessions per week, and spent 28 weeks in the Reading Partners program. 

•	 Reading Partners had a positive and statistically significant impact on three different measures 

of student reading proficiency. These impacts are equivalent to approximately one and a half to 

two months of additional growth in reading proficiency among the program group relative to the 

control group and are robust across a range of student characteristic subgroups as well as across 

groups of students who had different levels of reading comprehension skills at the start of the 

study. 

•	 Reading Partners is a low-cost option for underresourced schools because a majority of the costs 

are in-kind contributions, primarily from community volunteers. On average, schools bear only 

about 20 percent ($710 per program group student) of the total cost of the resources required to 

implement the program, and over half of these costs are in-kind contributions of space and staff 

time from the school. 

iii 

Contents 

Overview iii
 

List of Exhibits vii
 

Preface ix
 

Acknowledgments xi
 

Executive Summary ES-1
 

Chapter 

1	 Introduction and Evaluation Overview 1
 

The Policy Context 2
 

Evidence-Based Approaches to Improving Literacy in Elementary School 5
 

Overview of the Evaluation 7
 

2	 The Reading Partners Program 13
 

Target Population and Selection Criteria 13
 

Reading Partners’ Organizational Structure, Staff, and Volunteer Tutors 15
 

Core Components of Reading Partners 19
 

3	 Understanding Implementation Fidelity 25
 

Data Collection 25
 

Implementation Fidelity Measure 27
 

Factors That Facilitated Implementation 37
 

Barriers to Implementation 37
 

Summary 39
 

4	 The Impact of Reading Partners 41
 

Methods 42
 

Understanding the Contrast Between the Program and Control Groups 49
 

Impact of Reading Partners on Measures of Reading Proficiency 51
 

Impact of Reading Partners on Academic Behavior, Performance, and Attendance 55
 

Subgroup Analyses 55
 

Exploring Variation in Outcomes Based on Fidelity and Context 59
 

Summary 64
 

5	 The Cost of Reading Partners 67
 

Methods 69
 

Findings 72
 

Conclusions 78
 

v 
6	 Conclusions 81
 

Factors Contributing to the Observed Outcomes 85
 

Study Limitations 89
 

Recommendations for Future Research 91
 

Conclusions 91
 

Appendix
 

A Implementation Study Methods 93
 

B Impact Study Methods and Teacher Survey 99
 

C Tutor Background Characteristics and Additional Impact Findings 105
 

D Cost Study Methods 115
 

E Additional Cost Findings 125
 

References 131
 

vi 
List of Exhibits
 

Table 

1.1	 Characteristics of Reading Partners Study Schools and Other School
 

Samples (2011-2012) 10
 

3.1	 Summary of Implementation Data Collection, by Source 27
 

3.2	 Average Student’s Experience in Reading Partners 31
 

4.1	 Baseline Characteristics of Program and Control Group Students 44
 

4.2	 Summary of Data Collection for Impact Analysis, by Source 45
 

4.3	 Reading Instruction Received 50
 

4.4	 Primary Impacts of Reading Partners on Reading Proficiency 53
 

4.5	 Secondary Impacts of Reading Partners on Students’ Attendance and
 

Teacher-Reported Achievement and Behavior 56
 

4.6	 Subgroup Analysis of Primary Impacts 57
 

4.7	 Impacts of Reading Partners on Students’ Reading Comprehension
 

Relative to National Norms 59
 

4.8	 Subgroup Analysis of Primary Outcomes Based on Fidelity and Context
 

Measures 61
 

5.1	 Characteristics of Cost Study Schools and Other Reading Partners Study
 

Schools (2011-2012) 70
 

5.2	 Cost of Reading Partners per Program Group Student 73
 

5.3	 Site-Level Costs of Reading Partners and Other Supplemental Services 76
 

6.1	 Reading Programs with Rigorous Evidence Base 82
 

A.1	 Fidelity Index Codebook 96
 

B.1	 Availability of Standardized Scores for Primary Outcomes 102
 

C.1	 Tutor Background Characteristics, by Site 108
 

C.2	 Impact of Reading Partners on State Achievement Testing Outcomes 109
 

C.3	 Students’ Reading Growth Based on Their Number of Assigned Tutors 110
 

C.4	 Program Impacts, by School Level of Tutor Consistency 111
 

C.5	 Students’ Reading Growth Based on Background Characteristics of
 

Their Reading Partners Tutors 112
 

C.6	 Subgroup Analysis of Primary Impacts Based on Reading Partners
 

Target Status 113
 

vii 
E.1 Reading Partners Cost per Student: Main Study Findings and at Scale 127
 

E.2 Sensitivity Tests of Staff Prices for AmeriCorps Members and Volunteers 129
 

Figure 

2.1 Reading Partners Program 14
 

2.2 Staffing Support Structure for a Reading Partners Center 16
 

2.3 Backgrounds of Tutors in Study Schools 18
 

3.1 Fidelity Scores of Study Schools 29
 

4.1 Time Spent in Reading Instruction and Supplemental Services 52
 

B.1 Example of Teacher Survey 104
 

C.1 Student Reading Growth 114
 

Box 

1.1 The EMCF Social Innovation Fund 3
 

2.1 A Reading Partners Tutoring Session 21
 

3.1 Reading Center Materials 33
 

4.1 What Is an Effect Size? 54
 

viii
 
Preface
 

Over the last two decades, numerous federal, state, and local efforts have focused on improving 

the literacy skills of America’s young people. Yet, despite these efforts, only limited progress 

has been made. One approach that has consistently shown promise in improving literacy outcomes, especially for young children, is one-on-one tutoring. One-on-one tutoring delivered by 

certified teachers has repeatedly demonstrated large positive impacts on the reading proficiency 

of struggling readers. Yet, while this approach has a solid research base demonstrating its effectiveness, it is both time- and resource-intensive, placing a heavy burden on teachers and schools, 

and thus is an expensive way to ameliorate the problem of low literacy. As such, it may not always be a viable option for already underresourced schools. 

This report explores another model: using community volunteers to provide tutoring to 

struggling readers, but in a structured, programmatic framework designed and managed by a 

dedicated nonprofit organization. The Reading Partners program recruits community volunteers 

who devote a few hours each week to tutoring students in kindergarten through grade 5 in reading, using a structured curriculum. Tutors come from varied backgrounds, are not required to 

have experience working with children or teaching reading, and receive only limited training 

before beginning tutoring. But children are assessed and tutors use specific materials supplied 

by the program, while a site coordinator ensures that each student receives the intended instruction, advises tutors whose students have specific difficulties, and fills in when tutors are unable 

to make appointments. 

This evaluation reaches the encouraging conclusion that the Reading Partners program 

successfully improved students’ reading proficiency, even among children in the upper elementary grades. Furthermore, the cost to the schools was quite low and substantially less than the 

costs of other supplemental reading services that are typically offered to struggling readers. All 

this suggests that strong volunteer tutoring programs, like Reading Partners, may be a costeffective option for underresourced schools and deserve greater attention in the national effort to 

improve literacy skills. 

Gordon L. Berlin 

President, MDRC 

ix 

Acknowledgments

This evaluation of the Reading Partners program would not have been possible without the collaboration of Reading Partners and funding from the Edna McConnell Clark Foundation 

(EMCF) and the Social Innovation Fund. 

We owe special thanks to the many program managers and site coordinators on the 

Reading Partners staff, whose daily efforts and enthusiasm allowed us to implement the evaluation with integrity, and to Dean Elson, Michael Lombardo, Matt Aguiar, and the rest of the 

Reading Partners leadership team for their ongoing support. 

Additionally, we thank Rob Ivry, Fred Doolittle, William Corrin, Jean Grossman, 

Hank Levin, Rekha Balu, Marie-Andree Somers, Alice Tufel, Gabriel Rhoads, Jehan Velji, 

and the EMCF Evaluation Advisory Committee for their thoughtful comments on this report, 

and Andrea Shane, Claire Montialoux, Ellie Leahy, Nicole Clabaugh, Rachel Rifkin, Tom 

Smith, Brock Grubb, Mike Sack, Paulette Cha, and Joseph Quinn for their contributions to the 

project. Thanks also go to Stephanie Cowell and Carolyn Thomas, who prepared the report 

for publication. 

This material is based on work supported by the Corporation for National and Community Service (CNCS). The mission of CNCS is to improve lives, strengthen communities, and 

foster civic engagement through service and volunteering. CNCS, a federal agency, engages 

more than five million Americans in service through AmeriCorps, Senior Corps, the Social Innovation Fund, the Volunteer Generation Fund, and other programs, and leads the president’s 

national call-to-service initiative, United We Serve. 

The Authors 

xi 

Executive Summary 

The Reading Partners program uses community volunteers to provide one-on-one tutoring to 

struggling readers in underresourced elementary schools. Established in 1999 in East Menlo 

Park, California, Reading Partners is a not-for-profit corporation whose mission is to help children become lifelong readers by empowering communities to provide individualized instruction 

with measurable results. The Reading Partners model is based on the premise that too many 

children in low-income communities are not reading proficiently and that many teachers, 

schools, and parents in those communities lack the resources and infrastructure to address the 

problem. 

The evaluation that is described in this report finds that the Reading Partners program 

successfully improved students’ reading comprehension, sight word efficiency, and fluency 

over the course of the school year by an amount that is roughly equivalent to one and a half to 

two months of learning. In addition to demonstrating these measurable impacts, the evaluation 

provides evidence that the cost to the schools was less than half the costs to schools of other 

supplemental reading programs. The costs for Reading Partners were lower because the volunteer tutors accounted for a large share of the resources that were used. Thus, this study provides 

additional evidence that volunteer programs can work and that one-on-one tutoring is effective 

in improving academic outcomes. Furthermore, the results suggest that effective tutoring programs, like Reading Partners, may be a cost-effective option for underresourced schools, 

because they bring additional resources to the school through community volunteers. 

Background 

In March 2011, Reading Partners received a three-year True North Fund investment of up to 

$3.5 million in grants from the Edna McConnell Clark Foundation and the Social Innovation 

Fund, matched by $3.5 million from True North Fund co-investors, to further expand its 

early-intervention literacy program to elementary schools throughout the country and evaluate 

its effectiveness. 

In addition to answering questions about the effectiveness of the Reading Partners 

program, the evaluation was designed to examine the potential for volunteer tutoring more 

generally to help improve the reading proficiency of struggling readers. To meet its design 

objectives, the evaluation included an implementation study, an impact study, and a cost 

study. The implementation and impact studies included 19 schools with Reading Partners 

sites in three different states, and the cost study included a subsample of six of these schools, 

also across three different states. Together, the three facets of the evaluation are designed to 

address the following broad sets of research questions: 

ES-1 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1.	 In what context was the Reading Partners program implemented, and was it 

implemented as intended — that is, with fidelity to the model? How much 

variability in fidelity of implementation was observed across the sites? What 

factors contributed to any observed variability? 

2.	 On average, did the Reading Partners program have a positive impact on students’ reading proficiency across three key components of early reading ability: sight word efficiency, reading fluency, and comprehension? 

3.	 What resources are needed to implement the Reading Partners program as 

described in this evaluation and what proportion of the costs of implementing 

the program are borne by the school? 

The implementation study included site visits to all the schools participating in the 

evaluation in the winter of the study year, interviews with key program and school staff and 

volunteers, and the collection of programmatic data from the sites and from Reading Partners’ 

own management information system (MIS). The impact study used a student-level randomized 

controlled trial design, in which students were randomly assigned within each school either to a 

program group that would participate in Reading Partners during the 2012-2013 school year or 

to an “as is” control condition. A total of 1,265 students in grades 2 through 5 across the 19 

schools were randomly assigned. Students were assessed on three different measures of reading 

proficiency in the fall and spring of the study year. Finally, the cost study calculated the intervention’s total cost by summing the costs of all the resources that were necessary to implement 

the program. 

The Reading Partners Model 

Reading Partners is a “pull-out” program (meaning that students are pulled out of their regular 

classrooms or after-school program for a limited time period in order to meet with their tutors) 

that operates both during the school day and after school in designated spaces called “reading 

centers” at each of the partner schools. The program aims to serve students in kindergarten 

through grade 5 who are half a year to two and a half years behind grade level in reading, who 

are conversationally fluent in English, and who do not have any special needs (that is, do not 

have an Individualized Education Program).1 The Reading Partners program consists of twiceweekly, one-on-one tutoring sessions that last 45 minutes each. Students are tutored by community volunteers who need not have any experience working in education or with children. Tutor1

Individualized Education Programs are developed for children who are found through assessment to have 

a disability that affects their learning process. The program outlines how teachers will help these students learn 

more effectively, considering each of their learning styles and needs. 

ES-2 
 

 

 

 

 

 

 

 

 

 

 

ing sessions are overseen by site coordinators — full-time Reading Partners team members, 

who are usually AmeriCorps members and who are also responsible for managing the day-today operations of the program.2 

The Reading Partners model consists of six core components: 

• Regular, one-on-one tutoring 

A key feature of the Reading Partners model is to provide students with individualized 

reading instruction. This one-on-one support — as opposed to small-group support — is the 

main component that differentiates Reading Partners from many other supplemental services 

that are available to struggling readers. 

• Dedicated school space and use of materials 

Reading Partners tutoring is designed to take place on school grounds in a dedicated 

space, or reading center, which contains specific features designated by Reading Partners, such 

as work stations for tutor-student pairs, a library with materials that are suitable for readers at 

different levels, an area with resources for volunteers to use, and posters and other materials 

with a reading theme that can be displayed on the wall. 

• Structured and individualized curriculum 

Tutoring sessions follow a consistent structure. The tutor begins by reading aloud to 

model fluent reading and pausing periodically to ask the student comprehension questions or to 

check the student’s knowledge of key vocabulary. The tutor then uses curricular materials to 

introduce or reinforce a specific reading skill or concept. Finally, the tutor works with the student to apply the skill or content while the student reads aloud. 

• Data-driven instruction 

Reading Partners uses data to implement and support the model, including the results of 

student assessments that are given three times a year. These assessments are used to create and 

update a student’s Individualized Reading Plan (IRP), which identifies student goals and areas 

on which to focus. 

2

AmeriCorps is a program of the Corporation for National and Community Service that places young 

adults in service positions at nonprofit organizations, schools, public agencies, and faith-based entities. In return for their service, AmeriCorps members receive a living stipend, health insurance, and, when they complete 

their program, an education grant. See Corporation for National and Community Service, “Our Programs: 

AmeriCorps,” online publication (2014), at www.nationalservice.gov. 

ES-3 
 

 

 

 

• Rigorous and ongoing training 

Reading Partners school-based staff and AmeriCorps members participate in organized 

training sessions before the school year begins as well as ongoing (usually monthly) sessions 

throughout the school year. Topics covered in these training sessions include the Reading Partners model, a detailed review of the curriculum, how to train volunteer tutors, and how to use 

data and the Reading Partners data system. A short initial training session, as well as periodic 

training on specific topics, is offered to tutors as well. 

• Instructional supervision and support 

Instructional supervision and support are provided on an ongoing basis by and for 

Reading Partners staff and volunteers. Site coordinators supervise volunteers during tutoring 

sessions and provide guidance and suggestions to tutors who need additional support. Program 

managers — more experienced staff members who generally have a background in teaching — 

work with site coordinators to troubleshoot a range of issues, including those related to communications with school staff, managing tutors, or identifying best practices to better support the 

progress of a specific student. 

Key Findings 

• The Reading Partners Program was implemented with fidelity. 

Despite the myriad difficulties inherent in operating a program whose direct service 

providers are volunteers, Reading Partners was implemented in the schools with a relatively 

high degree of fidelity. 

Students received regular one-on-one tutoring in a dedicated school space. On average, students in the study received approximately 1.5 tutoring sessions per week, and spent 28 

weeks in the Reading Partners program. Although this intensity is slightly less than the program model recommends, on average students consistently received three tutoring sessions 

every two weeks. All sites in the study had a designated reading center where tutoring took 

place and where selected materials and resources were made available to program staff, tutors, 

and students. 

Students’ reading progress was monitored regularly. Ninety-five percent of the program group students who participated in the Reading Partners program for the entire year were 

assessed by Reading Partners staff using their own internal assessments at the three prescribed 

points that the Reading Partners model dictates: when a student first enters the program, midway through the school year, and at the end of the school year. 

ES-4 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

Staff and volunteers believed that they had adequate training and support to perform 

their jobs successfully. Although the volunteers received limited training before they began 

tutoring (approximately an hour of orientation and tutoring observation), most tutors indicated 

that they felt adequately trained for the role. They also felt well supported by the site coordinators, who provided monitoring and assistance during tutoring sessions as well as additional 

feedback on how to address specific issues with students outside of the tutoring session. The 

full-time Reading Partners staff and AmeriCorps members (including program managers and 

site coordinators) consistently indicated that they had access to ongoing support from their 

supervisors. 

The biggest challenge that Reading Partners faced in implementing the program was 

ensuring tutor attendance and retention. Reading Partners requests that tutors make a onesemester commitment, and site coordinators at the study sites reported that many volunteers, 

particularly high school and college students, did not stay beyond that period of time. As a 

result, new tutors had to be brought on throughout the year. Furthermore, volunteers varied in 

their consistency and commitment. Site coordinators reported that tutors sometimes failed to 

arrive at their scheduled time and at times did not notify the site coordinator beforehand. 

However, Reading Partners put structures in place to address these problems, including the 

use of substitute tutors, make-up days, and tutoring sessions conducted by site coordinators. 

These tactics meant that a student did not necessarily miss tutoring sessions as a result of tutor 

inconsistency. 

•	 Reading Partners had a positive and statistically significant impact on 

three different measures of student reading proficiency. 

The study quantified the impact of Reading Partners through three different assessments: the Stanford Achievement Test (SAT-10) for reading comprehension; the AIMSweb 

oral reading fluency measure; and the Test of Word Reading Efficiency, 2nd Edition (TOWRE2), for sight word efficiency. The effect size of each impact, which reflects the magnitude of the 

difference between the program and control groups, was 0.10 standard deviations for reading 

comprehension, 0.09 for reading fluency, and 0.11 for sight word efficiency. These statistically 

significant impacts are equivalent to approximately one and a half to two months of additional 

progress in reading among the program group members relative to students in the control group, 

who did not participate in the Reading Partners program.

3 

These results (between 0.09 and 0.11 standard deviations) are comparable in magnitude with results of other reading interventions that have been rigorously evaluated on a large 

3

Statistical significance indicates that the impact is likely a result of the intervention rather than a chance 

occurrence. 

ES-5 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

scale in grades 2 through 5. Although some interventions have produced large impacts for kindergarteners and first-graders, for students in grades 2 through 5 most rigorous evaluations of 

reading programs generally have found impacts between 0.10 and 0.25 standard deviations, and 

among rigorously evaluated tutoring programs in particular, impacts have been between 0.10 

and 0.15 standard deviations.

4 

Since students in the control group were also receiving supplemental reading services, 

the impact of Reading Partners should be interpreted as the impact of the program relative to 

other supplemental service receipt, not the impact of Reading Partners compared with no intervention. Approximately two-thirds of students in the control group received at least one supplemental reading service and were more likely to receive small-group intervention support than 

were their counterparts in the program group. Because the control group students were also 

receiving supplemental reading instruction, the program group members received, in total, only 

about an hour more of reading instruction each week than the control group received. 

Program impacts are robust across a range of student characteristic subgroups as well 

as across subgroups of students with varying levels of reading comprehension skills at baseline. 

Positive and statistically significant impacts were found on at least one measure of reading proficiency for each of the following student groups: male students, female students, English language learners (that is, students whose first language is not English), students who are fluent in 

English, students in lower grades (grades 2 and 3), students in upper grades (grades 4 and 5), 

and students with baseline reading comprehension scores in the lowest three quartiles of the 

study sample. 

Reading Partners had a positive and statistically significant impact on the percentage 

of students who moved out of the lowest national quartile in terms of reading comprehension. 

At the end of the year, 19 percent of the program group students who had scored in the bottom 

quartile nationally at baseline had moved out of the lowest quartile, as opposed to only 12 percent of the control group students. 

There were no consistent patterns between the impacts and various school-level 

measures of program context or fidelity. The study team explored the relationship between impacts and several aspects of program context and fidelity, including tutor consistency, fidelity of 

4

Yung Soo Lee, Nancy Morrow-Howell, Melissa Jonson-Reid, and Stacey McCrary, “The Effect of the 

Experience Corps® Program on Student Reading Outcomes,” Education and Urban Society 44, 1 (2010): 97­

118; Carrie E. Markovitz, Marc W. Hernandez, Eric C. Hedberg, and Benjamin Silberglitt, Impact Evaluation 

of the Minnesota Reading Corps K-3 Program (Chicago: NORC at the University of Chicago, 2014); Henry 

May, Abigail Gray, Jessica N. Gillespie, Philip Sirinides, Cecile Sam, Heather Goldsworthy, Michael Armijo, 

and Namrata Tognatta, Evaluation of the i3 Scale-up of Reading Recovery: Year One Report, 2011-12 (Philadelphia: Consortium for Policy Research in Education, 2013). 

ES-6 
 

 

 

 

 

 

 

 

 

 

 

 

implementation, years of operation, and dosage (the frequency and intensity of service delivery). Across these analyses, there is no indication of particular aspects of implementation or 

context that made Reading Partners more effective. 

• Reading Partners is a low-cost option for underresourced schools. 

The total resource value, or cost, of Reading Partners is approximately $3,610 per program group student. Other effective early literacy interventions that have been evaluated at scale 

are at least as costly as Reading Partners.5 However, unlike many other resource-rich programs, 

a majority of Reading Partners’ costs ($1,910 out of $3,610) are in-kind contributions, primarily 

from community volunteers. As a result, Reading Partners schools bear only a small portion of 

the total costs of the program. On average, schools contribute only 20 percent of the total 

resources required to implement the program ($710 per program group student), and over half 

of these costs are in-kind contributions of space and staff time.6 

The volunteer time and transportation represent the largest portion of the total resources needed to implement the program. Almost half (42 percent) of the resources required 

for Reading Partners can be attributed to volunteers. Volunteers contributed, on average, the 

equivalent of $1,520 per program group student, which included both their time and transportation costs. Because the tutors are not compensated for their time or transportation, they subsidize a large portion of the costs of the program. 

While Reading Partners is often more resource-intensive than the other supplemental 

services that are available to students in the study schools, many of those resources are provided in-kind and thus schools are required to contribute a much smaller portion of those costs. In 

addition to Reading Partners, the six school sites that were included in the cost study offered 

other supplemental services that provided reading instruction beyond what students received 

during regular classroom teaching. Those other services were provided to both program and 

control group students, although control group students received more of them than did the program group. The cost per student for the average of the other supplemental reading services of5

Fiona M. Hollands, Yilin Pan, Robert Shand, Henan Cheng, Henry M. Levin, Clive R. Belfield, Michael 

Kieffer, A. Brooks Bowden, and Barbara Hanisch-Cerda, Improving Early Literacy: Cost-Effectiveness Analysis of Effective Reading Programs (New York: Center for Benefit-Cost Studies of Education, Teachers College, Columbia University, 2013); Jessica Simon, “A Cost-Effectiveness Analysis of Early Literacy Interventions,” unpublished paper (New York: Columbia University, 2011). 6

To calculate the cost per program group student, first the total cost (or the total resource value) of the 

Reading Partners program at each site was divided by the total number of Reading Partners tutoring sessions 

provided to all students in the program during the year (not just the sessions provided to students participating 

in the study), including students in kindergarten and first grade, to obtain the cost per session. Then, to determine the average cost of serving each program group student, the average cost per session was multiplied by 

the average number of sessions that the students participating in the study received. 

ES-7 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

fered at the six sites was $1,780 and ranged from $1,050 to $4,890 per student. The range in the 

total resources provided by the supplemental services across the six sites depended on both the 

number and type of supplemental services that were offered. Some services, like computerbased programs, were much less resource-intensive. When the cost of Reading Partners is compared with the cost of other supplemental services that schools offered to struggling readers, the 

cost to the school for Reading Partners was $710 per program group student, as noted above, 

while the average cost of the other supplemental reading services borne by the school or school 

district was $1,700. 

Conclusions 

Overall, the evaluation finds that the Reading Partners model is effective. The program produced measurable impacts on reading skills among students who had a fairly broad range of 

reading abilities when the study began and among students from a wide range of grades 

(grades 2 through 5). Reading Partners produced these impacts despite the lack of experience 

among tutors, the somewhat limited training the tutors received, and the relatively high degree 

of tutor turnover. Furthermore, the findings illustrate the high value of the Reading Partners 

program from the perspective of the schools. The program uses many resources, but the volunteers account for a large part of those resources. As a result, the schools bear only 20 percent of the costs. 

Thus, the study provides evidence that if the right design and administrative structures 

are put into place, volunteer tutoring programs can be effective when implemented at scale, and 

volunteer tutoring programs may be a cost-effective option for underresourced schools. 

Reading Partners manages to be effective even in the absence of oft-cited key components to successful tutoring, including, in particular, extensive tutor training and tutor consistency.7 Further research is required to understand whether improving these components of the 

model would affect the magnitude of the impacts and whether the impact of Reading Partners is 

sustained for more than one year. 

7

Batya Elbaum, Sharon Vaughn, Marie Tejero Hughes, and Sally Watson Moody, “How Effective Are 

One-to-One Tutoring Programs in Reading for Elementary Students at Risk for Reading Failure? A MetaAnalysis of the Intervention Research,” Journal of Educational Psychology 92, 4 (2000): 605-619; Gary W. 

Ritter, Joshua H. Barnett, George S. Denny, and Ginger R. Albin, “The Effectiveness of Volunteer Tutoring 

Programs for Elementary and Middle School Students: A Meta-Analysis,” Review of Educational Research 79, 

1 (2009): 3-38; Robert E. Slavin, Cynthia Lake, Susan Davis, and Nancy A. Madden, “Effective Programs for 

Struggling Readers: A Best-Evidence Synthesis,” Educational Research Review 6, 1 (2011): 1-26; Barbara A. 

Wasik, “Using Volunteers as Reading Tutors: Guidelines for Successful Practices,” The Reading Teacher 51, 7 

(1998): 562-570. 

ES-8 
 

 

 

 

 

 

 

Chapter 1 

Introduction and Evaluation Overview 

Reading skills are the key building blocks of a child’s formal education. Yet, the national statistics on literacy attainment are profoundly distressing:1 Two out of three American fourthgraders are reading below grade level and almost one-third of children nationwide lack even 

basic 

reading skills. For children in low-income families, the numbers are even more troubling, with 

80 percent reading below grade level. Despite several decades of education reform efforts, only 

incremental progress has been made in addressing this reading crisis. From 1998 to 2013, the 

number of low-income fourth-graders reading at a proficient level increased by only 7 percentage points. 

The Reading Partners program that is described in this report uses community volunteers to provide one-on-one tutoring to struggling readers in underresourced elementary schools. 

Established in 1999 in East Menlo Park, California, Reading Partners is a not-for-profit corporation whose mission is to help children become lifelong readers by empowering communities to 

provide individualized instruction with measurable results. The Reading Partners model is 

based on the premise that too many children in low-income communities are not reading proficiently and that many teachers, schools, and parents in those communities lack the resources 

and infrastructure to address the problem. For these reasons, the program typically serves students in federally designated low-income schools and focuses on using community volunteers 

to provide the needed support for struggling readers. 

At each school, Reading Partners transforms a dedicated space into a “reading center,” places a full-time staff member on site to manage day-to-day operations, and recruits a 

corps of 40 to 100 community volunteers to work one-on-one with struggling readers in kindergarten through grade 5 (although this evaluation included only students in grades 2 

through 5). Operating independently within the school building, Reading Partners has grown 

to serve more than 7,000 students in over 130 schools throughout California, Colorado, Maryland, New York, Oklahoma, South Carolina, Texas, Washington, and Washington, DC, in the 

years since its inception.2 

In March 2011, Reading Partners received a three-year True North Fund investment of 

up to $3.5 million in grants from the Edna McConnell Clark Foundation and the Social Innovation Fund, matched by $3.5 million from True North Fund co-investors, to further expand its 

1

U.S. Department of Education, National Center for Education Statistics (2013). 2

Reading Partners (2013).
 

1 
 

 

early-intervention literacy program to elementary schools throughout the country and evaluate 

its effectiveness. Box 1.1 contains more information about the EMCF Social Innovation Fund. 

A policy brief released by MDRC in June of 2014 reported the initial findings from 

the evaluation, which was conducted during the 2012-2013 school year in 19 schools in three 

states and involved more than 1,100 students. It found positive impacts of the program on 

three different measures of reading proficiency.3 The findings suggest that Reading Partners 

can be a valuable source of support for elementary grade students who are struggling with 

reading in underresourced schools and deserves wider use in school reform efforts to help improve reading proficiency. 

This report builds on those initial findings by describing the Reading Partners program 

and its implementation in greater detail, exploring whether or not the program is more or less 

effective for particular subgroups of students, and assessing some of the potential explanations 

for the program’s success. In addition, this report includes an analysis of the cost of implementing the Reading Partners program in 6 of the 19 sites; it additionally compares the cost of Reading Partners with the cost of the other supplemental reading services that these six schools 

offered to their students. 

The report concludes that the program is effective for a wide range of student subgroups, including students who are officially designated by the school to be English language 

learners, students starting the program with the weakest skills, and both boys and girls. Furthermore, although Reading Partners is a resource-intensive program involving the time of 

many individuals, the volunteers themselves account for much of the cost of the program. 

Thus, the cost of implementing the program for schools and districts is quite low. In fact, the 

required school contribution for Reading Partners is substantially lower than it is for the other 

supplemental reading services that are available at the schools, precisely because the volunteers account for a large proportion of the program’s costs. Although further research is warranted, the findings suggest that the regular one-on-one attention that the students in the 

program group received may have been one of the key contributors to the positive impacts of 

the program. 

The Policy Context 

Reading Partners is being implemented and evaluated in a national context in which increasing 

emphasis is being placed on interventions that are designed to improve reading instruction for 

3

Jacob, Smith, Willard, and Rifkin (2014). 

2 
 

 

 

 

 

Box 1.1 

The EMCF Social Innovation Fund 

The Edna McConnell Clark Foundation (EMCF), in collaboration with MDRC and The 

Bridgespan Group, is leading a Social Innovation Fund (SIF) project that aims to expand the 

pool of organizations with proven programs that can help low-income young people make 

the transition to productive adulthood. The SIF, an initiative enacted under the Edward M. 

Kennedy Serve America Act, targets millions of dollars in public-private funds to expand 

effective solutions across three issue areas: economic opportunity, healthy futures, and youth 

development and school support. This work seeks to create a catalog of proven approaches 

that can be replicated in communities across the country. The SIF generates a 3:1 privatepublic match, sets a high standard for evidence, empowers communities to identify and drive 

solutions to address social problems, and creates an incentive for grant-making organizations 

to target funding more effectively to promising programs. Administered by the federal Corporation for National and Community Service (CNCS), the SIF is part of the government’s 

broader agenda to redefine how evidence, innovation, service, and public-private cooperation can be used to tackle urgent social challenges. 

The EMCF SIF is particularly focused on young people who are at greatest risk of failing or 

dropping out of school or of not finding work, who are involved or likely to become involved 

in the foster care or juvenile justice system, or who are engaging in risky behavior such as 

criminal activity or sexual activity that could lead to teenage pregnancy. 

EMCF, with its partners MDRC and Bridgespan, selected an initial cohort of nine programs 

and a second cohort of three programs to receive SIF grants: BELL (Building Educated Leaders for Life), Center for Employment Opportunities, Children’s Aid Society—Carrera Adolescent Pregnancy Prevention Program, Children’s Home Society of North Carolina, Communities in Schools, Gateway to College Network, PACE Center for Girls, Reading Partners, The 

SEED Foundation, WINGS for Kids, Youth Guidance, and Children’s Institute, Inc. These organizations were selected through a competitive process based on prior evidence of impacts on 

economically disadvantaged young people, a track record of serving young people in communities of need, strong leadership and a potential for growth, and the financial and operational 

capabilities necessary to expand to a large scale. 

The EMCF Social Innovation Fund initiative, called the True North Fund, includes support 

from CNCS and 15 private co-investors: The Edna McConnell Clark Foundation, The Annie 

E. Casey Foundation, The Duke Endowment, The William and Flora Hewlett Foundation, The 

JPB Foundation, George Kaiser Family Foundation, The Kresge Foundation, Open Society 

Foundations, The Penzance Foundation, The Samberg Family Foundation, The Charles and 

Lynn Schusterman Family Foundation, The Starr Foundation, Tipping Point Community, The 

Wallace Foundation, and the Weingart Foundation. 

3 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

early elementary school students. Several national initiatives have been instituted in recent years 

(such as the Reading Excellence Act of 1997 and Reading First) in an attempt to provide 

schools with better resources to support their struggling elementary students. In addition, a 

number of states have either enacted or are considering enacting legislation regarding third 

grade reading proficiency, with 14 states and Washington, DC, having adopted laws that require 

students to be retained if they are not reading at grade level by the end of third grade.4 This policy environment has implications for interpreting the results of the evaluation because it affects 

the degree to which other supplemental services are available to students in the study. 

The No Child Left Behind Act of 2002 requires that schools provide low-income families with extra academic assistance or supplemental education services if their children are 

attending a Title I school that is in Program Improvement.5 These supplemental education services come at no cost to families and can include tutoring or remedial help in subjects like reading, language arts, and math, as long as they are aligned with state content standards and 

grounded in high-quality research that provides evidence of their effectiveness.6 Students can 

receive this extra help, which is often provided by external programs and vendors, before or 

after school, on weekends, or in the summer.7 

Many of these supplemental education services use Response to Intervention (RtI) 

models, which provide a means of identifying and supporting struggling students. Under RtI, an 

approach for which funding was authorized under the Individuals with Disabilities Education 

Improvement Act of 2004, all students are monitored and assessed for placement into a multitiered system of intervention. Students who are identified as struggling learners are supported 

by an array of interventions at increasing levels of intensity. Increasing the intensity of an intervention can be accomplished in a number of ways, including conducting an intervention more 

frequently or adding to its duration, relying on more specialized instructors, or pulling students 

into smaller groups (“pull-out” programs). The progress of each at-risk student is monitored 

closely using research-based assessments to make decisions about intervention duration and 

intensity. Tier I, usually classroom instruction for all students, is considered the first level of 

intervention, with Tier II and Tier III being progressively more intense.8 For example, daily 

one-on-one work with a specially trained reading interventionist is considered a Tier III inter4

Rose (2012).

5

Title I schools are those that receive federal funds based on the number of low-income children in attendance. Funds are used to improve academic achievement in reading and math in a variety of ways — offering 

teacher professional development, hiring additional teachers, or modifying curricula. A school in Program 

Improvement has been designated by the state to be in need of improvement for more than one year (U.S. 

Department of Education, 2012). 6

Heinrich, Meyer, and Whitten (2010).

7

U.S. Department of Education (2012).

8

Fuchs and Fuchs (2006). 

4 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

vention. Reading Partners is typically considered a Tier II intervention (for students needing 

some additional support), although some schools report using Reading Partners as a Tier III 

intervention.9 Reading Partners is usually one among multiple Tier II interventions that are 

available for struggling readers at any given school. 

Evidence-Based Approaches to Improving Literacy 

in Elementary School 

A variety of interventions exist to help students who are learning to read in elementary school. 

Among them, two models — whole-school reform and one-on-one tutoring by trained teachers 

— have received widespread attention, in part because they have been rigorously evaluated at 

scale (meaning that they have been tested on hundreds of students in multiple schools) and 

found to be effective in improving reading skills. A third model, tutoring by volunteers or 

paraprofessionals, used by Reading Partners, has received less scrutiny. 

Whole-School Reform 

Whole-school reform involves schoolwide changes to the curriculum, assessments, and 

interventions for struggling students in an attempt to change a school’s instructional culture and 

practices. The approach is exemplified by the Success for All program, in which teachers implement a highly structured reading curriculum, cooperative learning techniques, whole-school 

improvement practices, frequent assessments, and daily one-on-one tutoring for struggling students. The Success for All program has been widely evaluated and has been involved in two 

large-scale randomized controlled trials, all of which have found positive impacts of the program on students’ reading skills.10 

One-on-One Tutoring Delivered by Trained Teachers 

One-on-one tutoring delivered by certified teachers has consistently demonstrated large 

impacts on the reading proficiency of struggling readers.11 Among the best known and most 

widely researched of these programs is Reading Recovery. Reading Recovery is a short-term 

early intervention designed to help the lowest-achieving 15 percent to 20 percent of readers in 

first grade by providing daily one-on-one tutoring. Reading Recovery teachers, who receive 

9

A 2011 Reading Partners survey asked principals to indicate how Reading Partners was used in their 

schools. Of the 32 principals who answered the question about whether they used Reading Partners as an RtI 

intervention in their schools, 81 percent indicated that they did. Of the 26 who answered a question about what 

tier they used Reading Partners tutoring services for, 50 percent said they used Reading Partners as a Tier II 

intervention, and another 35 percent said they used Reading Partners as a Tier II/III intervention. 10Borman et al. (2007); Quint et al. (2013); Quint et al. (2014).

11Slavin, Lake, Davis, and Madden (2011); May et al. (2013). 

5 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

extensive and ongoing training, provide participating students with daily 30-minute lessons tailored to individual student needs over a period of 12 to 20 weeks. 

Tutoring by Volunteers or Paraprofessionals 

While the whole-school reform approach and one-on-one tutoring by trained teachers 

each has a solid research base demonstrating its effectiveness, they are both time- and resourceintensive ways to ameliorate the problem of low literacy, and as such may not always be viable 

options for already underresourced schools. 

A third, less validated approach involves using volunteers or paraprofessionals, as 

opposed to trained teachers, to provide tutoring to struggling readers. Tutoring by volunteers 

or paraprofessionals has the potential to be a less costly method for addressing the problem of 

low literacy, but to date, little rigorous evidence exists regarding the efficacy of this approach. 

Although available evidence on the effectiveness of one-on-one tutoring provided by volunteers is encouraging, there is currently only limited evidence that such programs can be implemented effectively on a large scale. Among seven studies of volunteer tutoring programs 

that were reviewed in a meta-analysis, only two were evaluated experimentally with intervention groups larger than 100 students. Only one of those two, the Experience Corps program, 

was found to have positive impacts.12 Since then, Minnesota Reading Corps has also demonstrated effectiveness at scale.13 Furthermore, prior research on the implementation and effectiveness of programs that rely largely on volunteers generally suggests that bringing them to 

scale might be quite difficult.14 

The evaluation described in this report is designed to bring more information to bear on 

the potential effectiveness of such volunteer tutoring programs. This report reflects a third rigorous evaluation of a volunteer tutoring program delivered on a large scale (the other two being 

the evaluations of Experience Corps and Minnesota Reading Corps, described above), and the 

Reading Partners program is unique among these evaluations in that it relies on community volunteers who devote only a few hours each week to tutoring (as opposed to Minnesota Reading 

Corps, which primarily relies on AmeriCorps members who commit to tutoring full time or half 

time, and Experience Corps, an AmeriCorps affiliate in which senior-age volunteers commit up 

to 15 hours a week to tutoring).15 It is intended to provide rigorous evidence about the efficacy 

of the Reading Partners program implemented at scale. Furthermore, it assesses the cost of 

12Slavin, Lake, Davis, and Madden (2011).

13Markovitz, Hernandez, Hedberg, and Bilberglitt (2014). 14Grossman and Furano (1999); Hager and Brudney (2004). 15AARP Experience Corps (2014); Markovitz, Hernandez, Hedberg, and Bilberglitt (2014).
 

6 
 

implementing a volunteer tutoring program and explores the cost of implementing the program 

from the perspective of a school. 

Overview of the Evaluation 

To answer questions about the effectiveness of the Reading Partners program and the potential 

for volunteer tutoring more generally to help improve the reading proficiency of struggling 

readers, this evaluation includes an implementation study, an impact study, and a cost study. 

The implementation and impact studies included 19 schools with Reading Partners sites in three 

different states, and the cost study included a subsample of six of those sites. Together, these 

three facets of the evaluation are designed to address the following broad research questions: 

1.	 In what context was the Reading Partners program implemented, and was it implemented as intended or with fidelity (that is, with adherence to the program 

model as laid out by the program developers)? How much variability in fidelity of 

implementation was observed across the sites? What factors contributed to any 

observed variability? 

2.	 On average, did the Reading Partners program have a positive impact on students’ 

reading proficiency across three key components of early reading ability: sightword efficiency, reading fluency, and comprehension? 

3.	 What resources are needed to implement the Reading Partners program as described in this report and what proportion of the costs of implementing the program are borne by the school? 

Implementation Study 

The implementation study included site visits to all the schools participating in the 

evaluation in the winter of the study year, interviews with key program and school staff and 

volunteers, and the collection of programmatic data from the sites and from Reading Partners’ 

own management information system. It was designed to enable the study team to describe the 

program and the context in which it was implemented in detail and to assess whether the program was delivered as intended. In addition, it was designed to explore whether any specific 

factors helped facilitate the implementation of the program or posed barriers to effective implementation, and how sites addressed those challenges. Implementation studies such as this one 

are essential for understanding the results of impact evaluations and contribute valuable information on the generalizability of the findings, the feasibility of implementing the program in 

other locations, and elements that might facilitate or strengthen future implementation. 

7 
 

 

 

 

 

 

 

 

 

 

 

 

Impact Study 

The impact study was designed to test the effect of the Reading Partners program on 

student reading proficiency — that is, to determine whether the tutoring provided by the Reading Partners’ volunteers increased the reading skills of the students who participated. The study 

used a student-level randomized controlled trial design, in which students were randomly assigned within each school to either a program group that would participate in Reading Partners 

during the 2012-2013 school year or an “as is” control condition without Reading Partners. 

Random assignment helps ensure that there are no systematic differences between the two 

groups of students at baseline, so that any positive effects can be causally attributed to the program with a high degree of confidence. Any differences between the outcomes of the program 

and control groups at the end of the study period are considered impacts of the program. A total 

of 1,265 students in grades 2 through 5 across the 19 schools were randomly assigned (646 to 

the program group and 619 to the control group). Although Reading Partners typically serves 

students in kindergarten through grade 5, the funding priorities of the granting institution, 

EMCF, typically focus on older children and adolescents. For that reason, the decision was 

made to include only second- through fifth-graders in this evaluation. 

At the start of the school year, three assessments were given to all students in the study 

sample. The assessments measured reading comprehension, fluency, and sight word efficiency, 

all key components of early reading.16 These same three assessments were administered to students again in the spring, as close to the end of the school year as possible. 

Cost Study 

The cost study describes the cost of implementing the Reading Partners program during 

the evaluation, based on a comprehensive list of the resources (or “ingredients”) used in implementation, and analyzes who bears the burden of financing those costs. This method works by 

calculating an intervention’s total cost by summing the cost of all the resources needed to implement the program, regardless of who financed them.17 The analysis uses the same approach 

to examine the costs of the other supplemental reading services at a sample of study schools. 

This approach provides a rich description of the costs of Reading Partners, as well as the costs 

of the other supplemental reading services in a sample of the evaluation schools, and allows for 

a detailed analysis of who absorbs those costs and how those costs vary across sites. 

16Reading comprehension was assessed using the Stanford Achievement Test 10th Edition (SAT-10) reading comprehension subtest; fluency, the ability to read with speed and accuracy, was assessed using the 

AIMSweb one-minute oral reading fluency subtest; and sight word efficiency, the ability to quickly identify 

commonly used words without going through the process of decoding, was assessed using the Test of Word 

Reading Efficiency, 2nd Edition (TOWRE-2) sight word reading subtest. 17Levin (1975); Levin and McEwan (2001). 

8 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

For budgetary reasons, only a subsample of schools could be included in the cost study. 

The team selected sites with strong implementation, that were geographically representative of 

the other sites in the study, and where reliable data could be collected on the resources used in 

Reading Partners and other supplemental reading services offered during the 2012-2013 school 

year. In order to execute this component of the study, MDRC partnered with the Center for 

Benefit-Cost Studies of Education at Teachers College, Columbia University. Cost studies like 

these are essential for helping researchers, practitioners, and programs themselves understand 

what resources are required for implementing a program. 

Evaluation Sample of Schools 

The sample for this study consists of 19 schools with Reading Partners reading centers 

that were recruited for the evaluation after Reading Partners senior staff identified them as potential participants. Only schools in which Reading Partners had been operating for at least one year 

before the study began were eligible, although the staff in these schools included both new and 

returning staff members. Schools were also eligible to participate only if Reading Partners anticipated that they would not be able to provide services to all the students who needed help with 

reading. This stipulation ensured that each program site was able to serve the same number of 

eligible students as it typically would, and did not deny services as a result of random assignment. A total of 31 schools were contacted during recruitment, and 19 of them joined the study. 

The 19 participating schools were spread across 12 school districts and 6 Reading Partners programmatic regions (New York City; Washington, DC; and, in California, East Bay, 

South Bay, Sacramento, and Los Angeles).18 There were 16 schools in California, 2 in New 

York, and 1 in Washington, DC. All of the schools were established Reading Partners sites; 8 of 

the 19 schools were in their second year of operating a Reading Partners center, while the rest 

had been operating a Reading Partners center for at least three years. The schools in the Reading 

Partners study sample had high numbers of students receiving free or reduced-price lunch (a 

proxy for coming from a low-income family), minority students, and English language learners. 

The majority of the participating schools were schoolwide Title I schools. Eight of the study 

schools were in varying stages of federal School Improvement status and two were in the final 

year of a three-year School Improvement Grant (grants to state education agencies to support 

school improvement goals in the nation’s lowest-performing schools). 

Table 1.1 shows the characteristics of the study sample compared with other Reading 

Partners schools that were not included in the sample, with all federally designated Title I 

18While 19 schools participated in the study, there were only 17 distinct Reading Partners centers. In the 

study, two sets of two colocated schools shared a reading center. 

9 
Reading Partners Evaluation
 

Table 1.1
 

Characteristics of Reading Partners Study Schools and Other School Samples (2011-2012)
 

Characteristic 

Study 

Schools 

a Other Reading Partners Schools U.S. Title I Schoolsb 

Mean Difference P-Value Mean Difference P-Value 

c Average U.S. Schools

Mean Difference P-Value 

Eligible for Title I program (%) 88.89 96.20 -7.31 0.209 100.00 -11.11 *** <0.001 75.39 13.49 0.184 

Students eligible for 

free/reduced-price lunchd (%) 81.70 84.52 -2.82 0.411 69.06 12.65 *** 0.008 53.47 28.23 *** <0.001 

Race/ethnicity (%) 

Black 20.74 28.44 -7.71 0.304 20.74 0.00 1.000 15.92 4.82 0.404 

Hispanic 61.75 52.95 8.80 0.204 29.83 31.92 *** <0.001 23.34 38.40 *** <0.001 

Asian 9.60 8.30 1.29 0.639 3.43 6.16 *** 0.002 4.67 4.92 ** 0.025 

White 5.70 7.28 -1.58 0.496 41.28 -35.58 *** <0.001 51.60 -45.90 *** <0.001 

Other 10 2.22 3.03 -0.81 0.336 4.73 -2.51 0.263 4.47 -2.25 0.234 

Male (%) 51.89 51.82 0.07 0.970 53.54 -1.64 0.462 53.00 -1.1 0.564 

Average number of students 494 451 43 0.318 468 26 0.622 472 22 0.677 

Grade 2 79 69 10 0.169 68 10 0.217 70 8 0.331 

Grade 3 76 65 11 0.142 68 8 0.384 70 5 0.554 

Grade 4 74 65 9 0.227 67 7 0.396 70 5 0.591 

Grade 5 76 64 12 0.131 67 9 0.290 69 6 0.488 

School settinge (%) 

Urban 73.68 91.67 -17.98 ** 0.027 39.77 33.91 *** 0.003 31.77 41.91 *** <0.001 

Suburban 26.32 8.33 17.98 ** 0.027 21.28 5.03 0.592 29.50 -3.19 0.761 

Town 0.00 0.00 0.00 NA 10.07 -10.07 0.145 8.79 -8.79 0.176 

Rural area 0.00 0.00 0.00 NA 28.87 -28.87 *** 0.005 29.94 -29.94 *** 0.004 

Sample size 19 84 24,914 42,202 

(continued) 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 1.1 (continued) 

SOURCE: MDRC calculations from 2011 and 2012 National Center for Education Statistics Common 

Core of Data (CCD). 

NOTES: Sample sizes for individual outcomes may fall short of the reported sample sizes because of 

missing or unusable data. 

A two-tailed t-test is used for all statistical tests presented in this table. Statistical significance 

levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

a

"Other Reading Partners Schools" include all other nonstudy Reading Partners schools that meet 

the "Average U.S. Schools" criteria below.

b"U.S. Title I Schools" include all non-Reading Partners schools that meet the "Average U.S. 

Schools" criteria below and were all designated Title I schoolwide schools. c

"Average U.S. Schools" include non-Reading Partners schools that offer grade 2 through grade 5, 

are defined as "regular" schools by the CCD, and are located within the 50 U.S. states and the District 

of Columbia. 

dThe value given for students eligible for free/reduced-price lunch is calculated from the 2011 CCD 

because data are missing in the 2012 CCD. Data for all other variables are from the 2012 CCD. e

"Urban" is defined as territory inside an urbanized area and inside a principal city having a 

population greater than 100,000. "Suburban" is defined as territory outside a principal city and inside 

an urbanized area with a population of less than 250,000. “Town” is defined as territory inside an 

urban cluster that is (1) less than or equal to 10 miles from an urbanized area (“fringe”), or (2) more 

than 10 miles and less than or equal to 35 miles from an urbanized area (“distant”), or (3) more than 35 

miles from an urbanized area (“remote”). “Rural” is defined as territory that is (1) less than or equal to 

5 miles from an urbanized area and less than or equal to 2.5 miles from an urban cluster (“fringe”), or 

(2) more than 5 miles but less than or equal to 25 miles from an urbanized area and more than 2.5 

miles but less than or equal to 10 miles from an urban cluster (“distant”), or (3) more than 25 miles 

from an urbanized area and more than 10 miles from an urban cluster (“remote”). 

schools in the United States, and with all elementary schools in the United States.19 The Reading Partners study schools were somewhat less urban (and more suburban) than Reading Partners schools as a whole, but there were no other statistically significant differences between the 

two types of schools. (This discussion refers only to statistically significant differences — that 

is, differences that are unlikely to have arisen by chance.) The Reading Partners study schools 

include a higher percentage of Hispanic students than the average Title I school, reflecting the 

large concentration of California schools in the study. The study sample also includes more urban schools than are represented among all schoolwide Title I schools, since Reading Partners 

does not attempt to serve small, rural schools, for which the number of available volunteers is 

quite limited. Consistent with the Reading Partners model, the Reading Partners schools in the 

study sample are poorer, more urban, and have higher percentages of minority students than the 

19Title I schools are eligible for additional funding because of their high proportion of students from lowincome families. 

11 
 

 

 

population of U.S. schools as a whole. Thus, the sample of schools included in the evaluation 

appears to be broadly representative of Reading Partners sites across the country. 

▪ ▪ ▪ 

The remainder of this report is organized as follows: Chapter 2 provides a detailed 

overview of the Reading Partners program. Chapters 3, 4, and 5 present the findings from the 

implementation, impact, and cost components of the study, respectively, and Chapter 6 offers 

some interpretations and conclusions. 

12 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

Chapter 2 

The Reading Partners Program 

This chapter provides a detailed description of the Reading Partners program model as it was 

implemented during the 2012-2013 school year, including its staffing structure and core components. The information provided in this chapter is a combination of the model as described by 

Reading Partners and as understood from data collected over the course of this study. Reading 

Partners has continued to innovate and refine its program model since that time and thus this 

description may not fully reflect current operating practices. Figure 2.1 provides an overview of 

the program in its ideal implementation. 

Reading Partners is a “pull-out” program that operates both during the school day and 

after school.1 The highly structured, modular curriculum is delivered by volunteer tutors on a 

one-on-one basis in 45-minute sessions, twice a week. Each school that participates in the Reading Partners program has a designated “reading center” where the tutoring takes place. Nationally, the average student is enrolled in the program for more than five months, and many students 

participate in the program for a full school year or longer. The curriculum used by Reading 

Partners during the 2012-2013 study year was research-based and aligned with California state 

content standards. From 2012 to 2014, Reading Partners has worked to systematically align the 

curriculum with the Common Core State Standards, which establish a single set of suggested 

national educational standards for students in kindergarten through twelfth grade in English language arts/literacy and mathematics.

2 

Target Population and Selection Criteria 

Reading Partners aims to serve students in kindergarten through grade 5 in underresourced 

schools who are half a year to two and a half years behind grade level in reading. In addition, 

Reading Partners aims to serve students who are conversant in English and do not have any 

special needs (that is, do not have an Individualized Education Program).3 Many Reading Partners students are in fact designated by their schools as English language learners (ELLs) — 

1

“Pull-out” refers to a type of program that takes students out of the mainstream classroom or after-school 

program for a portion of the day, with the goal of providing specialized instruction. 2

Since 2010, 46 states have adopted the Common Core (National Council of State Legislatures, 2014). 3

Individualized Education Programs are developed for children who are found through assessment to have 

a disability that affects their learning process. The program outlines how teachers will help these students learn 

more effectively considering each of their learning styles and needs. 

13 
Schools refer eligible students to Reading Partners. 

Reading Partners targets students who: 

• Are 6-30 months (2.5 years) below grade level in reading 

proficiency 

• Have at least conversational English skills 

• Do not have an Individualized Education Program 

Reading Partners administers an assessment to confirm 

target students and place them into the curriculum. 

Tutors are then matched with students. 

Students participate in twice-weekly Reading 

Partners sessions that include: 

• One-on-one tutoring from a volunteer tutor 

• Data-driven instruction, monitored by the site 

coordinator 

• Structured, individualized curriculum 

• Instruction in a dedicated space 

Reading Partners administers a midyear student 

assessment to monitor student progress and modify 

the student’s Individualized Reading Plan. 

Student reading proficiency 

measured at end of year to 

determine overall growth. 

Reading Partners staff and 

volunteer tutors receive 

ongoing training. 

Site coordinators and volunteer 

tutors receive instructional 

supervision and support. 

Reading Partners Evaluation
 

Figure 2.1
 

Reading Partners Program
 

14
 
 

however, ELL students who are enrolled in Reading Partners are generally conversationally 

fluent in English. 

Students are referred to Reading Partners by teachers or principals.4 At each site, teachers, principals, and reading specialists identify students who are reading below grade level and 

consider the different reading interventions that are available to students. Often, Reading Partners is one program on a menu of supplemental services that are available to students in underresourced schools, particularly if the school has been designated under the No Child Left Behind act to be “in need of program improvement” for more than one year.5 School staff members often use a combination of information to determine whether or not a student should be 

referred to Reading Partners, including students’ state standardized test scores, students’ performance on local assessments, academic performance in the classroom, and behavior (based on 

teacher observation). 

Reading Partners’ Organizational Structure, Staff, 

and Volunteer Tutors 

An overview of the Reading Partners staffing structure that supports program delivery at each 

Reading Partners center is provided in Figure 2.2. Each Reading Partners center is managed by 

a site coordinator. Site coordinators oversee the instruction that the volunteer tutors provide and 

the day-to-day operations of the Reading Partners program at school sites, managing 40 to 100 

volunteer tutors over the course of the year. The number of active volunteer tutors at any one 

point in time, however, can vary significantly. In addition, site coordinators provide training and 

support to volunteer tutors; use Reading Partners materials, tools, and activities to ensure that 

students’ learning needs are met; and serve as liaisons with school staff (teachers, reading coordinators, administrators, and so forth). Site coordinators also tutor students directly, especially 

when a substitute tutor is needed and there are more students than available tutors. 

Site coordinators are almost exclusively AmeriCorps members and most are recent 

college graduates.6 The site coordinator position is intended to be a two-year post. All site 

4

To accommodate the study, school staff members were asked to identify and refer a pool of eligible students to Reading Partners earlier in the school year than usual. From this pool, all eligible students were randomly assigned into the program, rather than hand-picked and enrolled into the program on an ongoing, rolling 

basis, as is typical. 5

U.S. Department of Education (2012).

6

AmeriCorps is a program of the Corporation for National and Community Service that places young 

adults in service positions at nonprofit organizations, schools, public agencies, and faith-based entities. In return for their service, AmeriCorps members receive a living stipend, health insurance, and, when they complete 

their program, an education grant (Corporation for National and Community Service, 2014). 

15 
Reading Partners Evaluation
 

Figure 2.2
 

Staffing Support Structure for a Reading Partners Center
 

Regional Executive 

Director 

Program Manager 

Site Coordinator 

Volunteer 

Tutors 

(40-100 

per site) 

Outreach 

Coordinator 

NOTE: Within Reading Partners, regional executive directors supervise multiple program 

managers, and program managers supervise multiple site coordinators. Additionally, 

outreach coordinators are responsible for recruiting volunteers for multiple site 

coordinators. 

16 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

coordinators report to and are supported by program managers, who often have classroom 

teaching experience. Program managers are charged with being the primary literacy program 

expert for a portfolio of school sites and manage a team of up to five site coordinators. The program manager’s roles and responsibilities include building and cultivating relationships with 

schools, districts, and other community organizations; conducting monthly training sessions for 

volunteer tutors and site coordinators; ensuring that Individualized Reading Plans (IRPs) for all 

students are created; and ensuring that data are used to maximize students’ reading skills development. Another key responsibility of program managers is to visit the reading centers in their 

portfolio of school sites regularly to monitor the implementation of the programs at each site, 

and to provide support where needed — by, for example, communicating with school staff. 

Outreach coordinators also play a key role in the Reading Partners model. However, 

their primary role does not involve direct interaction with students. They are responsible for recruiting volunteers year-round in order to meet the need for volunteers at each program site. 

Outreach coordinators, who are also AmeriCorps members, oversee the entire recruitment process and are responsible for initiating contact with every individual who expresses interest in 

volunteering for Reading Partners. As part of the recruitment effort, outreach coordinators are 

also charged with identifying and establishing relationships with partner organizations such as 

universities, religious institutions, and businesses that can provide a steady stream of tutors. Historically, high schools and postsecondary institutions have served as the largest source of tutors. 

Outreach coordinators and program managers report directly to a regional executive 

director, a senior member of the organization. Regional executive directors oversee programs 

within a given region and are charged with supporting tutor recruitment, ensuring school support, and engaging in additional fundraising. 

Volunteer tutors are responsible for providing one-on-one tutoring to Reading Partners 

students and implementing the Reading Partners curriculum. Tutors at the study schools often 

learned about Reading Partners through their schools, through community organizations, or 

through online avenues. Reading Partners publicizes tutoring opportunities on a variety of websites, including Volunteer Match, Volunteer Center, Go Volunteer, Christian Volunteering, Idealist, and Craigslist. Others learned about Reading Partners through friends or community organizations. All tutors are asked to make a one-semester commitment (generally around four 

months), and as part of that commitment, to identify regular days and times that they can come 

in to work with one or more students every week. Additionally, tutors must pass a background 

check that allows them to work with students. 

No prior experience working with children is necessary to become a Reading Partners 

tutor, and as such the tutor pool is diverse. Tutors range in age (14 to 70 years of age and older), 

17 
 

 

 

 

 

Reading Partners Evaluation
 

Figure 2.3
 

Backgrounds of Tutors in Study Schools
 

11.1% 

17.2% 

35.6% 

17.6% 

8.1% 

10.3% 

High school students College or graduate students 

Working adults Retired adults 

Other Missing 

SOURCE: MDRC calculations from Reading Partners management information 

system data. 

NOTE: Tutors characterized as "Other" include, but are not limited to, caregivers,

nonworking individuals, and those who describe themselves as "transitioning." 

gender, and race/ethnicity, and are represented by people from many walks of life. As shown 

in Figure 2.3, about 36 percent of tutors included in this study were college or graduate students, 18 percent were working adults, 17 percent were high school students, 8 percent were 

retired adults, and 10 percent were caregivers, unemployed, or making the transition from being unemployed to working; a final 11 percent did not provide this information. However, as 

shown in Appendix Table C.1, the tutor pool composition within each site varied widely, with 

some sites being composed almost entirely of college or graduate students (up to 83 percent in 

one site) and other sites having large proportions of working adults (up to 40 percent) or high 

school students (up to 31 percent). Typically, the composition of the tutor pool was influenced by proximity to, and relationships built with, local colleges or high schools, and access 

to public transportation. 

18 
 

 

 

 

 

 

 

 

 

 

 

 

 

A little less than half (22 of the 49) of the tutors who were interviewed for the study had 

some experience working with children in some capacity, whether it was leading a church youth 

group, helping in an after-school program, working at a children’s summer program, or previous teaching experience. However, interviews with volunteer tutors also revealed that fewer 

(11) had experience in an organized tutoring program. 

Core Components of Reading Partners 

The Reading Partners model consists of six core components, each described in turn below. The 

core components are (1) regular, one-on-one tutoring; (2) dedicated school space and use of materials; (3) a structured and individualized curriculum; (4) data-driven instruction; (5) rigorous 

and ongoing training for staff and volunteers; and (6) instructional supervision and support. 

Regular, One-on-One Tutoring 

A key feature of the Reading Partners model is to provide students with individualized 

reading instruction. As such, volunteer tutors deliver the Reading Partners curriculum to students on a one-on-one basis. The one-on-one support — as opposed to small-group support — 

is the main component of the program that sets Reading Partners apart from other supplemental 

services that are available to struggling readers. The one-on-one sessions are intended to be delivered twice a week for 45 minutes by a volunteer tutor. While not explicitly part of the model, 

the assumption is that a student will also benefit from a supportive relationship with a caring 

adult, suggesting that consistency is an implicit goal. However, as described in more detail below, tutors often visit schools only once a week for up to a few hours (working with one student 

for 45 minutes or multiple students consecutively), so many students are paired with two different tutors throughout the school year, typically meeting with one tutor on the first day and another tutor on the second day. 

Dedicated School Space and Use of Materials 

The Reading Partners tutoring is designed to take place on school grounds in a dedicated tutoring space — typically a designated classroom — that is transformed into a reading center. Every reading center features a “read-aloud” library where students and tutors can select 

books at different levels of difficulty, a corner of the room with couches or bean bags for readaloud sessions, work stations for tutor-student pairs, a “take-home reading” area where students 

can select books to read at home, and reading-themed materials on the walls. In addition, Reading Partners provides each site coordinator with a list of items that all Reading Partners centers 

are expected to have — for example, a resource table for volunteers, a “word wall” showing 

common words for students to use as a reference, and a bulletin board to display student work. 

19 
 

 

 

 

 

 

 

 

 

These components and materials are designed to create an engaging learning environment in 

which multiple tutor-student pairs can be working at the same time. 

Structured and Individualized Curriculum 

The Reading Partners curriculum that was implemented during the study year was 

modularized, with each lesson following a consistent structure. The curriculum consisted of two 

modules: a beginning readers module (consisting of one level) and a comprehension readers 

module (consisting of four levels). 

The beginning readers module (Level 1) contained 50 lessons, which were primarily 

phonics-based and covered various letter-sound combinations and practice with high-frequency 

words. Typically, students were able to get through one beginning readers lesson in one 45-

minute session. However, some students in this module took up to two sessions to complete a 

lesson. Supplemental lessons were also available for some lessons in case a student needed 

additional practice. 

The comprehension readers module comprised Levels 2, 3, 4, and 5, each of which consisted of 24 lessons. The lessons in this module focused on topics such as cause and effect, making inferences, fact and opinion, summarizing, and predicting. Typically, students finished a 

comprehension readers lesson in two to three sessions. 

The tutoring session typically began with the tutor picking up the student from the 

classroom. During the walk to the Reading Partners center, the tutor engaged in friendly conversation with the student, helping to build rapport and a personal connection. The lesson itself followed a consistent structure. Each lesson began with the tutor reading aloud from a text that the 

student chose. During that time, the tutor modeled fluent reading, paying close attention to tone, 

pronunciation, and punctuation. Tutors paused to ask the student comprehension questions or to 

check the student’s knowledge of key vocabulary. After the read-aloud, the tutor turned to the 

Reading Partners curriculum materials, which introduced a new skill or concept or reinforced a 

skill or concept that had been taught previously. The tutor worked with the student to complete 

the task in the curriculum materials. Finally, the student applied the new skill or concept while 

reading aloud from a text that was provided to support the lesson, and with support from the 

tutor. A description of a typical Reading Partners tutoring session can be found in Box 2.1. 

Data-Driven Instruction 

Reading Partners uses data to implement and support the model. For example, in 

determining whether or not a student should be referred to Reading Partners, a combination 

of data are considered: student performance on state standardized tests, performance on 

20 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Box 2.1 

A Reading Partners Tutoring Session 

1.	 Meeting the student (5 minutes). Tutors pick Reading Partners students up from their 

classrooms or after-school programs and walk with them back to the Reading Partners 

center. During that time, tutors talk with the students about how they’re doing and engage 

in caring and friendly conversation. This approach lays the groundwork for a healthy tutor-student relationship and helps to build rapport between the tutor and student. 

2.	 Tutor read-aloud (10 to 15 minutes). Once the tutor-student pair reaches the Reading 

Partners center, the lesson begins with a student choosing a book at the appropriate level 

from the Reading Partners library and finding a quiet space to sit with the tutor. The tutor 

then reads the book aloud to the student, modeling how to read expressively, pay attention 

to punctuation, and pronounce words correctly. During the read-aloud, the tutor pauses to 

ask the student open-ended questions and to discuss the book’s content and vocabulary. 

3.	 Introduction of a new skill or concept (10 to 15 minutes). The tutor introduces a new 

skill or concept and helps the student complete an introductory task. For example, a 

beginning student may learn about words with an “oa” sound, while a more advanced 

comprehension student may learn about the concept of cause and effect. 

4.	 Student read-aloud and application of a new skill or concept (15 to 20 minutes). While 

the student reads a literary or informational text aloud, the tutor provides support and encourages the student to apply the new skill or concept by asking targeted questions. 

school-administered assessments (for example, Developmental Reading Assessment, Basic 

Phonics Skills Test), and information provided by teachers on the Reading Partners referral 

form (such as observational assessments of a student’s level of comprehension or ability to 

use a knowledge of letters and sounds to read written words correctly — often referred to as 

“phonics”). 

After the teacher or principal refers the student to Reading Partners, the site coordinator 

or program manager administers the Rigby PM Benchmark assessment, an established tool that 

measures oral fluency, accuracy, and comprehension.7 Site coordinators and program managers 

receive training and a protocol for administration, and set aside practice time before conducting 

assessments one-on-one with students. Using a dedicated set of reading materials at various levels of difficulty, the trained administrator asks the student to read the book aloud. The administrator measures accuracy by recording any errors or corrections that the student makes during 

7

The Rigby is published by Houghton Mifflin Harcourt. 

21 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

the read-aloud. The student is also asked comprehension questions about the main idea of the 

story, primary characters, setting, and other salient features of the text. Oral fluency is assessed 

by asking the student to participate in a second read-aloud that is timed for one minute. Administration of the Rigby PM assessment takes approximately 30 minutes or less depending on the 

student’s reading level. However, several iterations of the assessment may be administered to 

correctly identify a student’s independent reading level and to accurately place the student within the Reading Partners curriculum. 

The Rigby PM assessment is administered again midway through the school year, and 

once more at the end of the school year. Each student’s progress is documented in an IRP, a 

document that contains a student’s Rigby scores at the beginning, middle, and end of the school 

year; goals for the student; and areas to focus on. After the midyear Rigby assessment, Reading 

Partners staff use the data to update the IRP, identify areas for growth, and adjust student 

placement where needed. For students who are struggling or are not making gains during the 

first half of the school year, a targeted action plan is developed. Students’ Rigby assessment 

data and progress are also shared with teachers and principals; some teachers also share these 

data with parents when report cards are distributed. 

Rigorous and Ongoing Training 

Reading Partners staff and AmeriCorps members undergo formal, instructional training 

before they are placed at a school. Formal training for Reading Partners staff takes two forms: 

organized training sessions held before the school year starts, and ongoing (usually monthly) 

sessions held throughout the school year. New program managers receive two weeks of training 

that provides an introduction to the Reading Partners program model, its history, core instructional beliefs, and culture, as well as rigorous training on the curriculum and other systems, such 

as how to manage internal databases throughout the program year. Experienced program managers and all site coordinators (both new and returning) also undergo intensive training before 

each school year begins. Topics that are covered during these training sessions include a detailed review of the curriculum, how to set up a Reading Partners reading center, how to train 

volunteer tutors, and how to use data to inform instruction. The ongoing sessions for Reading 

Partners staff, which are provided by more senior Reading Partners staff, are offered on specific 

topics and range from training that focuses on improving organization and relieving stress to 

how to better support beginning readers. 

For volunteer tutors, the primary training is a “shadow session,” which is a combination of orientation and tutoring observation led by the site coordinator. The entire training 

takes less than one hour and then tutors begin working with their own students. Ongoing 

training sessions, led by site coordinators, are also available to volunteer tutors but are not 

mandated by Reading Partners. Topics range from how to better support English language 

22 
 

 

 

 

 

 

learners to managing students’ behavior and attention. These sessions are offered on a regular 

basis, often twice a month. 

Instructional Supervision and Support 

Reading Partners staff provide instructional supervision and support on an ongoing 

basis. Site coordinators provide supervision, coaching, and support to volunteer tutors, and program managers provide support and guidance to site coordinators. 

Program managers provide instructional supervision and support to site coordinators in 

two ways. First, program managers provide technical support related to implementing the Reading Partners model. In this role, program managers work with site coordinators to troubleshoot 

problems on a range of issues, including challenges related to specific students and their progress or behavior, working with tutors and managing their performance and attendance, helping 

to manage communication with both teachers and administrators, and identifying best practices 

to better support a struggling reader. Second, program managers may also support site coordinators, who are often recent college graduates, by working with them on issues such as time management and organization. Program managers have regular check-in meetings with site coordinators, but also provide informal, impromptu support on an as-needed basis. 

Site coordinators also provide tutors with instructional supervision and support on an 

ongoing basis. As managers of the reading center, site coordinators are there every day and observe instruction as volunteer tutors work one-on-one with students. Site coordinators may provide suggestions and guidance on what reading or engagement strategies to use with a particular 

student, and may also sit in on a tutoring session to provide additional instructional support. 

These aspects of the Reading Partners program are revisited in Chapter 3. That chapter 

includes an assessment of the degree to which the program was implemented with fidelity to the 

model across the study sites and a discussion of both the stronger aspects and the challenges of 

program implementation. 

23 

Chapter 3 

Understanding Implementation Fidelity 

The primary goal of exploring implementation and measuring fidelity in this study is to assess 

the extent to which the Reading Partners model is implemented as intended across all 19

school sites.1 Measuring the program’s adherence to the model helps the reader understand

how the program impacts relate to implementation. Measuring fidelity also ensures that the

outcomes from a well-designed impact study like a randomized controlled trial can in fact be 

attributed to the program model itself.2 Finally, a thorough understanding of implementation

fidelity can help readers understand whether the findings are generalizable to other Reading

Partners locations. 

Specifically, this chapter explores three broad categories of implementation questions: 

•	 Context: What were the characteristics of the schools included in the study 

and the students served by Reading Partners in those schools? Who delivered 

the program, and how were they recruited and trained? How did the school 

setting affect program delivery? 

•	 Service delivery: Was the program delivered as intended — that is, with fidelity to the model? What were the barriers to and facilitators of program 

delivery? Did particular aspects of the model pose special problems? Were 

there categories of schools with noticeably stronger or weaker implementation, and did those differences affect program impact? 

•	 Implementation lessons: How did local contextual factors influence the implementation of the Reading Partners instructional model? What implementation challenges were encountered and how were they addressed? What are 

lessons for other programs that may wish to implement similar strategies? 

Data Collection 

To collect the data necessary to answer these questions, in the winter of 2013, two-person teams 

made one-day visits to each of the participating sites to assess the context in which the program 

 1

Other implementation studies may also focus on other dimensions of fidelity, such as quality of implementation and participant responsiveness (Dane and Schneider, 1998; Durlak and DuPre, 2008). Those dimensions were not a primary focus of this fidelity analysis. 2

O’Donnell (2008). 

25 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

was administered and to gauge the quality of program implementation. Seven members of the 

research team conducted these visits, with each member making between two and eight visits. 

Before they made their visits, the research team attended a half-day training session that included a review of all protocols, discussion of appropriate behavior in schools, and guidelines for 

conducting effective interviews. Written guidelines for the site visits were distributed before the 

training was conducted, which the research team members reviewed independently. 

The site visits included interviews with the program manager, site coordinator, outreach 

coordinator, regional executive director, school principal, school reading coordinator or reading 

specialist (if there was one), and approximately three tutors at each site.3 Site visitors also conducted short, unstructured observations of three tutoring sessions and filled out a “facilities 

checklist” for each site, in which they recorded whether the room was set up as intended by 

Reading Partners and whether or not it had all required materials. Finally, the site visit included 

a systematic review of 10 randomly sampled student folders. The student folders contained information about session attendance, the person who conducted the session, results of the 

assessments, the content of each tutoring session, and notes from tutors about what was covered 

in each session. An overview of the data that were collected as part of the implementation study 

is shown in Table 3.1, and copies of protocols from site visits are available upon request. 

To help augment the information collected during site visits, extant program data were 

obtained from Reading Partners’ own management information system (MIS). These data included information about individual student tutoring start and end dates, attendance, the number 

of different tutors assigned to a student over the course of the school year, and the background 

characteristics of both students and tutors. 

As described in more detail in Appendix A, interview transcripts, notes, and observation write-ups were analyzed using a systematic coding process. A coding scheme was developed to align with the overall framework of the implementation study and was organized to capture data in the three main categories outlined above, and team members were assigned a set of 

codes to apply across all of the qualitative data. Once all data had been coded, team members 

provided a written synthesis that described emerging themes. 

The discussion of findings begins with an overall assessment of implementation fidelity 

as measured by a study-developed fidelity index, and is followed by a more in-depth discussion 

of each component. The chapter concludes with a discussion of the factors that affected the implementation fidelity of the Reading Partners program. 

3

Some interviews were conducted by phone when an in-person interview was not possible. 

26 
Reading Partners Evaluation
 

Table 3.1
 

Summary of Implementation Data Collection, by Source
 

 

 

 

 

 

 

 

 

Number Number 

Data Source Targeted Collected 

Interviews 

Principal 19 19 

Reading coordinator 11 11 

Regional executive director 6 6 

Outreach coordinator 7 6 

Program manager 10 8 

Site coordinator 17 17 

Tutora 51 50 

Student folder reviewsb 170 171 

Reading center checklists 17 17 

Tutor observationsc 51 51 

SOURCE: Site visits to study schools. 

NOTES: aAt two schools, only two tutor interviews were conducted. At one school, four 

tutor interviews were conducted. 

bStudent folder reviews looked for documentation of student Rigby assessments, notes 

from tutoring sessions and on progress through the curriculum, and students' 

Individualized Reading Plans. At one site, 11 folders were reviewed. cAt three schools, only two tutor observations were conducted. At one school, four tutor 

observations were conducted, and at one other school, five tutor observations were 

conducted. 

Implementation Fidelity Measure 

Implementation fidelity was assessed along the six core components of the Reading Partners 

model, as described in detail in Chapter 2: (1) regular, one-on-one tutoring; (2) dedicated school 

space and use of materials; (3) structured and individualized curriculum; (4) data-driven instruction; (5) rigorous and ongoing training; and (6) instructional supervision and support. To help 

summarize and quantify the overall level of implementation fidelity across all 19 schools, the 

study team developed a fidelity index based on five of the six core components. The third component listed above — structured and individualized curriculum — was not incorporated into 

the index because the data collected were not appropriate for quantifying fidelity along this 

dimension. This component is discussed qualitatively. 

27 
The fidelity index contained 17 items that were created using quantitative data from the 

Reading Partners’ MIS; qualitative assessments of fidelity based on interviews with Reading 

Partners staff, school staff, and volunteer tutors; a review of student folders that are kept on site 

in reading centers; and a facilities checklist, which assessed whether the reading centers contained the required components. Criteria for each item were developed by the implementation 

study team in collaboration with Reading Partners. The respective weights given to each component of the fidelity index were determined by Reading Partners’ a priori assessment of the 

most important characteristics of the program and not necessarily what the research team concluded retroactively. For example, within the category of “dedicated school space and use of 

materials,” Reading Partners maintained that having a dedicated space that could not be shared 

was more important than having all required materials in place in the reading center. For this 

reason, having a dedicated space that did not have to be shared was weighted more heavily than 

having all required materials. Each Reading Partners school was scored across all 17 items. 

Two members of the research team and the project’s principal investigator reviewed all scoring 

decisions. Fidelity was assessed using data from the entire program year. A copy of the fidelity 

index, its key indicators, data sources, and criteria for scoring can be found in Appendix A. 

Overall, the fidelity index reveals that implementation fidelity across all 19 schools was 

relatively high, and that the program outcomes can likely be attributed to the Reading Partners 

program model. As shown in Figure 3.1, the fidelity scores for the 19 schools range from a low 

of 12 to a high of 22; the maximum possible score on the index is 23. Scores were grouped into 

three categories of implementation based on their overall distribution. Scores of 15 and below 

(less than 65 percent of the total points possible) were considered “low fidelity”; scores of 19 or 

higher (over 83 percent of the total possible) were considered “high fidelity”; and scores between those delineations (15.5 to 18.5) were considered “moderate fidelity.” Of the 19 sites, two 

schools were designated as low fidelity; nine schools had moderate fidelity; and eight schools 

fell into the high-fidelity category. Chapter 4 explores the extent to which this variability in 

fidelity explains variation in program impacts. 

Regular, One-on-One Tutoring 

The Reading Partners model states that all students who participate in the program 

should receive one-on-one tutoring, as opposed to small-group support, which is more typical of 

literacy programs across the country. The one-on-one support is the main component of the 

program that sets Reading Partners apart from other supplemental services that are available to 

struggling readers. As one regional executive director described, “I think a lot of the other 

organizations that we come across provide literacy in different formats and usually it’s in small 

groups, so I think that the one-on-one piece is critical.” 

28 
 

Reading Partners Evaluation 

Figure 3.1 

Fidelity Scores of Study Schools 

25
 

22 22
 

20 5 

18.5

19. 19.5 20
 
 19 19 19 18.5 

17 17 17 17 17 16.5 

e 

15.5
 

ro

14 c

15
 

 sy

12
 

liteid10
 F

5
 

0 

A B C D E F G H I J K L M N O P Q R S 

 

Study schoolsa 

Regular 1:1 tutoring Space and materials Data-driven instruction Training Supervision and support 

(maximum score = 3) (maximum score = 5) (maximum score = 5)	 (maximum (maximum score = 7) 

score = 5) 

29 

SOURCE: Site visits to study schools. 

NOTE: The fidelity index comprises metrics evaluating the five key quantifiable components of the Reading Partners model. The total maximum 

score is 23. A further explanation of the fidelity index and scoring guidelines can be found in Appendix A. a

To prevent identification of individual schools or programs, site letter identifiers are not consistent across analyses. 
Analysis of the MIS data and site coordinator interviews revealed that except for one 

school, tutoring was exclusively one-on-one at all sites. At one school, students sometimes met 

with a pair of tutors, who worked with the student at the same time — a strategy employed by a 

site coordinator to help manage the large number of volunteer tutors. The site coordinator at this 

site noted that a partnership with a local high school had in the past led to an influx of volunteer 

tutors that caused a “chaotic” environment in the after-school tutoring sessions. In response, the 

site coordinator paired the high school tutors in order to reduce the number of pairs she had to 

monitor from six one-on-one tutoring sessions to three two-to-one tutoring sessions. Other sites 

did not seem to have this challenge. 

In addition to one-on-one instruction, the Reading Partners model has a frequency component. The program is designed to provide students with two 45-minute tutoring sessions per 

week. As shown in Table 3.2, analysis of the MIS data reveals that, on average, students 

received approximately 1.5 sessions per week for 28 weeks (about three sessions every two 

weeks), which is slightly less than the model specifies. The MIS data indicate that students 

attended approximately 79 percent of the sessions that were available to them, after accounting 

for school closures and vacations. There was, however, variation in participation levels by 

school, with students in some sites maintaining an average of 1.8 sessions per week while those 

from others averaged just 1.1 sessions per week. Some site coordinators (4 of 17) noted that the 

students who did not receive tutoring twice a week were usually those who were absent from 

school. School attendance and the dosage of Reading Partners services that were received are 

correlated with one another, but the correlation is relatively low (around 0.18). Thus, lack of 

consistent attendance among volunteers also likely contributed to the problem of low frequency 

of participation in some instances.1 

While one-on-one support and frequency of tutoring are explicit goals of the model, one 

of the implicit goals of Reading Partners is that a student will benefit from a supportive relationship with a caring adult as a result of working consistently with the same tutor. However, since 

tutor consistency is an implicit goal of the model, it was not factored into the fidelity index and 

is thus discussed only descriptively in this section. 

Tutor consistency can be explored along three different dimensions — whether students 

were typically assigned to work with one or two tutors each week, the duration of the tutoring 

relationship, and how consistently the assigned tutors appeared for their regularly scheduled 

sessions. The MIS data indicate that students did not typically work with the same tutor for both 

of their weekly sessions. As shown in Table 3.2, about three-fourths of primary tutors (76 

 1

The research team could not assess the correlation between tutor attendance and dosage because of limitations of the data. 

30 
 

 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Table 3.2
 

Average Student's Experience in Reading Partners
 

Program School-Level Averages 

Average Student Experience Group Minimum Maximum 

Number of sessions per week 1.55 1.11 1.76 

Length of participation in program (weeks) 28.13 24.24 32.01 

Student attendance ratea

 (%) 78.76 55.75 88.98 

Number of tutors assigned 2.52 1.67 3.60 

Duration of each tutoring relationship (weeks) 19.81 11.20 26.01 

Scheduled sessions per week with primary tutorb (%) 

Scheduled once per week 76.38 39.58 91.24 

Scheduled twice per week 23.62 8.76 60.42 

Sample size 594 

SOURCE: Reading Partners management information system data. 

NOTES: Calculations for this table used data for all program group students in the respondent sample. 

Sample sizes for individual outcomes may fall short of the report sample size because of missing or 

unusable data. a

Student attendance is calculated as the number of sessions attended divided by the number of 

possible sessions, assuming two sessions every week that school was open after the student entered 

Reading Partners. 

bAmong students' assigned tutors, primary tutors were those who had the longest assigned 

relationship with the student. 

percent) worked with their students only once a week, and the remaining one-fourth (24 percent) worked with their students twice a week.2 One of the factors that made maintaining consistency with the same tutor difficult is that many tutors were available to volunteer only once a 

week, meaning that it was often not possible for a student to have the same tutor for both sessions each week. In the study schools, tutors generally came once a week — some only for one 

session, others for two sessions or more, working with a different student in each one.

3 

2

Primary tutors were those who had the longest assigned relationship with a student, among that student’s 

assigned tutors. 3

The exception to this scenario was work-study students who were recruited from local colleges and universities and a few paid tutors who worked in centers for more significant periods of time and on multiple days. 

31 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Although students were typically assigned to work with two different tutors each week, 

trust and relationship-building could still occur if tutors worked with the student for an extended 

period of time and appeared regularly and consistently for their sessions with students. As 

shown in Table 3.2, the typical duration of the tutor-student relationship was 19.8 weeks, or a 

little less than five months — a little longer than the one-semester commitment that Reading 

Partners asks tutors to make. Data regarding tutor attendance were not available from the MIS, 

so the study team conducted a review of randomly selected student folders (10 at each site) to 

explore the number of tutoring sessions and the number of unique tutors a student had over a 

two-week instructional period. During this time, students ideally would have received four 

tutoring sessions by either one or two individuals. 

The student folder review revealed that among students who received at least the full 

dosage of tutoring sessions (meaning four over the course of two weeks), more than half (56 

percent) worked with three tutors or more over the two-week period of time. Among students 

who received three sessions over the two weeks, approximately one-fourth (27 percent) worked 

with a different tutor on each occasion. These snapshot data suggest that tutor consistency was 

somewhat limited. 

As a result of tutor absences, both program managers and site coordinators did a fair 

amount of tutoring themselves. On average, site coordinators reported tutoring four or five 

times a week (about once a day), but at the high end, site coordinators reported tutoring three or 

four times a day. This is still a relatively small percentage of the total number of tutoring sessions offered during a given week (sites can accommodate up to 40 tutoring sessions in a given 

day), but it is still probably more than is ideal. While Reading Partners expects that site coordinators and program managers will do some tutoring, too much takes away from their ability to 

coach and provide guidance and support to the regular volunteer tutors. 

Dedicated School Space and Use of Materials 

All tutoring is designed to take place in a dedicated reading center within a school. Visits to each study site revealed that all but one of the sites had dedicated space. The one site that 

did not have dedicated space was operating in a library. The reading center was partly walled 

off from the larger part of the library by bookshelves, but the space included computer work 

stations for general school use. Despite this set-up, Reading Partners staff appeared to be making the best of the limited space and doing what they could to ensure that the environment was 

conducive to tutoring. All other sites had dedicated classrooms for the Reading Partners reading 

center, which were used by up to eight tutor-student pairs at a time. 

Within the dedicated classroom space, Reading Partners also has a set of materials 

(such as literacy-themed posters, signs, and other decorations) and required classroom components (for example, a “take-home reading” area) that each reading center should have in place. 

32 
 

 

 

During their site visits, the study team reviewed the space and materials against a checklist of 36 

items provided by Reading Partners. Box 3.1 presents a full list of the items in the Reading Center Materials Checklist. Of those 36 items, only 20 items were mandatory and incorporated into 

the fidelity index scoring. Of the 20 items that Reading Partners expects sites to have in place, 

only two sites had fewer than 19 items in place. Taken together, analysis of the space and materials at each site suggests that this component of the Reading Partners model displayed a high 

level of implementation fidelity. 

Box 3.1 

Reading Center Materials 

8 work stations 

– Chairs and tables 

– Pencils and erasers 

– Post-it® notes 

– Bookmarks 

Alphabet strip 

Additional supplies 

Bins and boxes for “Read Aloud” 

Box for completed work 

Bulletin boards for student work 

Clock 

Curricula crates 

“Read Aloud” library 

Resource table for volunteers 

– Calendar 

– Shadow session packets 

– Tutor information binder 

– Tutor resources 

Site coordinator desk 

Student boxes 

– Sight word ring 

– Student folder 

– Tutor folder 

– Word journal or reading/writing journal 

– Worksheets 

“Take-Home Reading” area 

– Book crates 

– Chart and stickers 

– Reflection forms 

– “Take-Home Reading” sign 

Tutoring schedule 

United States map 

Wall resources 

Whiteboards 

Word wall 

World map 

Structured and Individualized Curriculum 

As described in Chapter 2, each lesson begins with a tutor reading aloud to the student 

from a literary or informational text that the child has selected. While reading, the tutor pauses 

to discuss the book’s content and vocabulary. The tutor read-aloud and vocabulary review is 

followed by the introduction of a new skill or concept, which the student is then asked to apply. 

Brief observations with students and interviews with tutors revealed that this approach is very 

closely aligned with the way in which the curriculum is implemented in practice. However, the 

study team did not conduct the tutor observations with the intention of monitoring the curriculum implementation in a detailed and fine-grained way. Observations were designed to provide 

33 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

the research team with a “broad strokes” understanding of how the curriculum was operationalized during a tutoring session. As such, this component of the model is not included in the fidelity index score. 

Overall, interviews with site coordinators and tutors, as well as brief, unstructured observations of the tutoring sessions, suggest that the Reading Partners curriculum was implemented with a moderate to high level of fidelity across all sites. Tutors generally followed the 

steps outlined above. Some of the lesson components include scripted language that tutors can 

read verbatim. However, this is not a requirement; the scripts and outlines are available as 

guidelines. Tutors who are new sometimes rely on the scripts more heavily until they feel more 

comfortable with the curriculum. 

Tutors and site coordinators describe the beginning of a session as starting when a tutor 

picks up a student from the classroom, well before the actual tutoring begins. This suggests that 

relationship- and rapport-building is an important, implicit feature of the model. One site coordinator described the conversation and rapport being built during that time as a way “to prepare 

them [the students] to come to Reading Partners.” Often tutors ask students about their weekend 

or their day so far, which lays the groundwork for the one-on-one relationship that helps to 

facilitate the session. 

In observations, some variation was observed in the quality of delivery of tutoring. For 

example, occasionally a tutor did not pause for comprehension or vocabulary checks or did not 

do a good job of modeling fluent reading during the tutor read-aloud portion of the session. 

However, these observations were largely anecdotal. Site coordinators were also asked to comment on the quality of their tutors, but for the most part, site coordinators equated tutor quality 

with consistency and commitment. Tutors who came when they were supposed to and called or 

e-mailed if they were unable to make their scheduled time were generally considered high 

quality. Assessing the quality of the tutor-student interaction may be an area for future exploration within Reading Partners. 

Data-Driven Instruction 

Nearly all Reading Partners program staff and volunteer tutors are aware that data are 

integral to the Reading Partners instructional model. As described in Chapter 2, at the center of 

this data-driven instructional model is the use of the Rigby PM Benchmark assessment, which 

determines placement within the Reading Partners curriculum and assesses students’ progress 

by testing their performance at baseline before they begin the Reading Partners curriculum, their 

growth midway through the school year, and their status at the end of the school year. Analysis 

of the MIS data revealed that 95 percent of eligible students were in fact assessed using the 

Rigby all three times. Also, Reading Partners staff appeared to be doing a consistent job of shar34 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

ing these data with schools; the study team found that only one site failed to share data with the 

school principal. 

However, there was somewhat less evidence regarding the use of the Rigby data to inform, customize, and update individual students’ Individualized Reading Plans. Although a 

majority of the site coordinators mentioned that they used the midyear Rigby assessment to 

either create an action plan or advance students, program managers and site coordinators provided more limited information about the specifics of how individual student progress was 

monitored and what was done with students who were not making progress. The use of student 

folders to document each tutoring session was another source of data that Reading Partners used 

to monitor student progress as well as particular successes and challenges that the students 

faced. While most of the student folders that were included in the folder review contained 

detailed documentation about student work during the tutoring sessions, student work was not 

documented in at least one instance in 10 sites over the course of the two-week review period. 

Rigorous and Ongoing Training 

The Reading Partners model requires that all Reading Partners school-based staff members undergo formal, instructional training before they are placed at a school site. Interviews 

with Reading Partners staff revealed that there was a high level of participation in the initial 

training provided to Reading Partners staff. Virtually all had completed a carefully designed 

training regimen. The only exceptions were two program managers who did not complete the 

initial, formal training because they were brought on mid-year to replace departing program 

managers. However, these program managers were able to briefly overlap with their predecessors and were able to receive on-the-job training and ongoing support from colleagues. 

Interviews with site coordinators also revealed that there was a high level of participation in the ongoing training sessions, which occurred monthly. With hardly any exceptions, staff 

at all levels found the training to be comprehensive, useful, and successful in preparing them to 

carry out their roles effectively. 

For tutors, the key — and really sole — training they received before beginning their 

first tutoring session with a student was through “shadow sessions,” a combination of orientation and tutoring observation led by the site coordinator at the school where the tutor was assigned. Analysis of interviews with tutors revealed that the majority of tutors participated in a 

shadow session; only three volunteers did not participate in any shadow session before beginning to tutor. Those who did attend reported that these shadow sessions were adequate for 

orienting the volunteers to the Reading Partners reading center and curriculum, and equipped 

them to tutor students successfully. In fact, at least 80 percent of tutors who were interviewed 

indicated that they felt adequately trained for the role of tutor. 

35 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Ongoing training opportunities for tutors appear to have been plentiful, but few volunteers availed themselves of those opportunities. Site coordinators and tutors alike reported that 

additional training sessions were offered, often twice monthly, and tutors reported often that 

they were aware of the training. These training sessions covered topics that site coordinators 

thought would be helpful to tutors, such as how to motivate students or how to improve tutoring 

skills. Still, almost half of the tutors (23 of 51) reported that they never attended any of the training sessions. By far the most frequent reasons given for not attending were scheduling conflicts 

and other commitments. 

Instructional Supervision and Support 

For ongoing support, tutors relied almost exclusively on site coordinators and in general felt well supported by them. (Forty-two of 50 tutors who were interviewed indicated that 

they felt adequately or well supported.) Each Reading Partners center in the study sites had a 

regular site coordinator present and managing the classroom. Tutors received support from 

site coordinators in several different forms. First, there was ongoing monitoring and occasional direct involvement by the site coordinators during tutoring sessions. In addition, site 

coordinators held “check-outs” with tutors following a session, during which they reviewed 

how a session had gone. Third, site coordinators annotated student folders: Four tutors reported finding sticky notes regarding individual students, giving advice on how to proceed (for 

example, specific topics to emphasize or ways to address problems that students seemed to be 

having). By far the most common kind of support was student-specific. Tutors brought issues 

or problems they were having with specific students to site coordinators for a consult. These 

issues might be as simple as how to pronounce a certain word, a more complex problem (such 

as how to get across certain consonant or vowel sounds, or how to define a word or concept), 

or motivational challenges. 

Site coordinators’ descriptions of their own jobs suggest that there was some variability 

in the amount and quality of coaching and support they provided to tutors. Some site coordinators described their role as more of a monitoring role — ensuring that tutors knew their schedules and were following the curriculum, while others discussed providing advice and guidance 

to tutors on a broader range of issues. 

Broadly speaking, site coordinators, program managers, and outreach coordinators all 

indicated that they had adequate ongoing support and clear lines of accountability that worked 

for them. One challenge, however, was the lack of a program manager for two sites, which reduced the level of supervision and support available to the site coordinators in those programs. 

36 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Factors That Facilitated Implementation 

A number of factors stand out as helping schools maintain high levels of overall implementation fidelity. First, the structured and scripted nature of the curriculum enables volunteers with 

a variety of backgrounds to easily integrate into the program and deliver instruction without a 

lot of formal training. Second, the overall quality of the Reading Partners staff and AmeriCorps members is quite high, particularly at the school level (program managers and site 

coordinators): Interviews with program managers and site coordinators revealed that they 

were dedicated and effective in their work on the whole. Most Reading Partners staff at the 

program manager or regional executive director levels have come to the organization with 

classroom teaching experience and a passion for the organization’s mission — a clear strength 

of the organization. The support of an effective organizational structure also helps ensure that 

a high level of quality is maintained among staff. 

In addition, the training provided to Reading Partners staff is strong and helps ensure 

that staff members know what is expected of them and that they have the appropriate skills and 

tools to carry out their jobs effectively. 

Certain contextual factors, such as Reading Partners’ relationship with teachers and 

administrators, have also helped maintain high levels of implementation fidelity. Through its 

school-level staff (program managers and site coordinators), Reading Partners maintains strong 

working relationships with principals and teachers. As a result, principals and teachers in the 

study schools reported generally being very supportive of the program, citing the one-on-one 

relationships that are fostered between students and tutors as one of its most helpful features. 

Barriers to Implementation 

While several factors facilitated effective program implementation, a number of other contextual factors posed barriers. The pull-out nature of the program was one such barrier. Although a 

majority of teachers and principals were very supportive of the Reading Partners program, in 

some schools, particularly those with many pull-out services in addition to Reading Partners, 

some teachers were resistant to having students pulled out from their classrooms for Reading 

Partners, particularly during instructional time. In such cases, site coordinators had to work 

harder to schedule tutoring sessions and had less flexibility in providing make-up sessions for 

students who missed a tutoring session because they were absent or a tutor was unavailable. 

Schools also varied in the space they were able to provide for the reading center to operate. 

While most were able to provide dedicated space, some classrooms were small, which meant 

that tutor-student pairs were spaced close together and the setting was noisier as a result. Smaller spaces also meant that fewer students could easily be tutored at one time. 

37 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Student attendance also played a role in program implementation. Student absences 

made it difficult to ensure that they received two 45-minute sessions per week. A small number of tutors also mentioned some difficulties working with a student population that had 

greater needs than average students, particularly English language learners or students with 

behavioral issues, but generally did not believe that these difficulties defined their overall 

tutoring experience. 

With respect to volunteer tutor recruitment, school location and characteristics of a region were important factors that affected implementation of the program. Schools that were not 

close to public transportation or were located in neighborhoods that were perceived as unsafe 

had the hardest time with tutor recruitment. In addition, some regions had fewer opportunities 

for fundraising and less potential for community partnerships that lead to volunteerism. 

The biggest barrier that Reading Partners faced in implementing the program was maintaining consistent tutor attendance and retention. As described in Chapter 2, tutors are asked to 

make a one-semester commitment, but several site coordinators reported that many volunteers, 

particularly high school and college students, did not stay beyond that period of time, making it 

more difficult to establish a strong one-on-one relationship with the student. As already noted, 

the duration of the average tutoring relationship was 19.8 weeks — a little less than five 

months, or just over a semester. In one school, the average duration of the tutoring relationship 

was only 11 weeks. 

Furthermore, volunteers varied in their consistency and commitment. Site coordinators 

reported that tutors sometimes failed to report at their scheduled time and at times did not notify 

the site coordinator beforehand. This situation made it challenging to ensure that students 

received two sessions each week. However, there were structures in place to address these problems. As noted earlier, a full-time employee was charged with recruitment on an ongoing basis, 

which helped to ensure an adequate supply of tutors to respond to the relatively high level of 

turnover. Reading Partners sometimes used substitute tutors if tutor absences could be anticipated, and site coordinators or other tutors often provided make-up sessions on Friday to students 

who had missed a session earlier in the week. These two tactics helped ensure that tutor inconsistency did not mean that a student necessarily missed sessions because a tutor was absent. Additionally, tutors are required to leave notes in students’ folders indicating what they worked on 

together and any problems that were encountered; because it is expected that many students will 

see two or more tutors regularly, this documentation is necessary to provide continuity of instruction. Site coordinators also stay well informed about the progress of the students in their 

centers and serve as a point of continuity for students who participate in the program. Chapter 4 

explores the impact of tutor consistency on student outcomes. 

38 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

While these factors made implementation more complicated, the obstacles were not 

insurmountable and did not significantly reduce Reading Partners’ ability to implement the program with fidelity. In fact, overall, Reading Partners maintained an impressively high level of 

fidelity and its staff members were clearly able to overcome many of these challenges. These 

factors, however, may become more significant barriers for Reading Partners programs that 

may not be as mature or established as the program sites that participated in this study. As such, 

these programmatic and contextual factors should be examined more closely when implementing the Reading Partners model at newer, less established school sites and at sites that are working to further strengthen existing programs. 

Summary 

The Reading Partners program was implemented with a moderate to high level of fidelity across 

each of the key dimensions of the program: (1) regular, one-on-one tutoring; (2) dedicated 

school space and use of materials; (3) structured and individualized curriculum; (4) data-driven 

instruction; (5) rigorous and ongoing training; and (6) instructional supervision and support. 

•	 On average, students received approximately 1.5 tutoring sessions per week 

for 28 weeks in a dedicated space with appropriate materials. 

•	 Ninety-five percent of eligible students were assessed three times using the 

Rigby assessment, and Reading Partners staff consistently shared student 

data with schools. 

•	 Site coordinators, program managers, and outreach coordinators all indicated 

that they had adequate training for their roles as well as ongoing support in 

their work and clear lines of accountability. 

•	 Despite the limited initial training that tutors received, they generally reported that they felt adequately trained and supported in their role, though individuals relied on support from the site coordinator to deal effectively with 

more challenging students. 

•	 The biggest barrier that Reading Partners faced in implementing the program 

was maintaining tutor attendance and retention. However, there were structures in place to address these challenges and, as a result, most students were 

tutored on a regular basis throughout the year, even if the same person did 

not consistently deliver that tutoring. 

39 

Chapter 4 

The Impact of Reading Partners 

Because Reading Partners was not able to serve all students who would have qualified for the 

program, MDRC was able to employ a rigorous evaluation design, called a randomized controlled trial, to determine the magnitude of the effect — or impact — of Reading Partners. Randomly assigning students into a program group and a control group helps ensure that there are 

no systematic differences between the two groups of students when the study begins and that 

any positive effects can be causally attributed to the program with a high degree of confidence. 

The analysis of impacts was designed to answer the following key research question: 

•	 Do students who have been randomly assigned the opportunity to participate in the Reading Partners program attain different reading proficiency 

levels from those of a similar group of students who were not offered this 

opportunity? 

The study also explored a set of secondary questions that were not considered key outcomes because they were more distal to the intervention, but that the study team was interested 

in exploring: 

•	 Do students in the program group exhibit different attention levels and different behaviors in the classroom from the control group? 

•	 Do students in the program group complete their in-class assignments and 

homework at a different rate from students in the control group? 

The analyses also explored the effects of the program among different subgroups of 

students. The subgroups that were examined reflect five characteristics that are both theoretically relevant and policy-relevant to reading instruction: 

•	 Do impacts differ for students entering the program with higher levels of 

pre-intervention reading achievement versus those with lower achievement 

levels? 

•	 Are impacts different for younger students (in grades 2 and 3) compared with 

older students (in grades 4 and 5)? 

•	 Do impacts differ for male versus female students? 

•	 Do impacts differ for students identified as English language learners compared with students who are fluent in English? 

41 
 

 

 

 

 

 

 

 

 

•	 Do impacts differ for students with prior exposure to the Reading Partners 

program compared with students who were exposed to Reading Partners 

tutoring for the first time during this study? 

To put the impact findings in context, the study team also explored questions related to 

service contrast, or the differences in the services received by students who were assigned to the 

program group versus students who were assigned to the control group: 

•	 What is the difference in academic support offered to students in the program 

and control groups? What was the resulting service difference in the amount 

of reading instruction on average between the program and control groups? 

If participating in the Reading Partners program did not increase the quality or frequency of the overall supplemental reading services that the program group received, either 

because the program group did not fully participate in the program or because the control 

group received other services of equal quality or intensity, then it is unlikely that an impact of 

the program would be observed, even if the Reading Partners program were of high quality. 

Finally, for exploratory purposes, this chapter includes an investigation of the ways in 

which the program impacts vary based on measures of fidelity and context. 

Methods 

Student Sample 

The study team used a systematic process to obtain and randomly assign the student 

sample at each school. At the start of the school year, schools and Reading Partners staff provided the study team with a list of students they wished to refer to the Reading Partners program, who had parental permission to participate in the program and the study. These lists were 

developed by individual schools and were based on a combination of test score review and recommendations by teachers or school staff (or both). Within two days of receiving the list, the 

study team randomly assigned students within schools (using a random number generator) to 

the program group that was eligible to participate in the Reading Partners program or to an “as 

is” control group that would not participate in Reading Partners but was eligible to receive any 

other supplemental reading services. A total of 1,265 students in the 19 participating schools 

were randomly assigned. Students were randomized within grade groups (grades 2 through 3 

and grades 4 through 5) to ensure that equal numbers of upper and lower elementary school students were represented in the sample, since one of the research questions examines the differential impact of Reading Partners by grade level. Schools often approach reading instruction differently in these two sets of grades. In grades 2 and 3, reading instruction generally focuses on 

42 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

learning to read, with more attention given to basic reading skills such as decoding, or the use of 

phonics to read written words correctly. In grades 4 and 5, the focus is on reading to learn, with 

greater emphasis on vocabulary and comprehension.1 Equal grade-group sizes maximize the 

study’s ability to detect potential differential impacts between these groups. 

The final respondent sample (those who took at least one follow-up assessment) includes 1,166 students (594 students in the program group and 572 in the control group). These 

totals represent response rates of 92.0 percent for the program group and 92.4 percent for the 

control group, which were not statistically significantly different from one another. Descriptive 

statistics for the final sample at baseline (that is, at the start of the study) are shown in Table 4.1. 

More than half of the sample was Hispanic (65 percent) and one-fifth of the students were black 

(19 percent). About half were officially designated as English language learners (55 percent) 

and over 90 percent of the students were eligible for free or reduced-price lunch. Aside from a 

slightly larger number of students in the program group who had previously participated in the 

program (32 percent compared with 26 percent of the control group), there were no statistically 

significant differences between the program and control groups on these variables. When the 

team performed a correction to account for the multiple characteristics that were analyzed at the 

same time, even this difference is not statistically significant.2 

During the course of the study, 589 students across the 19 participating schools received over 25,000 tutoring sessions and over 19,000 hours of tutoring, administered by 901 

community volunteers. 

Data Collection 

Following random assignment in the fall of 2012, reading assessments (one group assessment and two individually administered assessments) were given to all students in the study. 

Assessments were administered by independent assessors hired by the study team. Assessors 

attended a two-day training program before beginning assessments. Baseline testing occurred as 

soon as possible after random assignment. Follow-up testing, using the same three assessments, 

was conducted in the spring of 2013, as close to the end of the school year as possible. In addition, in the spring of 2013, teachers were asked to complete a survey about the academic behavior and performance of each student in their classrooms who was part of the study sample (in 

both the program and control groups) and to report on the full range of supplemental reading 

services that each student had received during the school year. A copy of the teacher survey is 

1

Chall (1983).

2

Corrections for multiple hypothesis testing were made using the Benjamini-Hochberg procedure (Benjamini and Hochberg, 1995). Regardless, a full set of covariates was included in all analyses to control for any 

unobserved differences between the two groups at baseline and to improve the precision of the estimates. 

43 
 

Reading Partners Evaluation
 

Table 4.1
 

Baseline Characteristics of Program and Control Group Students
 

 

 

 

 

 

 

 

 

 

 

 

 

 

Program Control 

Characteristic Group Group Difference P-Value 

Male (%) 54.88 54.50 0.38 0.899 

Ethnicity (%) 

Black 19.39 19.03 0.37 0.864 

Hispanic 65.09 65.27 -0.18 0.944 

Asian 8.43 9.09 -0.66 0.691 

White 5.90 5.40 0.50 0.709 

Other 1.18 1.02 0.16 0.856 

Special education (%) 11.62 10.49 1.13 0.550 

English language learner (%) 55.21 55.50 -0.29 0.916 

Eligible for free/reduced-price lunch (%) 91.35 90.22 1.13 0.540 

Prior Reading Partners participation (%) 31.82 25.75 6.07 * 0.074 

Average age (years) 8.79 8.76 0.03 0.413 

Overage for gradea (%) 9.83 8.40 1.43 0.408 

Average baseline achievement scoresb 

Comprehension 574.74 573.49 1.26 0.544 

Sight word efficiency 91.15 90.38 0.77 0.328 

Fluency 0.03 0.00 0.03 0.573 

Students by grade level (%) 

Grade 2 23.06 25.29 -2.23 0.256 

Grade 3 29.46 27.23 2.23 0.256 

Grade 4 24.58 25.72 -1.14 0.572 

Grade 5 22.90 21.76 1.14 0.572 

Sample size 594 572 

SOURCES: MDRC calculations using study team-administered baseline SAT-10, TOWRE-2, and 

AIMSweb scores; district-provided demographic data; Reading Partners management information 

system data. 

NOTES: Calculations for this table used data for all students in the respondent sample. Sample sizes for 

individual outcomes may fall short of the reported sample sizes because of missing or unusable data. 

The model allows estimates to vary randomly across schools and controls for the random assignment 

block. 

A two-tailed t-test was applied to differences between research groups. Statistical significance 

levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

aStudents are classified as "overage" for a certain grade if they were the following specified ages on 

September 1, 2012: 8 or older for 2nd grade, 9 or older for 3rd grade, 10 or older for 4th grade, and 11 

or older for 5th grade. 

bThe analyses for student reading achievement were conducted using scaled scores for the SAT-10 

(comprehension) and the TOWRE-2 (sight word efficiency) and sample-normed scores for the 

AIMSweb (fluency). Sample-normed AIMSweb scores range between 0 and 1. 

44 
 

 

 

 

 

Reading Partners Evaluation
 

Table 4.2
 

Summary of Data Collection for Impact Analysis, by Source
 

Number Randomly Number Percentage 

Assigned Collected Collected 

Program Control Program Control Program Control 

Data Source Group Group Group Group Group Group 

Baseline reading assessments 

Comprehension 646 619 617 591 96 95 

Sight word efficiency 646 619 616 592 95 96 

Fluency 646 619 619 592 96 96 

Follow-up reading assessments 

Comprehension 646 619 579 567 90 92 

Sight word efficiency 646 619 585 562 91 91 

Fluency 646 619 589 562 91 91 

Teacher survey 646 619 593 565 92 91 

Study year attendance 646 619 615 599 95 97 

Study year state test scores 646 619 567 550 88 89 

SOURCES: MDRC calculations from study team-administered baseline and follow-up SAT-10, 

TOWRE-2, and AIMSweb scores; study team-administered teacher survey; district-provided 

attendance and state achievement data. 

NOTE: Reading comprehension was assessed using the SAT-10; sight word efficiency was assessed 

using the TOWRE-2; fluency was assessed using the AIMSweb. 

included in Appendix B. A summary of the data collected on students is provided in Table 

4.2. In addition, participating districts supplied the research team with state test score data, 

attendance information, and the demographic characteristics of the individuals in the sample. 

General demographic information about the schools (school size, racial/ethnic composition, 

number of students receiving free or reduced-price lunch, and so forth) was extracted from the 

publicly available Common Core of Data (CCD), published by the National Center for Education Statistics.

3 

3

See http://nces.ed.gov/ccd. 

45 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Primary Outcomes 

The primary outcome of interest in this study is student reading proficiency, as measured by test scores. The research team wanted to select a set of assessments that would measure 

proficiency in multiple domains of reading (described below) and were reliable, valid, well 

aligned with the curricular emphasis of Reading Partners, able to detect small improvements 

among lower-scoring students, and feasible to administer in a relatively short amount of time. 

After reviewing a range of possible reading assessments and talking with reading experts, the 

following assessments were selected: a group-administered reading comprehension test (the 

Stanford Achievement Test, 10th Edition, or SAT-10), an individually administered sight word 

efficiency assessment (the Test of Word Reading Efficiency, 2nd Edition, or TOWRE-2), and 

an individually administered oral reading fluency assessment (the AIMSweb). 

•	 The SAT-10 reading comprehension subtest is a 30-minute assessment that 

measures students’ comprehension across four different modes: initial understanding, interpretation, critical analysis, and awareness and usage of reading 

strategies. Students read short passages and then answer questions about 

them.4 

•	 The TOWRE-2 sight word efficiency subtest took approximately five 

minutes to administer. The sight word efficiency subtest is a measure of an 

individual’s ability to pronounce printed words accurately and fluently. It assesses the number of real words printed in vertical lists that an individual can 

accurately identify within 45 seconds.5 

•	 The AIMSweb is an individually administered one-minute assessment of 

oral reading fluency, conducted by measuring accuracy and speed of reading. 

Students read a passage aloud for one minute and the assessor records omissions of words, misreading, and substitution of words. The count of the number of words read correctly is used as the outcome.6 

Secondary Outcomes 

Based on the Reading Partners program model, the study team also identified several 

secondary outcomes that were more distal to the intervention but still of interest. 

4

NCS Pearson, Inc. (2004). Kuder-Richardson 20 reliability on this measure ranged from 0.84 to 0.93. 5

Torgensen, Wagner, and Roshotte (2012). Alternate-form reliability on this measure ranged from 0.90 to 

0.92. 6

Pearson Education, Inc. (2012). Alternate-form reliability on this measure ranged from 0.93 to 0.95. Splithalf reliability also ranged from 0.93 to 0.95 and inter-rater reliabilities on this measure were 0.99. 

46 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

•	 Academic behavior. One secondary outcome of interest was the impact of 

Reading Partners on students’ academic behavior (for example, attentiveness, 

assignment and homework completion, and disruptiveness). Four individual 

items were included on the teacher survey (reproduced in Appendix B) to assess each of these aspects of student behavior and were combined to create a 

single composite measure.7 The possible score ranged from a low of 4 to a 

high of 16. 

•	 Teacher-rated academic performance. Teachers were also asked to assess 

each student’s performance in reading, performance in math, and academic 

performance overall, relative to other students in the same school. The impact of the Reading Partners program on teachers’ ratings of student academic performance was explored separately for these three items. 

•	 Attendance. Finally, attendance data obtained from individual school 

districts were used to explore the impact of Reading Partners on student 

attendance. 

The difference between the outcomes of the program and control group students is related to the service contrast, or the difference in reading services received by students in the 

program and control groups. Service contrast was measured using additional data provided 

through the teacher survey.8 Teacher surveys asked teachers to identify how much time each 

student spent receiving in-class instruction in reading and how much of that total time was 

one-on-one instruction in class. Teachers were also asked to indicate any supplemental reading services, such as participating in pull-out reading instruction with a reading specialist that 

the student had received over the course of the school year and the amount of time spent in 

those supplemental activities. Reading Partners was included among the list of possible supplemental services. 

Analytic Approach 

The analysis pools together the sample of students across schools and compares outcomes for students who were assigned to participate in Reading Partners (the program group) 

with those who were assigned to the control group and thus did not receive Reading Partners’ 

7

The composite score was created by reverse-coding each of the items (except disruptiveness) and then 

summing the scores to obtain a single measure of academic behavior, with higher scores representing better 

composite behavior. Cronbach’s alpha for this composite measure, which assesses the coherence of the items 

within the measure, was 0.82. 8

School-specific teacher surveys included questions about supplemental services offered to students at that 

school. These services were identified during the site visits that took place earlier in the school year. 

47 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

services.

9 The model includes information about each student’s school and grade level to account for the study design.10 The model also controls for several baseline covariates to improve 

precision. These covariates include an individual-level pretest measure and the time lapse 

between baseline testing and follow-up testing, student’s gender, race/ethnicity, free or reducedprice lunch status, age, English language learner status, special education status, and whether 

the student is overage for grade. The complete model used in the analysis can be found in 

Appendix B. 

The same basic model is used to estimate the impact of the program on service contrast; 

however, in this case the outcome of interest was participation in all supplemental services 

(including Reading Partners) or time spent in reading instruction. 

As noted above, the analyses also explore the effects of the program among subgroups 

of students defined by characteristics depicting a student’s pre-random assignment condition, 

including grade level, baseline academic performance, gender, prior exposure to Reading Partners tutoring, and English language learner status, and the ways in which the program impacts 

vary based on measures of fidelity and context. Subgroup and other exploratory impacts — 

those that cannot be causally attributed — were estimated using only the members of a studentor school-level subgroup. 

The remainder of this chapter includes a description of the findings from the random 

assignment study. It begins by assessing whether or not the program had a statistically significant impact on the amount of instructional time in reading that the program group received 

and on their receipt of all supplemental reading services (including Reading Partners). Next, 

there is a description of the impact of the program on reading proficiency (measured by primary outcomes) and on the secondary outcomes of interest. Finally, this chapter presents an 

exploration of whether the program was more effective for some subgroups of students than 

for others and the contextual and implementation factors that may have contributed to any 

variation in impacts. 

9

“Intent-to-treat” impact estimates are used to characterize the program’s impact, meaning that students 

are grouped based on whether they were assigned to participate in Reading Partners or not as opposed to 

whether they actually received Reading Partners services. 10The impact estimate is allowed to vary randomly across schools — this is considered a “random-effect” 

estimate of the program’s impact and can be generalized to the broader population of schools and students 

served by Reading Partners. 

48 
 

 

 

 

 

 

 

 

 

 

 

 

 

Understanding the Contrast Between the 

Program and Control Groups 

The design of this study compared students who were offered the opportunity to participate in 

the Reading Partners program with equally reading-challenged students who were not offered 

that same opportunity. However, since reading is such an important component of elementary 

education, it is unlikely that schools allowed struggling readers who were not assigned to the 

Reading Partners program to continue to struggle without providing some additional support. 

Moreover, given the limited resources available to schools, it is possible that once students were 

involved with Reading Partners, the school determined that they did not need other types of 

reading services as much. Thus, it is important to assess how different the program group’s services were compared with the services that the control group received. 

To assess this difference, teachers were invited to respond to a short survey about their 

students who were participating in the study. The survey, which was administered in the spring 

of the study year, asked teachers to report the amount of time students spent in classroom reading instruction each week and the supplemental reading services, including Reading Partners, 

that students who participated in the study received over the course of the school year. The findings from this survey are shown in Table 4.3. 

Time Spent in Reading Instruction 

The top panel of the table shows the amount of time that students in the program and 

control groups spent in classroom reading instruction each week, as well as the difference in the 

amount of instructional time between the two groups. On average, students in both the program 

and control groups spent around six hours each week on reading instruction in the classroom. 

Of that time, though not shown in the table, approximately 35 minutes was spent working with 

an instructor one-on-one. There was no statistically significant difference between the two 

groups in the total amount of in-class instructional time spent on reading instruction or in the 

time students spent working in class with an instructor one-on-one. 

However, the program group did spend more time receiving supplemental reading instruction than did the control group. The Reading Partners group received about one more hour 

(57 minutes) of supplemental reading instruction per week than did the control group. This 

amount is less than would have been expected given the planned intensity of the Reading Partners program, which is designed to provide services to students for an hour and a half each 

week. The next section explores the source of this difference: whether students did not receive 

the intended dosage (frequency and intensity) of Reading Partners services, whether the students in the control group were also spending a substantial amount of time in other supplemental services, or both. 

49 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Table 4.3
 

Reading Instruction Received
 

Program Control 

Outcome Group Group Difference P-Value 

Average reading instruction time (weekly minutes) 

In class 352.39 360.29 -7.89 0.446 

In supplemental services 177.66 120.48 57.18 *** <0.001 

Students receiving any supplemental service (%) 97.24 64.75 32.49 *** <0.001 

Homework help 11.21 10.86 0.35 0.844 

One-on-one tutoringa 95.17 20.81 74.36 *** <0.001 

Small-group intervention support 22.59 31.53 -8.94 ** 0.014 

Technology-based programs 8.28 9.95 -1.68 0.279 

Other programs 7.24 7.48 -0.23 0.647 

Sample size 580 554 

SOURCES: MDRC calculations using study team-administered teacher survey; district-provided 

demographic data; study team-administered baseline SAT-10 scores; Reading Partners 

management information system data. 

NOTES: Calculations for this table used data for all students in the respondent sample with teacher 

survey data. Sample sizes for individual outcomes may fall short of the reported sample sizes 

because of missing or unusable data. 

The analysis model allows estimates to vary randomly across schools and controls for the 

random assignment block, the student-level pretest measure, the time lapse between baseline and 

follow-up testing, and student-level demographic covariates. 

A two-tailed t-test was applied to differences between research groups. Statistical significance 

levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

a

One-on-one tutoring includes the receipt of Reading Partners and other one-on-one 

supplemental services. 

Receipt of Supplemental Services 

To explore the difference between the program and control groups’ receipt of supplemental services more closely, teachers’ survey responses on this subject were categorized into 

five types: 

1.	 Homework help: unstructured before- or after-school assistance and support 

focused on completion of teacher-assigned work 

2.	 One-on-one tutoring: Reading Partners and programs similar to Reading Partners, 

where the student worked one-on-one with an adult on reading-related activities 

50 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

3.	 Small-group intervention support: usually a group-administered pull-out 

program for struggling readers, conducted by a school reading specialist or other 

educator 

4.	 Technology-based programs: those in which the student worked on improving 

literacy skills using a computer- or tablet-based application 

5.	 Other programs: reading services provided to students that did not fall into one of 

the above categories 

The bottom panel of Table 4.3 shows the impact of the Reading Partners program on 

the percentage of students receiving each type of supplemental service. The program had a positive impact on receipt of total supplemental reading services overall. Ninety-seven percent of 

the program group students received some sort of supplemental reading instruction (including 

Reading Partners). However, 65 percent of the control group also received some type of supplemental reading instruction. Not surprisingly, the biggest difference was in the percentage of 

students receiving supplemental one-on-one tutoring, the category that included Reading Partners. Teachers reported that 95 percent of students in the program group were receiving supplemental one-on-one tutoring compared with 21 percent of the students in the control group. 

This table also shows that, in addition to Reading Partners, many students in the program group 

also received other supplemental services, though not always to the same extent as the control 

group students. Students in the program group were less likely to receive small-group intervention support than were students in the control group; 23 percent of program group students 

received this type of support as opposed to 32 percent of the control group. 

As shown in Figure 4.1, these findings suggest that through their participation in Reading Partners, program group students received around 50 minutes of additional instruction in 

reading each week relative to the control group, in large part because the control group also 

spent a substantial amount of time in supplemental instruction, particularly in small-group intervention support. As such, the impacts reported below should be interpreted as the impact of 

Reading Partners relative to other supplemental service receipt, not the impact of Reading Partners compared with no intervention. 

Impact of Reading Partners on Measures of Reading Proficiency 

The results shown in Table 4.4 represent the primary outcomes of interest in this study — 

namely, the impact of the program on reading comprehension, sight word efficiency, and fluency. They show that overall the program had a positive and statistically significant impact on 

all three measures of students’ reading proficiency, with effect sizes equal to 0.10 for reading 

51 
Reading Partners Evaluation
 

Figure 4.1
 

Time Spent in Reading Instruction and Supplemental Services
 

 

 

 

 

 

 

 

 

 

600 

500 48 minutes 

Reading Partners 

es tu

400 

nim

Supplemental services, 

y l

excluding Reading 

eek

300 Partners 

e w

In-class one-on-one g

instruction 

eravA200 

In-class group instruction 

100 

0 

Program group Control group 

SOURCES: MDRC calculations using study team-administered teacher survey; districtprovided demographic data; study team-administered baseline SAT-10 scores; Reading 

Partners management information system data. 

NOTES: Calculations for this table used data for all students in the respondent sample 

with teacher survey data. Sample sizes for individual outcomes may fall short of the 

reported sample sizes because of missing or unusable data. 

The analysis model allows estimates to vary randomly across schools and controls for 

the random assignment block, the student-level pretest measure, the time lapse between 

baseline and follow-up testing, and student-level demographic covariates. 

The sample size for the program group is 580; the sample size for the control group is 

554. 

Rounding may cause slight discrepancies between the numbers shown here and those 

shown in Table 4.1. 

52 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Table 4.4
 

Primary Impacts of Reading Partners on Reading Proficiency
 

Outcome 

Program 

Group 

Control 

Group Difference 

Standard 

Error 

Impact 

Effect Size P-Value 

Comprehension 592.42 588.94 3.48 ** 1.71 0.10 0.043 

Sight word efficiency 92.78 91.37 1.42 *** 0.50 0.11 0.004 

Fluency 0.06 -0.03 0.09 ** 0.04 0.09 0.031 

Sample size 594 572 

SOURCES: MDRC calculations using study team-administered baseline and follow-up SAT-10, 

TOWRE-2, and AIMSweb scores; district-provided demographic data; Reading Partners 

management information system data. 

NOTES: Calculations for this table used data for all students in the respondent sample. Sample 

sizes for individual outcomes may fall short of the reported sample sizes because of missing or 

unusable data. 

The impact analyses for student reading achievement were conducted using scaled scores for 

the SAT-10 (comprehension) and the TOWRE-2 (sight word efficiency) and sample-normed 

scores for the AIMSweb (fluency). Sample-normed AIMSweb scores range between 0 and 1. The 

model allows estimates to vary randomly across schools and controls for the random assignment 

block, the student-level pretest measure, the time lapse between baseline and follow-up testing, 

and student-level demographic covariates. 

A two-tailed t-test was applied to differences between research groups. Statistical significance 

levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

comprehension (measured by the SAT-10), 0.11 for sight word efficiency (measured by the 

TOWRE-2), and 0.09 for fluency (measured by the AIMSweb).11 An explanation of effect sizes 

can be found in Box 4.1, but these impacts mean that at the end of the school year, students in 

the Reading Partners group were scoring, on average, two to three percentile points higher than 

were students in the control group on these assessments. For example, on the sight word reading 

assessment, the average student in the program group scored at the 36th percentile compared 

with the average student in the control group, who scored at the 33rd percentile. This difference 

is equivalent to approximately one and a half to two months of additional progress in reading 

relative to the control group. When looking at growth between the beginning and end of the 

11All three impacts remain statistically significant after controlling for multiple hypothesis testing using 

the Benjamini-Hochberg procedure (Benjamini and Hochberg, 1995). 

53 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

__________________________ 

Box 4.1 

What Is an Effect Size? 

An effect size is a way of quantifying the size of the difference in outcomes between two 

groups (for example, a program group and a control group, or two different subgroups), or the 

“impact estimate.” Calculating an effect size, which is done by dividing the impact estimate by 

the standard deviation of the outcome measure, allows researchers to compare impacts across 

variables that are measured in different units. For example, increasing a test score by two 

points on a test that is scored on a scale of 1 to 5 points is much more meaningful than increasing a score by two points on a test that is scored on a scale of 1 to 100. One way to interpret the 

magnitude of the impact estimates is to use the following rule of thumb: Effect sizes of about 

0.20 or less are considered “small”; effect sizes of about 0.50 to 0.80 are considered “moderate”; and effect sizes of about 0.80 or more are considered “large.”* 

*

Cohen (1988). 

school year on these three assessments, the evaluation finds that both groups scored higher at 

the end of the year than they did at the beginning of the year, but growth was greater for the 

program group than for the control group. (See Appendix Figure C.1.) 

To put these findings in context, it is helpful to think about the total amount of additional instruction students in the program group received. The typical Reading Partners student 

received around an hour of additional supplemental instruction each week for approximately 28 

weeks, or a total of 28 additional hours of supplemental instruction.12 The typical student 

receives approximately 30 hours of classroom-based instruction in reading each month (assuming a 90-minute reading block each day).13 Thus, Reading Partners provided students with about 

a month’s worth of reading instruction. Yet, as noted earlier, the program’s impact on reading 

achievement is equivalent to about one and a half to two months of learning. This means that 

students in the program group progressed a bit more than might have been expected given the 

amount of additional instruction time they received. 

12The 48-minute incremental difference presented in Figure 4.1 takes into account the total difference in 

instructional time, including the small and non-statistically significant difference in the amount of whole-class 

instruction. As indicated in Table 4.3, the additional supplemental instruction is closer to one hour per week. 13This is consistent with the amount of time teachers in this study reported that students spent in reading 

instruction. As shown in Table 4.3, teachers in this study reported that students in the control group received 

approximately 480 minutes of reading instruction each week (in-class plus supplemental), or 96 minutes a day. 

54 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Appendix C also presents exploratory analyses of the impact of Reading Partners on 

state assessment scores. No statistically significant impacts were found. 

Impact of Reading Partners on Academic Behavior, 

Performance, and Attendance 

The research team was also interested in understanding whether or not the program improved 

students’ academic behavior (for example, attentiveness in class) and overall academic performance in the classroom, which were secondary outcomes. Table 4.5 shows the impacts of the 

program on attendance and teacher ratings of school performance and academic behavior. The 

impacts were generally positive (with the exception of attendance) but not statistically significant. For example, teachers indicated that 73 percent of the students who were assigned to the 

program group completed their homework “usually” or “always” (shown as “Satisfactory 

homework completion” in the table), while only 70 percent of the control group students completed their homework “usually” or “always,” but the difference was not statistically significant. 

Teachers also rated the academic performance of the students who participated in the Reading 

Partners program somewhat, though not significantly, higher than they did the control students’ 

performance. For example, teachers indicated that 51 percent of the students in the program 

group were above average relative to their peers in overall academic performance (“Satisfactory 

performance overall” in the table) as opposed to 47 percent of the control group. 

Subgroup Analyses 

To explore whether or not Reading Partners was more effective for some groups of students 

than for others, the study team conducted subgroup analyses based on five prespecified characteristics of interest: baseline achievement (based on the fall reading comprehension assessment),14 prior receipt of Reading Partners services, gender, English language learner status, and 

grade level. The results are shown in Table 4.6. These analyses suggest that the Reading Partners program is effective for a wide variety of students: Impacts did not differ significantly for 

students from different grade levels or baseline achievement levels, for male or female students, 

for those who were still learning English, or for those who had previously been served by the 

Reading Partners program. 

There are, however, a few patterns in these findings that are worth mentioning, although 

these patterns should be interpreted with caution since the subgroups are not statistically 

14The SAT-10 was chosen as the basis for determining the baseline reading performance of the students 

because it is a more comprehensive assessment of reading proficiency than the TOWRE-2 or the AIMSweb. 

55 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Table 4.5
 

Secondary Impacts of Reading Partners on Students' Attendance
 

and Teacher-Reported Achievement and Behavior 

Outcomea 

Program 

Group 

Control 

Group Difference 

Standard 

Error 

Impact 

Effect Size P-Value 

Attendance (%) 95.65 95.83 -0.18 0.24 -0.04 0.466 

Academic outcomes (%) 

Satisfactory performance 

in reading 43.55 40.59 2.97 2.76 0.06 0.283 

Satisfactory performance 

in math 55.40 53.84 1.56 3.26 0.03 0.632 

Satisfactory performance 

overall 51.40 47.40 4.00 2.81 0.08 0.155 

Behavioral outcomes (%) 

Satisfactory homework 

completion 72.96 70.11 2.85 2.59 0.06 0.271 

Satisfactory assignment 

completion 78.16 76.59 1.57 2.41 0.04 0.515 

Satisfactory level 

of attentiveness 

65.16 

65.16 

64.82 

64.82 

0.34 

0.34 

2.78 

2.78 

0.01 

0.01 

0.901 

0.901 

Unsatisfactory level 

of disruptiveness 9.71 9.67 0.03 1.79 0.00 0.986 

Composite behavior 12.49 12.37 0.12 0.15 0.04 0.412 

Sample size 594 572 

SOURCES: MDRC calculations using study team-administered baseline SAT-10 scores; study teamadministered teacher survey; district-provided demographic and attendance data; Reading Partners 

management information system data. 

NOTES: Calculations for this table used data for all students in the respondent sample. Sample sizes for 

individual outcomes may fall short of the reported sample sizes because of missing or unusable data. 

The analysis model allows program impact estimates to vary randomly across schools and controls 

for the random assignment block, the student-level pretest measure, the time lapse between baseline and 

follow-up testing, and student-level demographic covariates. 

A two-tailed t-test was applied to differences between research groups. Statistical significance 

levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

aFor academic outcomes, satisfactory performance corresponds to ratings of "Average," "Above 

Average," or "Far Above Average." For behavioral outcomes, satisfactory corresponds to "Usually" or 

"Always," except for disruptiveness, which is reverse-coded. 

56 
 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Table 4.6
 

Subgroup Analysis of Primary Impacts
 

Subgroup 

Sample 

Size 

Impact 

Effect 

Size P-Value 

Comprehension 

Impact 

Effect 

Size P-Value 

Sight Word Efficiency 

Impact 

Effect 

Size P-Value 

Fluency 

Prior achievement 

1st quartile 

2nd quartile 

3rd quartile 

4th quartile 

310 

305 

291 

260 

0.02 

0.16 * 

0.12 

0.00 

0.839 

0.056 

0.258 

0.975 

0.22 ** 

0.08 

0.11 

0.14 

0.030 

0.295 

0.276 

0.106 

0.19 * 

0.05 

0.08 

0.10 

0.080 

0.538 

0.348 

0.300 

Bottom 3 quartiles 906 0.11 ** 0.043 0.10 ** 0.022 0.09 ** 0.040 

Prior receipt of 

Reading Partners 

Prior receipt 

No prior receipt 

316 

812 

0.11 

0.07 

0.178 

0.142 

0.05 

0.12 *** 

0.629 

0.010 

0.03 

0.11 * 

0.722 

0.060 

Gender 

Male 633 0.08 0.179 0.13 ** 0.014 0.07 0.212 

Female 522 0.11 * 0.070 0.09 0.137 0.14 ** 0.023 

English language 

learner (ELL) status 

ELL 634 0.11 ** 0.048 0.11 ** 0.041 0.14 ** 0.015 

Non-ELL 503 0.01 0.845 0.11 * 0.061 0.04 0.577 

Grade level 

Grades 2 and 3 

Grades 4 and 5 

605 

561 

0.09 

0.13 * 

0.132 

0.096 

0.17 *** 

0.05 

0.002 

0.479 

0.09 

0.10 

0.182 

0.101 

SOURCES: MDRC calculations using study team-administered baseline and follow-up SAT-10, 

TOWRE-2, and AIMSweb scores; district-provided demographic data; Reading Partners 

management information system data. 

NOTES: Calculations for this table used data for all students in the respective subgroups of the 

respondent sample. Sample sizes for individual outcomes may fall short of the reported sample sizes 

because of missing or unusable data. 

The impact analyses for student reading achievement were conducted using scaled scores for the 

SAT-10 (comprehension) and the TOWRE-2 (sight word efficiency) and sample-normed scores for 

the AIMSweb (fluency). The model allows program impact estimates to vary randomly across 

schools and controls for the random assignment block, the student-level pretest measure, the time 

lapse between baseline and follow-up testing, and student-level demographic covariates. 

A two-tailed t-test is used for all statistical tests presented in this table. Statistical significance 

levels between the program and control groups are indicated as: *** = 1 percent; ** = 5 percent; * = 

10 percent. Differences between subgroups are not statistically significant. 

57 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

significantly different from one another. First, the subgroup analyses based on the incoming 

reading achievement of the students in the study sample indicate that the sight word efficiency 

and oral reading fluency impact effect sizes were largest for students who performed in the lowest quartile of the study sample on the baseline reading comprehension test, with effect sizes 

equal to 0.22 and 0.19, respectively. In other words, the program group students who were lowest-performing at the beginning of the year performed the best on the sight word efficiency and 

fluency measures compared with their counterparts in the control group. Reading comprehension impacts were largest for those in the second-lowest quartile, with an effect size equal to 

0.16. Although the impacts for these subgroups (that is, by reading achievement at baseline) 

were not statistically significantly different from one another, they do suggest that the program 

may have been more effective for the students who were lowest-performing at baseline. Furthermore, the impacts on reading comprehension are the largest for the middle two quartiles, 

suggesting that the program may have been more effective in improving the reading comprehension skills of students after they had mastered the basic mechanics of decoding. 

To further investigate these findings, the study team explored the impact of the program on the percentage of students whose performance on the reading comprehension 

assessment (SAT-10) at baseline placed them in the lowest quartile of reading achievement 

nationally. These students (who represent approximately 60 percent of the study sample) performed less well on the SAT-10 at the beginning of the study than 75 percent of their peers 

nationwide. The Reading Partners program had a statistically significant positive impact on 

the percentage of these students who had moved out of this category by the end of the study 

year. As shown in Table 4.7, at the end of the year, 19 percent of the program group students 

who scored in the bottom quartile nationally at baseline had moved out of the lowest quartile 

by the end of the year, as opposed to 12 percent of the control group. The team also explored 

the impact of the program on the percentage of students who moved up a national quartile on 

the SAT-10 (including students from the lowest three quartiles who would thus be able to 

move up quartiles over the course of the year). There was no statistically significant impact on 

this outcome measure. 

Second, the program appears to be particularly effective for English language learners, 

with positive and statistically significant effects on all three reading assessments for this subgroup. Third, Reading Partners appears to have made a greater impact on reading comprehension scores for students who had previously participated in Reading Partners, while it appears to 

have made a greater impact on sight word efficiency and fluency for those in their first year. 

Finally, in the study sample, Reading Partners appears to have had a greater impact on 

sight word efficiency among students in grades 2 and 3, while there was a greater impact on 

58 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Table 4.7
 

Impacts of Reading Partners on Students’ Reading Comprehension

 Relative to National Norms 

Outcome 

Sample Program Control 

Size Group Group Difference 

Standard Impact 

Error Effect Size P-Value 

Moved out of lowest 

national quartile on SAT-10 

reading assessment (%) 675 19.16 11.61 7.55 ** 3.07 0.22 0.014 

Moved up a national 

quartile on SAT-10 

reading assessment (%) 1,094 14.10 10.71 3.39 2.50 0.11 0.175 

SOURCES: MDRC calculations using study-administered baseline SAT-10 scores; district-provided 

demographic data; Reading Partners management information system data.
 

NOTES: Calculations for this table used data for all students in the respondent sample. The sample size 

for the first outcome includes only students who started the study year in the lowest national quartile, 

while the second outcome includes students who started the study year in the lowest 3 national quartiles. 

The impact analyses for student reading achievement were conducted using percentile scores for the 

SAT-10 (comprehension). The model allows estimates to vary randomly across schools and controls for 

the random assignment block, the student-level pretest measure, the time lapse between baseline and 

follow-up testing, and student-level demographic covariates. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels 

are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

reading comprehension for students in grades 4 and 5. This finding is consistent with the developmental progression of reading proficiency and with the focus of reading in these grades.15 

Again, these subgroup differences are not statistically significant and thus should be interpreted 

with caution. 

Exploring Variation in Outcomes Based on Fidelity and Context 

In an effort to understand the factors that influenced program outcomes, the study team also 

explored several potential sources of variation in program impacts: tutor consistency, tutor 

background, school-level fidelity, the number of years that Reading Partners had been in operation in the school, and the average number of sessions per week and per year in the school. 

15Chall (1983). 

59 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

All these factors were identified in the literature as potential sources of variability in the effectiveness of tutoring programs or were identified by Reading Partners as factors that were worthy of exploration. The research team hypothesized that tutor consistency, school-level fidelity, the number of years the Reading Partners program had been in operation at the site, and 

the dosage of services that students received would all be positively associated with outcomes. Previous literature also indicated that college students might be more effective than 

tutors with other backgrounds.16 

These analyses are nonexperimental and are intended to be used for generating hypotheses but not for confirming proposed theories. Unlike the student subgroups discussed earlier, 

factors like tutor consistency can be influenced by the intervention and, as a result, they cannot 

be interpreted as causal. The findings from these exploratory analyses are shown in Table 4.8. 

Tutor Consistency 

First, to ascertain whether tutor consistency was related to outcomes, schools were categorized into three equal groups of “fewest,” “moderate,” and “most” consistency based on the 

average number of tutors assigned to each student over the course of the school year, with fewer 

tutors per student reflecting higher consistency. The average number of tutors assigned per student in each school ranged from a high of 3.6 to a low of 1.7. There are no discernible patterns 

in the results. 

However, the average number of tutors assigned per student at the school level may not 

be the most appropriate measure of consistency. First, this is a measure of school-level consistency, and does not represent the experience that an individual student may have had. There 

was considerable variation within schools in the number of tutors to whom individual students 

were assigned. In fact, the variation among students within each of the study schools accounted 

for 84 percent of the total variation in the number of tutors assigned per student in the study 

sample, while the variation between school-level averages accounted for only 16 percent. Thus, 

most of the variation occurred within schools, not across schools. This means that a school-level 

categorization is likely a poor predictor of a student’s experience. 

Therefore, another analysis explored whether growth in reading proficiency varied 

based on the number of tutors to whom an individual student was assigned. (See Appendix Table C.3.) These analyses are correlational because there is no control group to use as a comparison. As an example, tutors who were working with difficult-to-teach students may have been 

more likely to stop volunteering. Nonetheless, there is no statistically significant relationship 

16Elbaum, Vaughn, Hughes, and Moody (2000). 

60 
 

 

 

 

 

Reading Partners Evaluation
 

Table 4.8
 

Subgroup Analysis of Primary Outcomes
 

Based on Fidelity and Context Measures 

School-Level 

Subgroup 

Sample 

Size 

Impact 

Effect 

Size P-Value 

Comprehension 

Impact 

Effect 

Size P-Value 

Sight Word Efficiency 

Impact 

Effect 

Size P-Value 

Fluency 

Average number of 

tutors assigneda 

Fewest 311 0.17 0.302 0.10 0.281 0.09 0.254 

Moderate 500 0.08 0.167 0.11 ** 0.045 0.08 0.332 

Most 355 0.10 0.217 0.12 * 0.099 0.09 0.249 

Fidelity of 

implementationb 

Low 137 -0.07 0.546 0.10 0.620 0.14 0.472 

Moderate 469 0.13 0.129 0.15 *** 0.008 0.01 0.883 

High 560 0.10 0.138 0.07 0.211 0.12 ** 0.034 

Years of Reading 

Partners operations 

Less than 2 years 

2 years or more 

493 

673 

0.17 

0.07 

0.129 

0.164 

0.08 

0.12 ** 

0.185 

0.018 

0.01 

0.15 ** 

0.873 

0.011 

Dosage in sessions 

per weekc 

Low 347 0.09 0.440 0.11 0.149 0.02 0.760 

Moderate 442 0.16 ** 0.039 0.12 * 0.088 0.07 0.313 

High 377 0.03 0.732 0.08 0.234 0.16 0.138 

Dosage in total 

sessionsc 

Low 354 0.04 0.688 0.12 0.162 -0.02 0.774 

Moderate 464 0.21 *** 0.001 0.15 ** 0.015 0.17 *** 0.008 

High 348 0.01 0.893 0.02 0.778 0.06 0.448 

SOURCES: MDRC calculations using study team-administered baseline and follow-up SAT-10, 

TOWRE-2, and AIMSweb scores; district-provided demographic data; Reading Partners 

management information system data. 

NOTES: Calculations for this table used data for all students in the respondent sample. Sample sizes 

for individual outcomes may fall short of the reported sample sizes because of missing or unusable 

data. 

(continued) 

61 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 4.8 (continued) 

The impact analyses for student reading achievement were conducted using scaled scores for the 

SAT-10 (comprehension) and the TOWRE-2 (sight word efficiency) and sample-normed scores for 

the AIMSweb (fluency). The model allows estimates to vary randomly across schools and controls 

for the random assignment block, the student-level pretest measure, the time lapse between baseline 

and follow-up testing, and student-level demographic covariates. 

A two-tailed t-test was applied to differences between research groups. Statistical significance 

levels between program and control groups are indicated as: *** = 1 percent; ** = 5 percent; * = 10 

percent. Differences between subgroups are not statistically significant. a"Fewest" reflects the lowest tercile of average number of tutors assigned per student; "moderate" 

reflects the second tercile of average number of tutors assigned per student; "most" reflects the third 

tercile of average number of tutors assigned per student.

bLow-fidelity schools are those with a fidelity score of 15.0 or lower; moderate-fidelity schools
 

are those with a fidelity score between 15.5 and 18.5; high-fidelity schools are those with a fidelity
 

score of 19.0 or greater.
 c

Low-dosage schools are those in the lowest tercile of average sessions attended per student; 

moderate-dosage schools are in the second tercile of average sessions attended per student; and highdosage schools are in the third tercile of average sessions attended per student. 

between the number of tutors assigned to a student over the course of the school year and 

growth in reading proficiency on any of the three measures. 

The number of tutors assigned to a particular student also does not provide any indication of how consistently the assigned tutor attended the tutoring sessions. If a student were assigned to only one tutor over the course of the year, but that tutor was consistently absent, that 

might be a less effective tutoring arrangement than a student who was assigned to two tutors 

who each came on a regular basis. Unfortunately, Reading Partners does not collect data on tutor absences, so this possibility cannot be explored directly. However, data from the student 

folders that were reviewed as part of the implementation site visits can provide a measure of the 

number of tutors an individual student saw over a two-week period of time. To explore this 

question, the team calculated the average number of sessions per tutor for a random sample of 

students over the two-week period. This number was then averaged to the school level and an 

analysis explored whether impacts varied across schools based on this calculated measure of 

tutor reliability. The results are shown in Appendix Table C.4. Again, there is no consistent pattern in the data. 

Tutor Background 

The background of students’ primary tutors and the associated student gains on the 

three reading assessments used in this study are shown in Appendix Table C.5. The primary 

tutor is defined as the tutor with whom the student was officially paired over the longest period 

of time. Although these analyses are only exploratory, among the sample there is no clear indi62 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

cation that the background of a student’s primary tutor has any effect on that student’s increased 

reading abilities. In contrast with prior research, student reading gains for individuals who are 

tutored by college or graduate students are not noticeably larger than those for volunteers with 

different backgrounds.17 While these students had relatively high gains on the SAT-10 (reading 

comprehension), their gains on the TOWRE-2 (sight word efficiency) and AIMSweb (fluency) 

are lower than the gains for many other groups. Additionally, the differences in test score 

growth across the background characteristics are not statistically significantly different from one 

another, for either reading comprehension or fluency. There is a statistically significant difference in the sight word efficiency scores, suggesting that high school students, working adults, 

and retired individuals may have been more effective than college or graduate students in improving students’ sight word efficiency. 

Fidelity of Implementation 

The next analyses explored whether impacts varied based on the fidelity scores of the 

schools in the study. As described in Chapter 3, the study team created a fidelity index based on 

the core components of the Reading Partners model, to assess how well the program was being 

implemented across sites. The scores on this fidelity index were used to categorize sites. Lowfidelity schools were those receiving a score of 15.0 or lower on the fidelity index; moderatefidelity schools received a score of 15.5 to 18.5; and high-fidelity schools received a score of 

19.0 or greater. Again, there is no consistent pattern in student outcomes based on the fidelity 

scores of the various sites. This may be because the program was implemented with relatively 

high fidelity across all the sites. 

Years of Operation 

There were also no consistent patterns based on the number of years that Reading Partners had been operating in a school. Sites that had been in operation for two years or more had 

larger effect sizes on both the sight word efficiency and fluency assessments than did schools 

that had been operating for less than two years, but the effect sizes were larger for reading comprehension for the newer sites. Again, these differences are not statistically significantly different from one another. Since the pattern of results varies across outcomes and in some cases the 

differences are relatively small, this pattern of results likely occurred by chance. 

17Elbaum, Vaughn, Hughes, and Moody (2000). 

63 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

Dosage 

Finally, schools were categorized based on the average number of tutoring sessions that 

students received each week, which ranged from a low of 1.1 to a high of 1.8 across the 19 

schools. Schools were divided into three equal groups, low, moderate, and high. No consistent 

patterns were found and the differences across the subgroups were not statistically significant. 

Similarly, looking at the average number of tutoring sessions offered over the entire year, which 

combines both the duration and frequency of the tutoring that students received, there is no evidence that high-dosage schools are more effective. In fact, the schools with moderate dosage 

appear to be somewhat more effective than the schools with the highest levels of dosage, though 

the difference is not statistically significant. It is not clear why moderate-dosage schools would 

be more effective than high-dosage schools. To put this finding in context, however, it should 

be noted that overall the dosage of Reading Partners was quite high. As noted in Chapter 3, on 

average, students in the program group attended 79 percent of the tutoring sessions that were 

available to them (once school closures and holidays were accounted for). 

Summary 

In summary, in this evaluation the Reading Partners program had a positive and statistically 

significant impact on the reading proficiency of the students who participated in the program. 

•	 Reading Partners had a positive and statistically significant impact on all 

three measures of student reading proficiency, with effect sizes of 0.10 on 

reading comprehension scores, 0.09 on reading fluency scores, and 0.11 on 

sight word efficiency scores. This is equivalent to approximately one and a 

half to two months of additional progress in reading relative to the control 

group. 

•	 Since students in the control group were also receiving supplemental reading 

services, these impacts should be interpreted as the impact of Reading Partners relative to other supplemental services receipt, not the impact of Reading Partners compared with no intervention. 

•	 The impacts were equivalent to what a student learns in approximately one 

and a half to two months of additional school and are therefore somewhat 

larger than might have been expected given the amount of additional instructional time that students in the program received — 28 hours, or approximately one month. 

64 
 

 

 

 

 

 

 

 

 

•	 Impacts did not differ significantly for students from different grade levels, 

for male or female students, for those who were still learning English, or for 

those who had previously been served by the Reading Partners program. 

•	 Reading Partners had a positive and statistically significant impact on the 

percentage of students who moved out of the lowest quartile nationally in 

terms of reading comprehension. At the end of the year, 19 percent of the 

program group students who had scored in the bottom quartile nationally at 

the start of the study had moved out of the lowest quartile, as opposed to only 

12 percent of the control group. 

These findings all suggest that the Reading Partners model “works” — that is, it produces measurable impacts in reading skills among participants. The findings indicate that the 

impacts may have been largest for those readers who struggled the most; however, the findings 

also indicate that the program was effective in helping students with a fairly broad range of 

reading abilities and from a variety of grade levels and backgrounds. Reading Partners produced 

these measurable and consistent impacts despite the lack of prior experience among tutors, the 

somewhat limited training they received, and the relatively short duration of the tutor-student 

relationships. Chapter 6 includes a discussion of the components of the Reading Partners model 

and which components may have contributed to the impact findings presented in this chapter. 

65 

 

Chapter 5 

The Cost of Reading Partners 

Cost studies in education provide important information to aid the policymaking process.1 They 

often take one of three forms: (1) a benefit-cost analysis in which the outcomes of a program are 

expressed as monetary benefits and related to the costs to determine whether the benefits of the 

program outweigh the costs; (2) a cost-effectiveness study that compares the relative costs and 

effects of two or more programs that target equivalent outcomes; and (3) a cost study that 

describes the costs of a program, based on a comprehensive list of the resources (or “ingredients”) used to implement the program, and analyzes which entities bear the burden of financing 

those costs.2 Since reading achievement is not easily expressed in dollars, a benefit-cost analysis 

could not be easily undertaken for this study. Similarly, a cost-effectiveness analysis could not 

be conducted because the evaluation did not include an alternative program with which Reading 

Partners could be directly compared. Therefore, this evaluation takes the third approach and 

provides a rich description of the costs of Reading Partners, as well as the costs of the other 

supplemental services available in a sample of the evaluation schools. 

Two types of “costs” are examined in this chapter. The first is the value of all goods and 

services used to implement the Reading Partners program, including in-kind (such as the classroom space and the time school staff spend selecting students for the program) or donated 

resources. This is what is often referred to as the “total cost” of the program.3 The total cost is 

useful for thinking about the resource richness of the program and when comparing the 

resources used to implement various programs. The second type of cost discussed is the direct 

cost of the program borne by the school (that is, the portion of the total costs financed by the 

school or school district). Since principals, superintendents, or school boards are the agents 

making the decision to bring Reading Partners into a school, the cost the school bears is relevant. To calculate cost, the ingredients method is used. This method is a rigorous approach to 

conducting cost studies that accounts for all resources that are used to achieve educational 

gains.4 The full resource cost, or the “total cost” to society, includes the value of all the ingredients, regardless of who paid for or contributed them.5 The cost borne by schools includes 

the value of the ingredients that the schools provided — both in-kind and financial. (Not all 

analyses of costs include the cost of in-kind contributions. Therefore, comparisons of the costs 

1

Hummel-Rossi and Ashdown (2002); Levin and Belfield (2013); Levin and McEwan (2001); Ross, 

Barkaoui, and Scott (2007). 2

Levin (1975); Levin and McEwan (2001). 3

Levin and McEwan (2001) 4

Dhaliwal, Duflo, Glennerster, and Tulloch (2012); Harris (2008); McEwan (2002); Rice (1997). 5

Levin (1975). 

67 
 

 

 

 

 

 

 

 

 

 

 

presented in this chapter with those in other reports will not be meaningful unless the ingredients covered by the cost estimates are similar.) 

This chapter provides information about the resources needed to replicate the Reading 

Partners program at the level it was implemented in the study sites, and documents both the 

financial and in-kind resources that a school typically must provide to implement Reading 

Partners. In addition to presenting overall costs averaged across sites, the chapter covers how 

and why resources varied across sites. Studies have shown the importance of examining sitelevel variability in costs because understanding the pattern of resource use across sites may be 

important for evaluating a particular replication plan or for achieving a particular effect.6 

As discussed in Chapter 4, an evaluation of Reading Partners is incomplete without 

also examining the other supplemental reading services provided in the schools, because the 

impacts presented in Chapter 4 are the impacts of Reading Partners relative to an environment 

that already has other such programs. Thus, in addition to assessing the total resources needed 

to implement the Reading Partners program, this cost study was designed to provide site-level 

estimates of the average cost of the other supplemental services provided at each school in the 

cost study sample. Each site had between one and four other supplemental reading services 

included in the study. The services included tutoring, read-aloud programs, reading specialists, and computer-based programs. Some of these programs also used volunteers. The cost 

estimates for these other supplemental reading programs provide valuable information about 

the resource richness of Reading Partners in relation to the menu of other services that are 

typically provided for struggling readers. 

Specifically, this chapter addresses the following research questions: 

1.	 What resources are needed to implement the Reading Partners program as 

described in this evaluation and what proportion of the costs of implementing 

the program are borne by the school? 

2.	 What resources are needed to implement the other supplemental services 

available to struggling readers at a sample of sites that were included in the 

impact evaluation? 

6

Bowden (2014); Hollands et al. (2014); Levin, Catlin, and Elson (2007). 

68 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Methods 

The costs of Reading Partners and of the other supplemental reading services were estimated in 

six schools using the ingredients method, described below.7 

Cost Study Sample 

Six school sites were selected from the larger evaluation for inclusion in the cost study. 

The team selected sites that had strongly implemented programs, that were geographically representative of the other sites in the study, and where reliable data could be collected on the resources used in Reading Partners and other supplemental reading services offered during the 

2012-2013 school year. The sites were all elementary schools that varied in the size of the 

school (that is, the number of students enrolled) and in the number of students served by Reading Partners. Table 5.1 shows the characteristics of the cost study sample compared with the 

other Reading Partners schools included in the evaluation. The schools in the cost study had 

more students enrolled and were composed of slightly more white students than the rest of the 

evaluation sample but were otherwise comparable.8 In addition, the average impact on reading 

proficiency among these six sites was comparable with the full sample impacts, with effect sizes 

equal to 0.09 on each of the three reading proficiency measures. 

Data Collection 

All of the resources that were used to implement Reading Partners were identified from 

discussions with program administrators, interviews with and surveys of program and school 

staff, and data collected during the observational school site visits. The resources included all 

in-kind or donated ingredients such as the time contributed by the principal or the time contributed by community volunteer tutors. Even though Reading Partners enlists tutors who volunteer 

at no cost to the school, the volunteers are a necessary and important ingredient for the program 

to work. To properly document the full resource richness of the program, a value is given to 

these donated hours. They are valued at the price that their services would cost if they had to be 

purchased. This full-cost valuation allows an appropriate comparison of the resource richness of 

programs that receive donated time with those that do not. 

Site by site, the study team compiled the full set of ingredients needed to replicate the 

program as it was implemented at the site. Because the focus of the study is on the cost of 

7

Additional information on the ingredients method can be found in Appendix D (Levin and McEwan, 

2001). 8

This report does not reveal the total number of students served at each school or the number of students in 

the program group at each site to protect the identity of each site. 

69 
 

Reading Partners Evaluation
 

Table 5.1
 

Characteristics of Cost Study Schools
 

and Other Reading Partners Study Schools (2011-2012) 

Other 

Cost Study Reading Partners 

Characteristic Schools Study Schools Difference P-Value 

Eligible for Title I program (%) 83.33 91.67 -8.33 0.621 

Students eligible for 

free/reduced-price luncha (%) 77.96 83.43 -5.47 0.331 

Race/ethnicity (%) 

Black 28.14 17.32 10.82 0.337 

Hispanic 49.67 67.32 -17.66 0.120 

Asian 10.34 9.25 1.08 0.831 

White 9.43 3.98 5.45 * 0.072 

Other 2.43 2.12 0.31 0.798 

Male (%) 51.93 51.88 0.05 0.971 

Average number of students 612.33 439.62 172.72 ** 0.049 

Grade 2 97.33 70.15 27.18 0.158 

Grade 3 93.00 67.08 25.92 0.156 

Grade 4 87.00 68.08 18.92 0.276 

Grade 5 88.83 69.17 19.67 0.281 

School settingb (%) 

Urban 66.67 76.92 -10.26 0.659 

Suburban 33.33 23.08 10.26 0.659 

Town 0.00 0.00 0.00 NA 

Rural area 0.00 0.00 0.00 NA 

Sample size 6 13 

(continued) 

replication, or incremental costs, the costs of general program administration and development were not included. By separating cost by site, the study team was able to examine the 

range of resources that contributed to the operation of Reading Partners. 

Analytic Methods 

After a comprehensive list of ingredients was compiled, each ingredient was matched 

with its national price, or cost in 2012 dollars. National prices were obtained from publicly 

70 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 5.1 (continued) 

SOURCES: MDRC calculations from 2011 and 2012 National Center for Education Statistics 

Common Core of Data (CCD). 

NOTES: Sample sizes for individual outcomes may fall short of the reported sample sizes because of

missing or unusable data.

A two-tailed t-test is used for all statistical tests presented in this table. Statistical significance

levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

a

The value given for students eligible for free/reduced-price lunch is calculated from the 2011

CCD because data are missing in the 2012 CCD. Data for all other variables are from the 2012 CCD.

b"Urban" is defined as territory inside an urbanized area and inside a principal city having a 

population greater than 100,000. "Suburban" is defined as territory outside of a principal city and 

inside an urbanized area with a population of less than 250,000. “Town” is defined as territory inside 

an urban cluster that is (1) less than or equal to 10 miles from an urbanized area (“fringe”), or (2) 

more than 10 miles and less than or equal to 35 miles from an urbanized area (“distant”), or (3) more 

than 35 miles from an urbanized area (“remote”). “Rural” is defined as territory that is (1) less than or 

equal to 5 miles from an urbanized area and less than or equal to 2.5 miles from an urban cluster 

(“fringe”), or (2) more than 5 miles but less than or equal to 25 miles from an urbanized area and 

more than 2.5 miles but less than nor equal to 10 miles from an urban cluster (“distant”), or (3) more 

than 25 miles from an urbanized area and more than 10 miles from an urban cluster (“remote”). 

available sources such as the Bureau of Labor Statistics, the Department of Labor, the National 

Center for Education Statistics, and AmeriCorps. (More details are provided in Appendix D.) 

As already noted, volunteer time is a critical ingredient for Reading Partners, but it 

does not have a clear market rate. The price used must therefore be an approximate value of 

the ingredient based on similar resources with market prices.9 The price reported here is the 

hourly rate for a paraeducator or teacher’s aide.10 The role of the paraeducator and the role of 

the volunteer tutors (for both Reading Partners and the other supplemental reading programs 

that used volunteers) seemed most similar with regard to expectations and responsibilities. 

Thus, the full cost that is reported below represents the cost of the program including the costs 

reflected by the volunteers. This estimate could also be interpreted as the cost of these services if the tutors had to be paid. 

Each Reading Partners program site served up to 80 students. However, only a portion 

of those students served were included in the evaluation, either because eligible students were 

identified after randomization occurred or because the school served students in lower grade 

levels that were not included in the evaluation. The data collected reflect the services provided 

9

Levin and McEwan (2001).

10However, as described in Appendix D, sensitivity tests using minimum wage and tutor education levels 

provide lower and upper bounds for the price of the volunteers. 

71 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

to all students who participated in Reading Partners and not just those in the evaluation sample. 

Therefore, the calculation of the cost per student must be adjusted to account for the additional 

students who were accommodated; otherwise, the cost per student would be overstated. The 

term “per program group student” is used throughout the report to refer to the students who 

were randomly assigned to participate in the Reading Partners program as part of the evaluation 

(the “program group”) and to distinguish them from the total pool of students served. 

To calculate the cost per program group student, first the total cost (or the total resource 

value) of the program at each site was divided by the total number of sessions provided to all 

students who received Reading Partners’ services during the year (not just the sessions provided 

to the study’s program group students) to obtain the cost per session. Then, to determine the 

average per-student cost of serving the program group, the average cost per session was multiplied by the average number of sessions that program group students attended. 

The resources needed to run the Reading Partners program come from many different 

sources, including schools or school districts, volunteers from the community, the AmeriCorps 

program, and the Reading Partners organization. Therefore, the costs borne by each constituent 

are described below, with a special emphasis on the costs borne by the schools. When the costs 

to the school are reported, the estimate is based on the resources contributed by the school inkind as well as the fee-for-service paid by the school to Reading Partners for the program. 

These Reading Partners costs are examined relative to the cost of the other supplemental reading services provided in the school. This analysis shows the relative “richness,” or 

resource intensity, of Reading Partners, as well as the relative cost of Reading Partners to the 

school. The costs of the other supplemental reading services provided at each school were calculated in the same detailed way and followed the same pricing approach using national prices. 

The costs of the other supplemental programs were aggregated by averaging them at the site 

level to protect confidentiality. 

Findings 

The total resource value, or cost, of Reading Partners is approximately $3,610 per program 

group student, of which $1,910 reflects in-kind resource contributions made by schools and the 

community volunteers and the remaining $1,700 reflects the financial costs borne by the Reading Partners program, the AmeriCorps program, and the schools. (See Table 5.2.) As discussed 

in Chapter 6, other effective early literacy interventions that have been evaluated at scale are at 

least as costly as Reading Partners.

11 However, unlike many other resource-rich programs, a 

11Hollands et al. (2013); Simon (2011). 

72 
Reading Partners Evaluation 

Table 5.2 

Cost of Reading Partners per Program Group Student 

Distribution of Cost per Student 

Cost to 

Cost per Cost to Cost to Reading Cost to 

Cost Student School Volunteers Partners AmeriCorps 

Ingredients ($) 

Reading Partners staff 690 690
 

AmeriCorps members 930 930
 

School staff 90 90
 

Volunteer time and transportation 1,520 1,520
 

Facilities 300 300
 

Materials and equipment 80 80
 

Total ingredients 3,610 390 1,520 1,700 0 

Fee for servicea

 ($) 320 -320 

AmeriCorps grantb ($) -270 270 

Net cost per student (total ingredients +
 

fee for service + AmeriCorps grant) ($) 710 1,520 1,110 270
 

Portion of net cost per student (%) 20 42 31 7 

SOURCES: MDRC calculations from cost data. 

NOTES: Costs are measured in 2012 U.S. dollars and are rounded to the nearest 10 in the tables and 

text. 

Averages are weighted to account for the size of the program group at each site. 

All costs are those for the respondents in the program group and are adjusted for the larger dosage 

they received. 

Rounding may cause slight discrepancies in sums and differences. aEach school pays a fee to Reading Partners to finance a portion of resources provided by the 

organization, such as staff and materials. The fee presented here is the national average fee per student. 

bAmeriCorps provides a grant to Reading Partners to finance approximately 30 percent of the costs of 

the AmeriCorps members employed by the program. Reading Partners pays the remainder of the costs. 

majority of the costs of Reading Partners is covered by in-kind contributions. Furthermore, 

Reading Partners schools bear only a small portion of the total costs of the program. Because 

the program relies heavily on volunteers, on average schools bear only 20 percent ($710 per 

program group student) of the cost of the total resources required to implement the program, 

and over half of these costs are covered by in-kind contributions. The cost findings for Reading 

Partners are discussed in more detail below, along with the costs of the other supplemental reading services provided at the schools. 

73 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The School’s Portion of the Total Cost of Reading Partners 

While the total resource value of the Reading Partners program is $3,610, the schools 

themselves paid only a small proportion of these costs. As shown in Table 5.2, the schools in 

the study contributed both in-kind resources and a fee to finance the program. The in-kind resources included a classroom for the reading center, space for program events, and time from 

the principal, teachers, administrative staff, and the reading specialist or coordinator. The value 

of these in-kind resources is approximately $390 per program group student, of which $300 is 

the cost of the space. In addition to space and time, the schools also covered a portion of the 

resources provided by the Reading Partners sponsor by paying a fee of $320 per program group 

student to Reading Partners. Thus, the school’s commitment of $710 per student leveraged almost $3,000 of additional resources, a majority of which is provided by the volunteer tutors. 

Volunteers’ Contribution to the Cost of Reading Partners 

In addition to the costs financed by the school, the remaining costs of the program were 

borne by three agents: the Reading Partners organization, the AmeriCorps program, and the 

volunteers themselves. 

As shown in Table 5.2, the largest proportion of the resources used in implementing the 

program (42 percent of the total) was financed through in-kind donations from the community 

volunteers. Volunteers contributed, on average, the equivalent of $1,520 per program group 

student, which included both their time and transportation costs, for which they were not reimbursed.

12 The Reading Partners organization financed about 31 percent of the required resources 

($1,110 per program group student), which included both staff and materials. The site coordinator and outreach coordinator positions are partially financed by the AmeriCorps program. 

AmeriCorps provided a grant to Reading Partners to finance approximately 30 percent of the 

costs of the AmeriCorps members. Thus, AmeriCorps funding finances about 7 percent of the 

total cost of Reading Partners. 

In addition to considering the costs borne by each of the entities and the volunteers, it 

is also important to consider what proportion of the total costs were in-kind contributions. In 

the Reading Partners program, the resources contributed in-kind by the schools and community volunteer tutors totaled over 50 percent of the total cost, or $1,910 per program group student. The remaining $1,700 per program group student was financed by the Reading Partners 

organization and the AmeriCorps program, with some financial assistance from the schools 

themselves. 

12Time and costs for travel are considered to be part of the in-kind donation made by the volunteers. The 

same approach was used for other programs that relied on volunteers. 

74 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Costs of Other Supplemental Reading Services 

In addition to Reading Partners, the six school sites that were included in the cost study 

offered other supplemental services that provided reading instruction beyond what students received during regular classroom teaching. In many cases, services were provided to students in 

need of additional support, irrespective of whether they had been assigned to the Reading Partners evaluation program group or control group. The costs of these other supplemental reading 

services are included here to provide a context for understanding the estimated level of resource 

richness (as measured by total cost) for Reading Partners. 

The other supplemental reading services included small-group support from reading 

specialists, tutoring programs, homework help, and computer-based programs. Table 5.3 shows 

the average cost per student for the other supplemental reading services at each school site. The 

costs of the other supplemental reading services are averaged at the site level to protect confidentiality, so that the cost per student represents the average cost per student of all supplemental 

programs at each school, weighted by the number of students served by each program. The 

number of services offered in each school ranged from one to four services in addition to Reading Partners. Some of these averages combine relatively resource-rich interventions (for example, a learning specialist who works intensively with small groups of children) and with interventions that require fewer resources (such as computer-based programs overseen by reading 

specialists or teachers on special assignment to supplement reading instruction). 

The cost per student for the average of the other supplemental reading services offered at the six sites was $1,780 and ranged from $1,050 to $4,890 per student. The range in 

the total resources provided by the supplemental services across the six sites depended on the 

type of supplemental services that were offered, which varied considerably in the resources 

they required. For example, computer-based programs serve groups of students on computers 

that the school also uses for other purposes, the costs of such programs tended to be relatively 

low (on average, around $940 per student in the study sites) when compared with more intensive programs. Reading specialists or other full-time school support staff who worked with 

small groups of students several times each week in a dedicated classroom tended to be more 

expensive (around $3,200 per student, on average, in the study sites). 

Table 5.3 also shows that although Reading Partners provides a rich set of resources to 

schools, the portion of costs borne by the school is substantially lower than for the other supplemental reading services provided. As already explained, the average cost to the school for 

Reading Partners was $710 ($480 to $1,270) per program group student, while the average cost 

of the other supplemental reading services borne by the school or school district was $1,700 

($1,040 to $4,890) per student. Most of the other supplemental reading services were funded 

solely by the school or school district. The few that were not fully funded by the schools relied 

on in-kind donations from volunteers or were subsidized by the program itself. 

75 
 

 

 

 

 

 

 

76 

Reading Partners Evaluation 

Table 5.3 

Site-Level Costs of Reading Partners and Other Supplemental Services 

Sitea 

Other 

Reading Supplemental 

Partners Servicesc 

School Contribution per Studentb ($) 

Other 

Reading Supplemental 

Partners Servicesc 

Total Resources per Student ($) 

Other 

Reading Supplemental 

Partners Servicesc 

Contribution Provided by School (%) 

Site A 690 1,840 3,450 1,840 20 100 

Site B 520 1,850 3,420 2,230 15 83 

Site C 940 2,680 3,570 2,680 26 100 

Site D 1,270 1,040 5,190 1,050 25 99 

Site E 660 1,310 4,210 1,980 16 66 

Site F 480 4,890 2,740 4,890 17 100 

Pooled 710 1,700 3,610 1,780 20 96 

SOURCES: MDRC calculations from cost data. 

NOTES: Costs are measured in 2012 U.S. dollars and are rounded to the nearest 10 in the tables and text. 

Averages have been weighted to account for the size of the program group (for Reading Partners) and students served at each site (for other 

supplemental services). 

All the Reading Partners costs in this table are those for the respondents in the program group and are adjusted for the larger dosage they 

received. The costs for supplemental services are those for all the students served at each site. aTo prevent identification of individual schools or programs, site letter identifiers are not consistent across analyses. 

bCosts borne by school include the fee for services paid by schools. 

cOther supplemental reading services include small-group support from reading specialists, tutoring programs, homework help, and 

supplemental curricula. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Thus, the portion of the costs borne by the school is greater for the other supplemental 

reading services than for Reading Partners, where most of the costs were covered by in-kind 

contributions from the volunteers and by funds from the Reading Partners organization itself. 

The portion of the costs borne by the school for Reading Partners was 20 percent on average (15 

percent to 26 percent), whereas the average portion of the costs of the other supplemental reading programs financed by the school was 96 percent (66 percent to 100 percent). Some of the 

other supplemental reading services offered at these schools may not be direct substitutes for 

Reading Partners because they serve students of differing levels of need, or a school may be 

required to provide a particular service (such as a Tier III reading specialist) where the principal 

is not able to select between the service and a program such as Reading Partners. 

Overall, schools provided students with a menu of supplemental reading services worth 

about $5,400 on average ($3,610 of Reading Partners services and $1,780 of other supplemental 

services). The average school was able to contribute around $2,400 worth of supplemental reading services per student for the students in their schools ($710 for Reading Partners plus $1,700 

in additional supplemental programs). The Reading Partners program, in part because it uses a 

volunteer-based model, provided schools with almost $3,000 worth of additional resources per 

program group student ($3,610 less the $710 contributed by the school) that the schools would 

otherwise not have been able to provide. 

Variability in Costs 

Table 5.3 also highlights the variation in the total resources devoted to Reading Partners 

across the six sites in the sample. The total resources used to provide Reading Partners ranged 

from a low of $2,740 to a high of $5,190 per program group student. The costs borne by the 

school ranged considerably, from a low of $480 per program group student to a high of $1,270 

per program group student. There were a number of sources of variation. First, there was variation in the amount of time devoted by Reading Partners staff and AmeriCorps members, which 

was, in part, dependent on the number of students served in the school. Second, there was variability in volunteer costs across schools, also depending on the number of students served, as 

well as differences in commuting time and transportation costs. At the school level, variation in 

costs depended on the amount of time school principals and teachers spent in coordination with 

the Reading Partners program, as well as some variation in space assigned to the program. The 

time devoted to referrals and formal or informal check-ins by the principal, for example, ranged 

from 30 minutes per year to over 10 hours per year across the sites. Teachers in all schools 

tended to spend some time on referrals or communicating with the site coordinator (or both). 

This time varied widely from 11 hours in total per year to over 3 hours per student per year. The 

space the schools were able to make available for the program also varied. As discussed in 

Chapter 2, most schools provided a dedicated classroom for the program, but some schools used 

space in the library or other available space rather than a standard classroom. 

77 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Variation in costs was also due to variation in the amount of services provided. Some 

sites offered more tutoring sessions per program group student than did others over the course 

of the school year. Sites that offered fewer sessions used fewer resources. The difference in the 

number of sessions provided per program group student between the sites with the highest cost 

and the lowest cost was about 10 sessions over the course of the year. 

However, for the most part the variability observed among schools was largely tied to 

scale — or the number of students served by the program. This is because serving more students allows the program to amortize its fixed costs (such as space and the site coordinator) over 

more students. The school sites with the lowest cost per student were those that served the highest number of students and provided the highest number of sessions per student. 

Scale Test 

A sensitivity test was included to adjust for scale by using the operational maximum 

number of students based on the space and equipment recommended by Reading Partners for 

each reading center. The program recommends eight distinct spaces for tutor-student pairs to 

read and work together during each session. The reading centers tended to serve students for 

five class periods each day, four days a week. Accordingly, each site’s costs were adjusted to 

ascertain the costs if 80 students were served. This analysis is discussed in detail in Appendix D. 

The results, presented in Appendix Table E.1, show that the total cost per student drops to 

$2,390 on average ($2,230 to $2,700). Because the total costs borne by the school were assumed to be fixed (up to 80 students), the portion of the total resources used to implement Reading Partners borne by the school dropped from about 20 percent to 14 percent. This drop suggests that some schools could reduce cost per student substantially by serving more students. 

Thus, if a school has eligible students who are not served by the program, the site could 

improve the efficiency of the program by increasing the number of students served. However, 

the space available at a school may not be large enough to support 80 students or, alternatively, 

the community may not have sufficient volunteers to support expanded services. Additionally, it 

could be the case that a school would increase the time invested by staff when more students are 

involved, increasing the cost to the school. Yet, even a small increase in the number of students 

served may reduce variability in costs among sites and improve efficiency substantially. 

Conclusions 

Schools pay $320 per student out of pocket for Reading Partners and provide $390 of in-kind 

resources, such as space, to support the program. For this outlay, the school receives a program 

worth, on average, $3,610 per student ($2,740 to $5,190) to supplement reading instruction for 

students who need support. 

78 
 

 

 

 

 

 

 

The cost borne by the school to implement Reading Partners is about $1,000 lower per 

student than the average cost for other supplemental reading services being provided. The other 

supplemental reading services provided to the study sample varied in intensity and purpose, but 

all were financed almost solely by the schools or school districts. Reading Partners provided 

schools with a twice-weekly tutoring program for students who were half a year to two and a 

half years behind, in which most of the costs are borne by outside entities (that is, Reading Partners, AmeriCorps, and in particular, the volunteers). 

One caution should be noted. When comparing the cost numbers presented in this chapter with those in other reports, readers should make sure that the resources covered by the cost 

estimate are similar. In this cost analysis, all ingredients, including those contributed in-kind 

such as space, school staff time for collaboration, volunteer transportation, and volunteer time, 

were included in the total cost estimate. Many cost studies rely on budgetary data, ignoring inkind resources, and thus might not be comparable with the numbers reported here. 

Overall, the findings of the cost study illustrate the high value of the Reading Partners 

program. The program used many resources (on average $3,610 per program group student), of 

which over half ($1,910) is contributed in-kind. Furthermore, the school bore only 20 percent of 

the costs of the program. Because Reading Partners relies largely on volunteers and is provided 

by an external organization, schools can capitalize on substantial resources without incurring a 

large burden. 

79 

 

 

 

 

 

 

 

 

 

 

 

 

Chapter 6 

Conclusions 

This evaluation finds a positive impact of the Reading Partners program on three different 

measures of reading proficiency, and the impacts appear to accrue to a wide range of students 

with differing incoming abilities, genders, and grades, including English language learners. 

Exploratory analyses also suggest that the impact of the program was largest for those students 

beginning the program with the lowest skills. Moreover, while the Reading Partners program is 

resource-intensive (as would be expected of a one-on-one tutoring program), a majority of the 

resources are in-kind contributions and schools and school staff bear a small portion of the 

financial burden of implementing the program. 

The findings from this evaluation are important because very few programs that have 

been rigorously evaluated at scale (using a randomized trial with a sample of more than 300 

students from multiple schools) have been shown to improve reading outcomes for elementary school children. As shown in Table 6.1, the study team was able to identify only five 

programs that met these criteria, and they are divided into two groups — four one-on-one 

tutoring programs (Reading Partners, Experience Corps, Minnesota Reading Corps, and 

Reading Recovery) and one whole-school reform program (Success for All). The tutoring 

programs are similar in that they all provide one-on-one instruction and have a staff member 

who is fully committed to managing the tutoring program.1 Success for All, on the other hand, 

includes a classroom-based curriculum, and coordinators must manage the many components 

of the program model.2 However, the Reading Partners model is different, along a number of 

dimensions, from each of these. 

First, Reading Partners is unique among rigorously tested tutoring programs in that service delivery relies almost exclusively on community volunteers who devote relatively short 

amounts of time each week to tutoring. The Reading Recovery program uses highly trained, 

certified teachers to provide reading instruction to struggling elementary school students. Not 

surprisingly, the program has been very successful in improving the reading proficiency of very 

low-achieving first-graders. Minnesota Reading Corps relies on AmeriCorps members who 

commit to participate in the program either half time or full time and are provided with uniform 

and comprehensive training.3 Similarly, Experience Corps volunteer members affiliated with 

the 

1

Lee, Morrow-Howell, Jonson-Reid, and McCrary (2010); Markovitz, Hernandez, Hedberg, and Silberglitt (2014); May et al. (2013). 2

Quint et al. (2013); Quint et al. (2014).

3

Markovitz, Hernandez, Hedberg, and Silberglitt (2014). 

81 
Reading Partners Evaluation
 

Table 6.1
 

Reading Programs with Rigorous Evidence Base
 

 

 

82 

Reading 

Program 

Program 

Description 

Population 

Served 

Service 

Providers Time 

Impacts for Students 

in Kindergarten or 

Grade 1 (Effect Size) 

Impacts for Students in 

Grade 2 or 3 (Effect Size)a 

ONE-ON-ONE TUTORING PROGRAMS 

Reading 

Partners 

Community 

volunteers work 

with students 

using a structured 

curriculum 

Elementary 

students 6-30 

months below 

grade level in 

reading 

Volunteers of 

varying 

backgrounds 

and ages 

45 minutes 

twice a 

week 

— 0.11 – Sight-word efficiency 

0.09 – Fluency 

0.10 – Comprehension 

Experience 

Corps 

Older adult 

volunteers tutor 

elementary school 

students 

Elementary 

students (grades 

1-3) at risk of 

academic failure 

Volunteers 

age 55+ 

60 to 160 

minutes per 

week 

0.10 – Decoding 

0.13 – Comprehension 

0.10 – Decoding 

0.13 – Comprehension 

Minnesota 

Reading 

Corps 

AmeriCorps 

members tutor 

elementary school 

students using 

scripted 

interventions 

Elementary 

students (K-3) at 

risk of academic 

failure 

AmeriCorps 

members 

20 minutes 

daily 

1.06 – Letter sounds 

0.37 – Decoding 

0.10 – Fluency 

Reading 

Recovery 

Teachers provide 

supplemental, 

individualized 

instruction to 

Lowest-achieving 

15%-20% of 

1st-grade readers 

Specially 

trained and 

supervised 

teachers 

30 minutes 

daily 

0.68 – Overall reading — 

students 

(continued) 
Table 6.1 (continued) 

 

 

 

83 

Reading 

Program 

Program 

Description 

Population 

Served 

Service 

Providers Time 

Impacts for Students 

in Kindergarten or 

Grade 1 (Effect Size) 

Impacts for Students in 

Grade 2 or 3 (Effect Size)a 

WHOLE-SCHOOL REFORM PROGRAM 

Success 

for Allb 

Classroom-based 

literacy instruction 

that includes 1:1 

tutoring in the 

context of a wholeschool reform 

model 

All students 

in the school; 

lowest-achieving 

students in 

grades 1-3 who 

are eligible for 

tutoring 

Elementary 

school 

teachers 

1.5 hours 

instruction 

and 20 

minutes 

tutoring 

daily 

0.09 – Word identification 

0.18-0.35 – Decoding 

0.22 – Word identification 

0.33 – Decoding 

0.21 – Comprehension 

SOURCES: Borman et al. (2007); Lee, Morrow-Howell, Jonson-Reid, and McCrary (2011); Markovitz, Hernandez, Hedberg, and Bilberglitt 

(2014); May et al. (2013); Quint et al. (2013); Quint et al. (2014). 

NOTES: Programs in this table had to be serving at least 300 students across multiple schools and evaluated using a randomized controlled trial. 

Only impacts on standardized student assessments significant at p < 0.1 have been included. a

The sample for the Reading Partners impacts provided in this column includes 2nd- through 5th-graders. The other evaluation samples in this 

column include 2nd- and 3rd-graders only.

b

Success For All findings come from three different reports (May et al., 2013; Quint et al., 2013; Quint et al., 2014). Findings for grades 2-5 

come from the randomized field trial, and those for kindergarten and grade 1 come from the ongoing i3 scale-up evaluation. 
 

 

 

 

 

 

 

 

 

 

 

 

AmeriCorps program commit varying amounts of time (between 4 and 15 hours a week depending on their location) to tutoring and receive between 15 and 32 hours of training.4 

Reading Partners, on the other hand, empowers its school-based AmeriCorps members 

to run reading centers and manage and support a corps of adult volunteers, who, with minimal 

background or training, then provide services to students for up to a few hours each week. The 

reliance on community volunteers means that the Reading Partners program has the capacity to 

serve large numbers of students within a school (up to 80 in a given week) with the support of 

only one AmeriCorps member (the site coordinator), because several tutors can engage in tutoring at the same time. Despite the limited training and the relatively low level of commitment 

required of the Reading Partners volunteers, this study still found that the program had a positive impact on students’ reading proficiency scores. 

Second, Reading Partners serves a wider range of students, particularly with respect to 

age and ability, than do the other tutoring programs that have been rigorously evaluated. Most 

elementary school reading interventions that involve tutoring focus almost exclusively on students in kindergarten through grade 3, and tend to find large impacts in kindergarten and 1st 

grade. Fewer have demonstrated impacts on students in upper elementary school grades (4 and 

5). Reading Partners, however, serves students through grade 5, and this evaluation finds a positive impact of the program on the reading comprehension skills of students in the upper elementary grades. Similarly, some programs, like Reading Recovery, target only the very lowestachieving students. Reading Partners, on the other hand, serves students across a wider range of 

incoming reading abilities — six months to two and a half years below grade level. This means 

that students who are struggling but might not otherwise be served by the school are able to 

receive additional support. As shown in Chapter 4, Reading Partners is broadly effective across 

the various student subgroups explored by the research team. Impacts did not differ significantly 

between the lower elementary cohort (students in grades 2 and 3) and students in the upper elementary cohort (students in grades 4 and 5), and although the impacts were largest for those 

students who were lowest-achieving when the study began, the impacts did not differ significantly between the various achievement quartiles. Finally, the program was also effective with 

English language learners, a group who face additional challenges in learning to read English. 

The low cost to schools of Reading Partners, especially relative to the costs of other 

types of supplemental reading programs, makes these findings particularly relevant for school 

administrators who are looking for practical ways to help the large numbers of elementary 

school students who are reading below grade level. For example, reports on Reading Recovery 

indicate that the total resource value of that program is about $4,400 to $7,300 per student, 

4

AARP Experience Corps (2014); Lee, Morrow-Howell, Jonson-Reid, and McCrary (2010). 

84 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

depending on the implementation.5 Similarly, the total resource value of Success for All is estimated to be around $13,000 per student.6 Almost all the costs of these programs are borne by 

the schools.7 Thus, volunteer tutoring programs like Reading Partners are able to channel resources to schools that otherwise might not have been available. 

There have been no cost studies of the Minnesota Reading Corps or Experience Corps 

programs, although they rely on a similar cost structure to that of Reading Partners. For example, the Minnesota Reading Corps program does not require schools to pay a fee for service and 

most of the resources are provided by the Reading Corps program, but schools are required to 

devote a percentage of time for a staff person who serves as the “internal coach” for the program. The internal coach is based at the school and provides ongoing literacy support and oversight to the AmeriCorps tutors.8 

Factors Contributing to the Observed Outcomes 

Given the findings described above, what factors in particular helped contribute to the program’s success? Previous research has indicated that a number of different program characteristics may lead to the success of tutoring programs in general and volunteer programs in particular. Although the findings are somewhat mixed, six factors have been cited in the literature 

as instrumental to the success of volunteer tutoring programs: a structured curriculum, the 

alignment between the classroom and tutoring curricula, tutor background, tutor training, the 

consistency of the tutoring and the tutoring relationship, and one-on-one instruction.9 Reading 

Partners has some, but not all, of these components. 

A Structured Curriculum 

Previous research has found that programs with either a structured curriculum or a 

structured division of time and activities had a larger impact on global reading outcomes than 

unstructured tutoring programs.10 Other studies also note the importance of having structure in 

volunteer programs.11 The structured use of time and materials is helpful because it ensures that 

consistent and uniform instruction is given on a regular basis and because it allows tutors with 

limited backgrounds or training to implement the program successfully. Reading Partners has a 

5

Costs are adjusted to 2012 dollars. Hollands et al. (2013); Simon (2011). 6

Costs are adjusted to 2012 dollars. Simon (2011). 7

Simon (2011).

8

Minnesota Reading Corps (2014).

9

Elbaum, Vaughn, Hughes, and Moody (2000); Ritter, Barnett, Denny, and Albin (2009); Slavin, Lake, 

Davis, and Madden (2011); Wasik (1998). 10Ritter, Barnett, Denny, and Albin (2009). 11Slavin, Lake, Davis, and Madden (2011); Wasik (1998). 

85 
 

 

 

 

 

 

 

 

 

 

 

very structured curriculum covering a wide range of reading skills — and each skill is broken 

into a specific lesson with accompanying text, materials, and activities that are specific to that 

skill. Tutors widely said that the structure of the lessons allowed them to jump right in, even 

with minimal training. On the other hand, the Experience Corps program, which has a somewhat similar model to Reading Partners and found similar impacts, includes no curriculum at all 

— volunteers use whichever curriculum is in place at the school where they are tutoring — and 

that program has been shown to be effective as well.12 It is possible that this model works for 

Experience Corps because its members receive more training and spend more time tutoring than 

do Reading Partners’ volunteers. 

Alignment with Classroom Reading Curricula 

Some have argued that coordination between the curriculum used in the classroom and 

that used in the tutoring intervention is important because it provides children with continuity in 

their approach to learning and prevents any confusion or discrepant approaches to instruction.

13 

For example, the Experience Corps program closely integrates tutoring with classroom instruction.14 Reading Partners, on the other hand, does not. The Reading Partners curriculum in use at 

the time of this study was aligned with California state standards (and Reading Partners is working to ensure that the program is well aligned with Common Core State Standards), but it is not 

tailored to the specific approaches or curricula used by individual schools. Instead, Reading 

Partners encourages staff members to communicate regularly with teachers and schools about 

individual student progress. However, the program managers and site coordinators who were 

interviewed for this study indicated that there was some variation in the types of communication 

that occurs and the coordination that results when knowledge is shared. In many cases, Reading 

Partners staff members merely provide progress updates to teachers or discuss troubling behavior; in other cases, the site coordinator might work with tutors in an attempt to align the students’ Reading Partners lessons with content being covered in the classroom. Thus, coordination with the school curriculum does not appear to be a critical factor leading to the effects 

found in this evaluation. 

Selection and Background of Tutors 

Previous research suggests that tutor background and experience may have an impact 

on effectiveness. For example, one meta-analysis found that programs using college-going tutors produced especially large gains for students.15 The tutors involved in Reading Partners have 

12Lee, Morrow-Howell, Jonson-Reid, and McCrary (2010). 13Wasik (1998).

14Lee, Morrow-Howell, Jonson-Reid, and McCrary (2010). 15Elbaum, Vaughn, Hughes, and Moody (2000).
 

86 
 

 

 

 

 

 

 

 

 

 

 

 

 

a wide range of backgrounds and experience, especially with regard to the amount of prior exposure they had to formal tutoring or working with struggling readers. However, the study team 

found no association between student test score gains and the tutor’s background. Thus, tutor 

background does not appear to be a key factor driving the results of this evaluation. 

Tutor Training 

Literature on volunteer tutoring indicates that tutor training is key to the success of a 

tutoring program, and training for the direct service providers is extensive in the Experience 

Corps, Minnesota Reading Corps, and Reading Recovery programs.16 Experience Corps tutors 

receive between 15 and 32 hours of training; Minnesota Reading Corps members receive four 

days of training; and Reading Recovery teachers complete university-level coursework as part 

of their training.17 While the training for Reading Partners full-time staff is comprehensive and 

ongoing, the amount of initial training that volunteers receive is minimal. As described in Chapter 4, most tutors receive only an orientation to the reading center and participate in a single 

shadow session before beginning to tutor students. The total time required for this up-front 

training is about an hour. Reading Partners does provide regular, ongoing training opportunities 

for tutors focused on particular challenges and student skills, but across the board Reading Partners staff reported that attendance at these training sessions was low. It seems unlikely that this 

single hour, during which the volunteer is not actively engaging with the curriculum or practicing techniques, is what is allowing these tutors to successfully help their tutees. On the other 

hand, site coordinators consider it one of their primary responsibilities to monitor and support 

tutors as necessary, and many tutors described their site coordinators as being particularly helpful and responsive to issues that they encountered with students. A meta-analysis of mentoring 

literature found that ongoing support was associated with bigger impacts than initial training.18 

Perhaps, while formal training is not the key, structured and accessible support infrastructures 

can partially explain how tutors with minimal training are successful, at least within the context 

of the Reading Partners program. 

Consistency of the Tutor-Student Relationship 

Research also suggests that the consistency of the tutor’s relationship with the student 

may be an important factor in improving academic outcomes.19 Wasik cites the importance of 

this relationship as creating trust in the student, an understanding of student needs for the tutor, 

16Elbaum, Vaughn, Hughes, and Moody (2000); Wasik (1998).

17Lee, Morrow-Howell, Jonson-Reid, and McCrary (2010); Markovitz, Hernandez, Hedberg, and Silberglitt (2014); May et al. (2013). 18DuBois, Holloway, Valentine, and Cooper (2002).

19Elbaum, Vaughn, Hughes, and Moody (2000). 

87 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

and a sense of stability to both.20 Research on mentoring also found that a mentoring pair with a 

strong relationship led to improved academic outcomes, whereas mentoring absent a close relationship did not significantly affect academic performance.21 A snapshot of tutor consistency 

over a two-week period as pulled from a random sample of Reading Partners student folders 

indicates that individual tutor consistency was not particularly high in this study. Among students who were tutored four times or more over a two-week period, almost 60 percent saw more 

than two tutors, suggesting that tutor consistency was somewhat limited. Moreover, at the 

school level, the average session-to-tutor ratio varied from 1.25 sessions per tutor to almost 2.25 

sessions. The average number of assigned tutors and the average number of tutors seen over a 

two-week period was not related to outcomes at the site level. 

One-on-One Instruction 

Even if Reading Partners students are not consistently tutored by the same volunteer, 

they still receive consistent individualized tutoring. On average, students participated in about 

1.5 sessions per week, and all tutoring sessions were individual. Research on mentoring suggests that individualized attention can be key to helping students succeed in that it allows for 

relationship development.22 As it relates specifically to reading instruction, evidence suggests 

that one-on-one tutoring is more effective than small-group instruction.23 To that end, all of the 

tutoring-specific programs included in Table 6.1 involve one-on-one work with students. The 

one-on-one nature of these programs is likely part of the key to their success, but what makes 

Reading Partners unique is that this model is implemented with volunteers who devote relatively short amounts of time to tutoring. Not only does the program staff have to recruit and retain a 

large cohort of volunteers for each site, they also must have in place the infrastructure to ensure 

that one-on-one tutoring can happen during anticipated or unexpected tutor absences. Their ability to consistently and reliably provide individualized tutoring in spite of volunteer turnover ensures that Reading Partners services are delivered with fidelity to their model and likely helps 

explain some of the impacts found in this evaluation. The use of volunteers to provide one-onone tutoring and the resulting minimal cost to schools also makes Reading Partners financially 

viable in communities with limited resources for supplemental programs. 

In summary, several of the characteristics that are cited as contributors to the success of 

tutoring programs are not a part of the Reading Partners model. The program does not require 

any prior tutoring or teaching experience, and volunteers come from a variety of backgrounds 

and experiences. Tutors participate in only limited formal training and tutor consistency is vari20Wasik (1998).

21Bayer, Grossman, and DuBois (2013).

22Bayer, Grossman, and DuBois (2013). 23Slavin, Lake, Davis, and Madden (2011).
 

88 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

able. Yet, the Reading Partners model appears to work because there are structures in place to 

help ensure that the primary goal of the program — to provide consistent, individualized tutoring to students — is not compromised by tutor characteristics or inconsistency and because the 

structured curriculum allows tutors to begin without much preparation. Site coordinators provide ongoing support and guidance to tutors in lieu of formal training. Communication structures are in place to help ensure that student learning is not interrupted simply because a different individual is conducting the tutoring, and backup systems are in place so that if a tutor is 

absent the session can often still be held, or if a student is absent the session can be rescheduled. 

While Reading Partners increases students’ reading skills by about a month and a half to two 

months using its current structure, it is possible that incorporating some of the characteristics of 

other tutoring programs that research has shown to be effective could result in larger impacts. 

For example, increasing the amount of formal training provided to volunteer tutors or increasing 

the amount of communication and coordination between Reading Partners and school staff 

might translate into a stronger tutoring program for students. 

Study Limitations 

While this evaluation was a rigorous one, there are some limitations to the conclusions that can 

be drawn from the findings and some questions that remain about the quality and importance 

of particular components of the Reading Partners program. First of all, the implementation data 

that were collected during the study emphasized a structural understanding of fidelity and how 

closely Reading Partners sites adhered to the national model. Tutors were observed to gain a 

more thorough understanding of the structural components of a tutoring session. While the 

research team noticed differences in how tutors explained concepts and modeled reading practices, no attempt was made to evaluate the quality of particular tutoring approaches or styles. 

Therefore, the question remains as to whether higher-quality tutors and tutoring would result in 

larger impacts and what changes would need to be implemented in order to see higher-quality 

tutoring. Similarly, assessing the quality of the Reading Partners curriculum was beyond the 

scope of this project. Aside from knowing that the curriculum is highly structured and (according to Reading Partners) increasingly aligned with the Common Core State Standards, the 

evaluation did not assess whether the individual lessons are effective at teaching students specific literacy skills and concepts. The only conclusion to be drawn is that it is designed in a 

way that allows tutors with minimal experience or training to successfully complete progressive sessions. 

Because data on program activities were collected directly from Reading Partners’ 

management information system, the study team was limited to the types of data that Reading 

Partners had already collected. One key question of interest is the role of tutor consistency in 

improving reading achievement, but the exploratory analyses conducted in this evaluation were 

89 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

only able to use a school-level snapshot of tutor consistency over a two-week period. More 

detailed data on the number of different tutors seen over the course of the year or information on 

the frequency of tutor absences is key to exploring the relationship between tutor consistency 

and program impacts. In particular, given the study team’s hypothesis that the one-on-one tutoring is the driving force behind the impacts found in this study, understanding whether simple 

one-on-one attention is sufficient or whether that attention needs to consistently come from one 

(or two) specific individuals would allow Reading Partners staff and other, similar practitioners 

to better serve students. 

It is also possible that the evaluation affected the implementation of Reading Partners. 

First, Reading Partners usually enrolls students on an ongoing, rolling basis as they are referred 

by teachers or other school staff, who prioritize the students who they think most need Reading 

Partners services. As a result of the study, however, schools identified and referred the whole 

pool of eligible students at the beginning of the school year; these students were then randomly 

assigned into the program or control group, and program group students were matched with 

tutors as quickly as possible. Absent random assignment, Reading Partners usually has a very 

targeted approach to identifying students for its program, a process that was disrupted by the 

needs of the study’s randomization and enrollment processes. At least one Reading Partners 

staff member indicated that some of the program group students were somewhat higherachieving students than usually participate in Reading Partners at the study schools, likely a 

result of this change in procedure. If the observations of Reading Partners staff were true, given 

that exploratory analyses indicated the lowest-achieving students experienced the largest impacts, it is possible that impacts on the typical Reading Partners student would be larger than 

those found for the overall sample. 

Another way in which the evaluation could have affected these findings is by changing 

the equilibrium of services being provided to struggling readers in the study schools. In the absence of Reading Partners, which brings additional resources to the school in the form of volunteer tutors, supplemental services would have had a different distribution across the student 

population. That is, teachers in the study may have been allocating more supplemental services 

(not including Reading Partners) to the control group students than they would have done normally. Small-group intervention services or other one-on-one tutoring programs represent the 

most common types of supplemental service provided to students in the control group. These 

services are typically costly with regard to staff time and place a large burden on school 

resources. (The cost study indicates that, on average, they cost schools $3,200 per student.) On 

the other hand, the cost of Reading Partners to schools is relatively low ($710 per student), 

meaning that more students can be served for the same amount of money. Thus, it is possible 

that absent Reading Partners, the “as is” condition at study schools would not have been able to 

serve as many students, thus decreasing the overall number of students receiving supplemental 

instruction at those schools. 

90 
 

 

 

 

 

 

 

 

A final limitation of this study is that student results and achievement are only measured at the end of one year of services. Reading skills are important for students of all ages and 

grade levels, so programs with long-lasting effects are poised to be more useful to schools that 

are trying to improve reading skills among their students. Follow-up studies on these students in 

subsequent grades would indicate whether Reading Partners is one such program. 

Recommendations for Future Research 

The discussion above suggests several areas for future research and consideration by the Reading Partners program. 

First, more information regarding tutor consistency and absences would help the program providers evaluate whether or not the lack of consistency reported by site coordinators is 

impeding student progress. To the extent that it is problematic, the Reading Partners program 

could consider putting structures in place to help mitigate this problem. 

Second, more information about the quality of the tutor-student relationship would help 

Reading Partners staff identify program strengths and weaknesses and might surface specific 

areas in which more training is warranted. Relatedly, research on factors that increase attendance at tutor training might help Reading Partners devise more creative ways to deliver training 

to their community volunteers. Gathering more information about tutor background and its relationship to student outcomes could also help Reading Partners target its recruitment activities 

more effectively. 

In addition, more information about the quality and effectiveness of the curricular materials themselves is warranted, since the curriculum may be related to observed outcomes. Finally, future studies should examine whether or not impacts are sustained beyond a single year. 

Conclusions 

The Reading Partners program successfully improved students’ reading comprehension, sight 

word efficiency, and fluency by about 0.10 standard deviations, or one and a half to two months 

of growth, over the course of the school year. While the program is relatively resourceintensive, a majority of the resources are in-kind contributions, and the portion of the resources 

financed by the schools is relatively low — only 20 percent of the total cost of the program. 

Moreover, among sites that were included in the cost study, the cost of Reading Partners borne 

by the schools was substantially less per student than the cost of other supplemental reading 

services offered to students. Thus, this study provides additional evidence that volunteer programs can work and that one-on-one tutoring is effective in improving academic outcomes. The 

results also suggest that effective volunteer tutoring programs may be a cost-effective option for 

91 
 

 

underresourced schools, because they bring additional resources to the school through community volunteers. Reading Partners manages to be effective even in the absence of oft-cited key 

components to successful tutoring, including, in particular, extensive tutor training and tutor 

consistency. Further research is required to understand whether improving these components of 

the model would affect the magnitude of the impacts and whether the impacts of Reading Partners are sustained for more than one year. 

92 
Appendix A 

Implementation Study Methods 

 

 

 

 

 

 

 

 

 

The qualitative data analysis coding scheme that was developed for the evaluation of Reading 

Partners was designed to align with the overall framework of the implementation study. The 

fidelity index is described in detail in Table A.1. A team-based approach was used to code all 

qualitative data (interview transcripts, notes, and observation write-ups). Each team member 

was assigned a set of codes to apply across all the qualitative data. The codes covered the 

following broad topics: instruction and delivery, Reading Partners staff roles and responsibilities, staff and tutor training, data-driven instruction, dedicated school space, program eligibility, 

treatment contrast and crossover, implementation context, volunteer tutor background and 

recruitment, Reading Partner staff background, school/district environment and characteristics, 

impact of the study on the program, Reading Partners history, and other contextual factors. 

After the systematic coding process, all team members developed analysis memos for 

each code they were assigned, which provided a summary of emerging themes, a school-byschool analysis of data that would be applied to the fidelity index (where applicable), and 

overall reflections and findings gleaned from the systematic review of the data. All qualitative 

data were analyzed using web-based, qualitative data analysis software. All analysis memos 

were discussed by several members of the study team and were revised based on feedback and 

discussion. The specific data used for the fidelity index were reviewed by at least two team 

members and the study’s principal investigator. 

95 
Reading Partners Evaluation
 

Appendix Table A.1
 

Fidelity Index Codebook
 

 

 

 

96 

Possible 

Components and Indicators Scoresa Criteria Data Sources 

Regular, one-on-one tutoring 

administered by volunteers 

Average number of tutoring sessions 

each week 

0,1,2 > 1.7 sessions/week = 2 

1.5-1.7 sessions/week = 1 

< 1.5 sessions/week = 0 

MIS data 

One-on-one: students are seeing tutors 

on an individual basis 

0,1 SC indicated that all tutoring was one-on-one = 1 

Otherwise = 0 

SC interview 

Dedicated school space and 

use of materials 

Required materials that were present in 

reading center during observation 

Dedicated vs. shared school space 

0,1 

0,2 

All materials = 1 

Missing any = 0 

“Space that is enclosed and unshared during RP time” = 2 

Otherwise = 0 

Materials checklist 

Site observations 

Data-driven instruction 

Whether or not Rigby is administered 

three times, at beginning, middle, and 

end of school year 

0,1 > 90% of eligible students took Rigby on 3 occasions = 1 

< 90% of eligible students took Rigby on 3 occasions = 0 

MIS data 

Whether or not Rigby is used to inform, 

customize, and update student’s 

Individualized Reading Plan (IRP) 

(including student’s progress in the 

curriculum) 

0,2 Reports that midyear Rigby was used to either create action plan or 

advance students, with the assumption that IRP was updated = 2 

No mention of how midyear Rigby was used, or of updating IRP = 0 

SC, PM interviews 

(continued) 
Appendix Table A.1 (continued) 

 

 

97 

Possible 

Components and Indicators Scoresa Criteria Data Sources 

Data-driven instruction (continued) 

Whether or not student work is 

documented after each tutoring session 

0,0.5,1 All “yes” = 1 

1 “no” = 0.5 

More than 1 “no” = 0 

Folder review 

Whether or not student data were 

shared with school principal 

0,1 PM indicated sharing data with PR = 1 

PM did not indicate sharing data with PR = 0 

PM interview 

Rigorous and ongoing training 

Reading Partners staff participated in 

initial training 

0,1 PM and SC reported participating in initial training = 1 

Either or both PM, SC reported no initial training = 0 

PM, SC interviews 

Do tutors receive in-service training? 0,1,2 Ratio of reported trainings to tutors: Training > tutors = 2; 

Training = Tutors = 1; 

Training < Tutors = 0 

Tutor interview 

Reading Partners SCs participated in 

ongoing, biweekly; or monthly training 

0,1 SC reported participating in ongoing training = 1 

SC didn’t participate = 0 

SC interview 

Tutors participating in shadow session 0,1 All tutors at the site reported participating in shadow session = 1 

Any tutor indicated no shadow session = 0 

Tutor interview 

Instructional supervision and support 

Presence of PM who oversees a specific 

area of schools and SCs 

0,1 Present PM = 1 

Missing PM = 0 

PM interview 

Presence of full-time SC who oversees 

operations at each school 

0,1 Present SC = 1 

Missing SC = 0 

SC interview 

(continued) 
Appendix Table A.1 (continued) 

 

 

 

 

 

 

 

98 

Possible 

Components and Indicators Scoresa Criteria Data Sources 

Instructional supervision 

and support (continued) 

Qualitative indicators of SC serving 

liaison between program and partner 

schools 

as 0,0.5,1 SC lists this as key part of job = 1 

SC talks regularly with tutors, but does n

key aspect of job = 0.5 

SC does not list this as key part of job an

infrequent contact with tutors = 0 

ot describe as 

d has 

SC interview 

Qualitative indicators of SC providing 

coaching and support to tutors and 

students 

0,1,2 SC discusses key roles as providing supp

(as opposed to monitoring) tutors = 2 

SC talks about training and monitoring tu

or SC says it’s hard to coach because doi

mentions coaching/supporting tutors but 

responsibilities = 1 

No mention of coaching or monitoring = 

ort/coaching 

tors, but not really coaching, 

ng a lot of tutoring, or 

at the end of list of 

0 

SC interview 

Qualitative indicators of PM providing 

coaching and support to SC 

0,1,2 PM lists this as key part of job = 2 

PM describes monitoring SCs but not su

them in other ways = 1 

No mention of coaching or supporting = 0 

pporting 

PM interview 

NOTES: MIS is management information system. SC is site coordinator. PM is program manager. PR is principal. RP is Reading Partners. a

Indicators with a maximum possible score of 2 are those indicators deemed most important for a faithful implementation of Reading Partners. 
Appendix B 

Impact Study Methods and Teacher Survey 

 

 

 

 

 

 

 

 

 

 

Obtaining, Cleaning, and Merging Administrative Data Sets 

All students had parental consent to participate in the study and for their records data to be 

collected. Demographic data, student attendance data, and state test score data were obtained 

directly from the participating districts for individual students. Data on student participation 

rates and tutor-student matches were obtained directly from Reading Partners. Raw data with 

student identifiers were kept on secure servers, and only key project staff had access to these 

data. In order to protect respondent confidentiality, student names and ID numbers were only 

stored together in one crosswalk file, which contained some or all of the following variables for 

each student: the student’s name, a district-assigned ID number, a Reading Partners-assigned ID 

number, an internal study-assigned ID number, and an encrypted unidentifiable ID number used 

for data processing and analysis. All data were cleaned in a multistep process that included 

(1) removing any student identifiers and replacing them with the encrypted ID, (2) identifying 

and resolving inconsistencies in each data source, (3) merging data across cleaned data sets, and 

(4) creating analytic variables to use in analyses. All data-cleaning programs were reviewed by 

at least two analysts. 

Standardization of Outcome Variables 

In order to compare students of different ages, grades, and states in the pooled impact analysis, 

steps were taken to standardize outcome variables. Documentation provided by the SAT-10, 

TOWRE-2, and AIMSweb include various types of conversion tables from raw scores, as 

shown in Appendix Table B.1. 

For the SAT-10 and TOWRE-2, the provided conversions were sufficient for analysis. 

However, the AIMSweb percentile conversions do not take into account the differing levels of 

difficulty across the different forms of the test. To standardize this outcome, a z-score was 

calculated for each student’s baseline and outcome scores using the baseline mean and standard 

deviation of scores for that grade-level and test form. 

District-provided state testing outcomes required additional standardization because 

scaled score ranges and proficiency level categories vary by state. Student proficiency scores 

were recoded into an indicator variable, representing whether or not students met the proficiency criteria established by their state. Scaled scores were converted to z-scores using a similar 

approach to that used with the AIMSweb scores. The 2013 control group mean and standard 

deviation for the student’s state and grade level were used for this standardization. 

101 
Reading Partners Evaluation
 

Appendix Table B.1
 

Availability of Standardized Scores for Primary Outcomes
 

 

 

 

 

 

Assessment 

SAT-10 

Scaled 

Score 

Available 

Percentile 

Score 

Available 

Grade 

Equivalence 

Available 

Age 

Equivalence 

Available 

TOWRE-2 Available Available Available Available 

AIMSweb Available 

Imputation of Covariates and Baseline Reading Skills 

Consistent with the analysis plan, in the case of covariate missing values, imputation was 

conducted using the mean of the school and grade level to which the student belonged. The 

overall rate of missing data was quite small and thus multiple imputations were not used. If 

more than 5 percent of observations are missing data for a given covariate, the model includes a 

dummy to indicate whether a student’s data were imputed for this covariate. No outcome 

variables were imputed. 

Analytic Model 

For each outcome, the basic model used in the analysis to estimate impacts was as follows: 

Yijk = γ 0Y−1,ijk + β 0Tijk +∑∑γ 1kjBijk +∑γ 2sXsijk + ukTijk + εijk (1) 

k j S 

where: 

Tijk =	 1 if student i from grade j in school k is assigned to Reading Partners and 0 

otherwise 

Y =	 the pretest score for student i from grade j in school k before random −1,ijk assignment 

Bijk =	 block dummy, 1 if student i from grade j is in a particular random assignment block, defined by grade block (2/3 or 4/5) and school k, and 0 

otherwise 

X = the Sth other student-level covariates for student i from grade j in school k; sijk 

these variables include the time lapse between baseline and follow-up 

testing as well as the demographic characteristics of age, gender, race, 

poverty status, special education status, and English language learner status 

102 
 

 

 

 

 

 

 

 

 

 

 

ukTijk =	 a random error term for the treatment group in school k, assumed to be 

independently and identically distributed; this term is included in the model 

to allow for variation in the impact of the program across schools 

εijk =	 a student-level random error for student i from grade j in school k, assumed 

to be independently and identically distributed 

Correction for Multiple Hypotheses 

Multiple hypothesis testing was accounted for in the three primary outcomes of interest (reading 

comprehension, fluency, and sight word efficiency) using a procedure developed by Benjamini 

and Hochberg.1 The adjustment is as follows: 

1.	 Conduct N separate t-tests, each at the common significance level α. 

2.	 Order the p-values of the N tests from smallest to largest, where p(1) <= p(2) <=...<= 

p(N) are the ordered p-values. 

3.	 Define k as the maximum j for which p(j) <= j/N*α. 

4.	 Reject all null hypotheses Ho(j) j = 1, 2,...,k. If no such k exists, then no hypotheses are 

rejected. 

1

Benjamini and Hochberg (1995). 

103 
Appendix Figure B.1
 

Example of Teacher Survey
 

 

 

 

 

 

 

 

 

 

 

 

 

 

1 

Reading Partners Teacher Survey 

Student Information Form [place student ID sticker here] 

1. Has this student been assigned to your classroom for the past four weeks? 

YES (continue to the 

next question) 2 NO If this student has not been assigned to your classroom for the past 

four weeks please stop here and go to the next student. 

For questions 2 and 3, please think about the amount of reading instruction that this student received in your 

classroom. By reading instruction we mean decoding, comprehension, vocabulary, and fluency. Do not include 

instruction that took place outside of your classroom (i.e., after school or via pull out). 

4.
 School personnel have indicated that the following supplemental reading programs or activities are available outside
 

of your classroom (i.e. after school or via pull out) at your school. Please mark all programs this student participated
 

in this academic year and indicate the amount of time that they participated outside of your classroom.
 

Total Number of 

Weeks Attended 

Average Minutes 

Per Week 

Alternate Program #1 

Reading Partners 

Alternate Program #2 

Alternate Program #3 

Another reading program or activity (please specify) 

weeks min/wk 

weeks min/wk 

weeks min/wk 

weeks min/wk 

weeks min/wk 

5. Please report on this student’s behavior over the 

past month. 

This student completed homework assignments 1 2 3 4 

This student completed in-class assignments 

This student was attentive in class 1 2 3 4 

This student was disruptive in class 1 2 3 4 

6. Compared to other students in your school, please
 

rate this child’s current academic performance in:
 

Far Below 

Average Average 

Reading 

Far Above 

Average 

1 

Above 

Average 

2 3 

Below 

Average 

4 5 

Math
 

Overall 

1 2 3 4 5

1 2 3 4
 5 

Thank you for your help! Please place this form in the envelope provided and store it securely in your classroom. It will be collected by the research team. 

Always Usually Sometimes Never 

1 2 3 4 

2. What was the total amount of time (in minutes) that 

this student received instruction in reading each 

week? 

Print the number of minutes using all three boxes. For 

example, write 015 if you taught for 15 minutes. 

Total minutes per week spent on 

reading instruction in your classroom. 

3. Of that time, how much time in minutes did you or an 

aide provide one-on-one help to this student in 

reading? 

Print the number of minutes using all three boxes. For 

example, write 015 if you taught for 15 minutes. 

Total minutes per week the student 

receives one-on-one reading help in your 

classroom. 

104
Appendix C 

Tutor Background Characteristics and
 

Additional Impact Findings
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Tutor Background Characteristics, by Site 

Appendix Table C.1 shows the background characteristics of the tutors at each of the 19 schools 

included in the study. 

Impact of Reading Partners on State Assessment Scores 

An exploratory analysis also examined the impact of Reading Partners on state assessment 

scores. This analysis has a number of limitations. First, the schools in the evaluation were 

located in three different states, each of which gave different assessments, measuring somewhat 

different skills. To combine assessments across states, scaled scores were converted to z-scores 

using the 2012-2013 control group mean and standard deviation for the students’ states and 

grade levels. However, even with these standardization efforts, assessments from different states 

may not actually be comparable with one another, limiting the conclusions that can be drawn 

from these analyses. 

Second, student proficiency scores were recoded into an indicator variable, representing whether or not students met the proficiency criteria established by their states. However, 

each state had its own criteria for determining proficiency; thus, this metric may not be 

measuring the same variables across states. Furthermore, because students who were referred 

to Reading Partners were half a year to two and a half years below grade level, it is not likely 

that a large percentage of students would obtain proficiency after 28 weeks of participating in 

the intervention. 

The findings from these analyses are shown in Appendix Table C.2. The program had 

no statistically significant impacts on the standardized state assessment scores in English 

language arts (ELA) or math, although the impact on ELA was positive, with an effect size of 

0.06. There was also no impact on the percentage of students reaching proficiency in either 

ELA or math. 

Additional Analyses Exploring Variation in Outcomes 

One way to explore the relative contribution of tutor consistency to student outcomes is to 

examine the relationship between a student’s reading growth and the number of tutors assigned 

to that student over the course of the year. It is important to note that the number of assigned 

tutors per student is not a true measure of tutor consistency because it does not take into account 

whether or not the assigned tutor was actually present for a student’s Reading Partners sessions. 

Additionally, any associations are correlational only, and are not evidence of causality, since the 

107 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Appendix Table C.1
 

Tutor Background Characteristics, by Site
 

College or 

High School Graduate Working Retired 

Sitea Students (%) Students (%) Adults (%) Adults (%) Otherb

 (%) Missing (%) 

Site A 13.21 30.19 39.62 7.55 9.43 -

Site B 7.04 66.20 11.27 2.82 11.27 1.41 

Site C 20.59 50.00 17.65 - 8.82 2.94 

Site D 23.33 35.00 13.33 10.00 15.00 3.33 

Site E 20.41 32.65 20.41 10.20 10.20 6.12 

Site F 31.25 34.38 21.88 6.25 - 6.25 

Site G 7.14 42.86 21.43 7.14 14.29 7.14 

Site H 14.29 35.71 21.43 7.14 14.29 7.14 

Site I - 83.33 - - 8.33 8.33 

Site J 19.74 26.32 28.95 2.63 13.16 9.21 

Site K 28.13 20.31 15.63 15.63 10.94 9.38 

Site L - 30.00 22.50 10.00 27.50 10.00 

Site M 16.18 47.06 7.35 8.82 10.29 10.29 

Site N 5.56 70.37 1.85 3.70 3.70 14.81 

Site O 22.22 23.61 22.22 11.11 5.56 15.28 

Site P 12.50 10.00 22.50 15.00 22.50 17.50 

Site Q 28.00 10.00 10.00 16.00 10.00 26.00 

Site R 20.55 26.03 16.44 6.85 2.74 27.40 

Site S 15.38 15.38 30.77 7.69 - 30.77 

All sites 17.20 35.63 17.65 8.10 10.32 11.10 

SOURCE: MDRC calculations from Reading Partners management information system data. 

NOTES: Rounding may cause discrepancies in sums. aTo prevent identification of individual schools or programs, site letter identifiers are not consistent 

across analyses. 

bTutors characterized as "Other" include, but are not limited to, caregivers, nonworking individuals, 

and those who describe themselves as "transitioning." 

108 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Appendix Table C.2
 

Impact of Reading Partners on State Achievement Testing Outcomes
 

Outcome 

Sample Program Control 

Size Group Group Difference 

Standard 

Error 

Impact 

Effect Size P-Value 

State reading achievement 

scaled scorea 1,079 0.08 0.02 0.06 0.05 0.06 0.234 

State math achievement 

scaled scorea 1,078 0.01 0.00 0.01 0.06 0.01 0.907 

Likelihood of achieving 

proficiency on state reading 

achievement test (%) 1,079 15.38 14.54 0.85 2.01 0.02 0.674 

Likelihood of achieving 

proficiency on state math 

achievement test (%) 1,078 28.99 31.75 -2.76 2.94 -0.06 0.348 

SOURCES: MDRC calculations using study team-administered baseline SAT-10 scores; districtprovided demographic and achievement data; Reading Partners management information system data.
 

NOTES: Calculations for this table used data for all students in the respondent sample with state testing 

outcomes. 

The model allows estimates to vary randomly across schools and controls for the random assignment 

block, the student-level pretest measure, the time lapse between baseline and follow-up testing, and 

student-level demographic covariates. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels 

are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

aTo account for differences across states, scaled scores have been normed to have a mean of 0. A full 

description of the the standardization of state testing variables can be found in Appendix B. 

number of tutors assigned was not determined randomly and there is no control group with 

which to compare the findings (that is, x number of tutors assigned to a program group compared with x number of tutors assigned to a control group). Appendix Table C.3 presents 

findings from an analysis looking at the relationship between the number of tutors assigned and 

student reading proficiency growth. 

As discussed in Chapter 4, Appendix Table C.4 shows how impacts on the three 

measures of reading proficiency varied by the school-level consistency of the tutors, based on 

folder review data. No consistent patterns between tutor consistency and outcomes were found 

in this analysis. 

109 
 

 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Appendix Table C.3
 

Students' Reading Growth Based on Their Number of Assigned Tutors
 

Comprehension Sight Word Efficiency Fluency 

Number of Tutors Sample Mean Standard Mean Standard Mean Standard 

Assigned to Student Size Growth Deviation Growth Deviation Growth Deviation 

1 143 18.96 30.87 0.84 8.63 18.17 19.44 

2 161 16.57 32.83 2.60 8.22 21.82 21.17 

3 167 17.37 27.29 1.00 8.38 20.93 19.54 

4 82 20.47 30.43 3.06 8.92 22.62 20.72 

5 to 8 34 21.94 19.62 0.73 6.35 15.82 30.44 

SOURCES: MDRC calculations using study team-administered baseline and follow-up SAT-10, 

TOWRE-2, and AIMSweb scores; district-provided demographic data; Reading Partners 

management information system data. 

NOTES: Calculations for this table used data for all students in the respective subgroups of the
 

respondent sample. Sample sizes for individual outcomes may fall short of the reported sample 

sizes because of missing or unusable data.
 

Mean growth reflects the average increase in scores from pretest to posttest. Analyses used
 

scaled scores for the SAT-10 (comprehension) and TOWRE-2 (sight word efficiency) and raw
 

scores for the AIMSweb (fluency).
 

Finally, as discussed in Chapter 4, Appendix Table C.5 shows how growth in reading 

proficiency on the three outcome measures was related to the background characteristics of the 

student’s primary tutor. Again, no consistent patterns were found. 

Subgroup Analysis of Primary Impacts Based on Reading 

Partners Target Status 

While Reading Partners serves a wide variety of students, it identifies “target students” as those 

who are 6 to 30 months below grade level in reading (as measured by the Rigby PM Benchmark assessment), who are at least conversationally fluent in English, and who do not have an 

Individual Education Program (IEP). At each of its reading centers, Reading Partners aims for 

target students to make up at least 80 percent of all students served, and in the sample for this 

study, approximately 75 percent of respondents met those criteria. One exploratory question of 

110 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Appendix Table C.4
 

Program Impacts, by School Level of Tutor Consistency
 

Comprehension Sight Word Efficiency Fluency 

Impact Impact Impact 

School Level of Sample Effect Effect Effect 

Tutor Consistencya Size Size P-Value Size P-Value Size P-Value 

Low 385 0.03 0.736 0.12 * 0.080 0.14 ** 0.048 

Moderate 424 0.03 0.643 0.08 0.236 -0.03 0.692 

High 357 0.30 ** 0.017 0.14 ** 0.045 0.15 * 0.089 

SOURCES: MDRC calculations using study team-administered baseline and follow-up SAT-10, 

TOWRE-2, and AIMSweb scores; district-provided demographic data; Reading Partners 

management information system data; review of Reading Partners folders. 

NOTES: Calculations for this table used data for all students in the respondent sample. Sample sizes 

for individual outcomes may fall short of the reported sample sizes because of missing or unusable 

data. 

The impact analyses for student reading achievement were conducted using scaled scores for the 

SAT-10 (comprehension) and the TOWRE-2 (sight word efficiency) and sample-normed scores for 

the AIMSweb (fluency). The model allows estimates to vary randomly across schools and controls 

for the random assignment block, the student-level pretest measure, the time lapse between baseline 

and follow-up testing, and student-level demographic covariates. 

A two-tailed t-test was applied to differences between research groups. Statistical significance 

levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. Differences between 

subgroups are not statistically significant. a

Schools with low tutor consistency are those in the lowest tercile of average sessions per tutor 

from the review of Reading Partners folders; schools with moderate tutor consistency are those in the 

second tercile of average sessions per tutor; schools with high tutor consistency are those in the third 

tercile of average sessions per tutor. 

this evaluation is whether or not Reading Partners has a different impact on those students who 

are designated as target students. Appendix Table C.6 shows the results of this subgroup 

analysis: 

•	 For the SAT-10 (which measures reading comprehension) and the AIMSweb 

(which measures oral fluency), the impacts for both target and nontarget 

students are similar in magnitude to the main impacts, though most are not 

statistically significant. 

111 
 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Appendix Table C.5
 

Students' Reading Growth Based on Background Characteristics of
 

Their Reading Partners Tutors 

Background of 

Primary Tutora 

High school student 

College or 

graduate student 

Working adult 

Retired 

Otherb 

Missing 

Sample Mean Standard Mean Standard 

Size Growth Deviation Growth Deviation 

Comprehension Sight Word Efficiency 

70 16.36 29.72 3.37 6.81 

207 19.27 30.48 0.43 8.80 

105 19.81 30.18 2.07 8.64 

79 19.16 31.07 2.81 8.67 

75 15.17 26.67 1.60 8.63 

58 17.81 29.41 1.68 6.75 

Mean Standard 

Growth Deviation 

Fluency 

18.54 25.43 

19.12 19.51 

20.12 21.20 

24.49 22.62 

22.63 20.07 

18.83 19.86 

SOURCES: MDRC calculations using study team-administered baseline and follow-up SAT-10, 

TOWRE-2, and AIMSweb scores; district-provided demographic data; Reading Partners 

management information system data. 

NOTES: Calculations for this table used data for all students in the respective subgroups of the 

respondent sample. Sample sizes for individual outcomes may fall short of the reported sample sizes 

because of missing or unusable data. 

Mean growth reflects the average increase in scores from pretest to posttest. Analyses used scaled 

scores for the SAT-10 (comprehension) and TOWRE-2 (sight word efficiency) and raw scores for 

the AIMSweb (fluency). a

Among students' assigned tutors, primary tutors are those who had the longest assigned 

relationship with the student. 

bTutors characterized as "Other" include, but are not limited to, caregivers, nonworking 

individuals, and those who describe themselves as "transitioning." 

•	 For the TOWRE-2 (which measures sight word efficiency), the impact on 

target students is slightly smaller (though not statistically significant) than for 

the full sample, but the impacts for the nontarget students are slightly larger. 

•	 Differences between these subgroups are not statistically significant. 

112 
 

 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Appendix Table C.6
 

Subgroup Analysis of Primary Impacts
 

Based on Reading Partners Target Status 

Reading Partners 

Target Status 

Sample 

Size 

Impact 

Effect 

Size P-Value 

Comprehension 

Impact 

Effect 

Size P-Value 

Sight Word Efficiency 

Impact 

Effect 

Size P-Value 

Fluency 

Target student 853 0.09 0.102 0.06 0.136 0.11 ** 0.020 

Not a target student 313 0.12 0.285 0.26 *** 0.003 0.10 0.309 

SOURCES: MDRC calculations using study team-administered baseline and follow-up SAT-10, 

TOWRE-2, and AIMSweb scores; district-provided demographic data; Reading Partners 

management information system data. 

NOTES: Calculations for this table used data for all students in the respective subgroups of the 

respondent sample. Sample sizes for individual outcomes may fall short of the reported sample sizes 

because of missing or unusable data. 

The impact analyses for student reading achievement were conducted using scaled scores for the 

SAT-10 (comprehension) and the TOWRE-2 (sight word efficiency) and sample-normed scores for 

the AIMSweb (fluency). The model allows program impact estimates to vary randomly across 

schools and controls for the random assignment block, the student-level pretest measure, the time 

lapse between baseline and follow-up testing, and student-level demographic covariates. 

A two-tailed t-test is used for all statistical tests presented in this table. Statistical significance 

levels between the program and control groups are indicated as: *** = 1 percent; ** = 5 percent; * = 

10 percent. Differences between subgroups are not statistically significant. 

113 
 

 

 

 

620 Comprehension 70 120 Sight Word Efficiency Fluency 

610 Grades 4-5 Grades 4-5 Grades 4-5 110 65 

600 

100 

60 

590 

90 

Score 

Score 

Score 

580 55 

Grades 2-3 Grades 2-3

50 

70

560 

45 60 550 

80 Grades 2-3 570 

540 40 50 

Pretest Posttest Pretest Posttest Pretest Posttest 

Program Group Control Group 

114 

Reading Partners Evaluation
 

Appendix Figure C.1
 

Student Reading Growth
 

SOURCES: MDRC calculations using study team-administered baseline and follow-up SAT-10, TOWRE-2, and AIMSweb scores; districtprovided demographic data; Reading Partners management information system data. 

NOTES: Growth analyses were conducted using scaled scores for the SAT-10 (comprehension) and raw scores for the TOWRE-2 (sight word 

efficiency) and AIMSweb (fluency). Since the equivalence of baseline test scores of the two groups has been established, a common pre-test 

score is assumed for both. 
Appendix D 

Cost Study Methods 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The cost study conducted for this evaluation used the ingredients method to calculate costs. The 

ingredients method consists of three main steps. First, all ingredients that are used in implementing a program are identified, described qualitatively, and specified by quantities. All ingredients 

are specified based on the requirements for replication. To replicate the particular implementation that produced a specific result, personnel and the amount of time they devoted to providing 

services must be identified, as well as training requirements, facilities that were used to provide 

services or training, materials used, and any transportation or food or other services that were 

provided. All ingredients are included regardless of who paid for or contributed them.1 

Identification of Other Supplemental Reading Services 

At all six sites that were included in the cost study, schools had other supplemental services or 

programs (in addition to the Reading Partners program) to provide additional reading instruction 

beyond the regular classroom teaching. Services were provided to all students in need of 

additional support, irrespective of whether they had been included in the Reading Partners study 

or whether they were assigned to the study’s program group or the control group. These 

supplemental reading services varied from reading specialists to tutoring programs to homework help. Some of the supplemental reading services offered at these schools may not be 

comparable with Reading Partners in terms of providing policy alternatives, because they serve 

students of differing levels of need (for example, more than two and a half years below grade 

level) or they may not be optional and so do not lend themselves to cost-effectiveness comparisons. Thus, the costs of other supplemental reading services are included here to provide a 

context for the cost of Reading Partners and the school environment in which the program is 

implemented. 

Ingredients Data Collection 

The cost study collected data on all ingredients that were used in the Reading Partners program, 

including the time contributed by volunteers to provide tutoring to students and to commute to 

the school. Before conducting interviews with Reading Partners staff, the research team collected data from publicly available sources such as websites and published reports to prepare a list 

of potential ingredients at each school site. Data from the implementation study helped the 

research team identify the materials used at each site. Data from the teacher surveys were used 

to identify the other supplemental reading services provided at each school. Program managers 

from Reading Partners at each site were then contacted for interviews. The interviews were held 

by phone and focused mostly on the numbers and types of personnel used to implement the 

1

Levin and McEwan (2001). 

117 
 

 

 

 

 

 

 

 

 

 

 

 

 

program. The program managers were also asked to describe the other supplemental reading 

services and to provide contact information for an individual involved in providing the other 

services. Often, a reading coordinator, reading specialist, or the school principal was interviewed regarding the school’s other supplemental reading services. 

Each ingredient was described in terms of the quantity and qualities of the ingredient 

needed to determine the value or price. For example, questions were included about the experience, education, and responsibilities of the program managers and site coordinators and other 

Reading Partners staff who were involved in providing the program services at the school. 

Descriptions were requested regarding the program’s use of time from teachers and principals 

because the time invested by these school-based personnel may have been important to the 

success of the program. The roles, qualifications, and time devoted by the volunteers were 

included. Even though the tutors volunteer at no cost to the school, the volunteers are a necessary input for the program to work and their services have a value that is included in this 

analysis. In essence, the estimated cost of volunteer time is what they would need to be paid in 

the market if they were unavailable as volunteers. Descriptions of space used to provide the 

program and any materials used by the program were also collected. 

Pricing the Ingredients 

The second step in the ingredients method is to assign prices to each ingredient based on market 

prices (in this case, for 2012). National prices are often the best option to account for prices 

when the intervention is applied in different locations so that the results are not dependent upon 

local variation in prices among different locations. If market prices are not available for an 

ingredient, a “shadow” or underlying price is used from the most similar resource with a market 

price. For example, no market rental rate is available for classroom space because classrooms 

are not typically rented. Therefore, to approximate the annual rental value of the space, the 

construction cost plus the costs for furnishings and interiors are priced and amortized over the 

life span of the building. 

In the cost study, each program ingredient was matched with its national price or cost in 

2012 dollars. When necessary, prices that were available only for other years were adjusted to 

2012 dollars using the Consumer Price Index. Ingredients that are used for longer than one year 

(such as initial training of tutors, facilities, books, and computers) were accounted for by their 

annualized costs, amortized with an interest rate of 3 percent. Tutor time for the initial training 

and the shadow session was amortized by the site-level average tutor participation if longer than 

one year; the ongoing training that occurred on a regular basis was viewed as an annual cost. 

The estimated cost of the other supplemental reading services provided at each school 

followed the same pricing approach. Each ingredient for the supplemental services was matched 

118 
 

 

 

 

 

 

 

 

 

 

 

 

with a national price. In some cases, specific prices were obtained from the program developer, 

such as the price of the program license and the price of the training package. The total cost and 

the cost per student for each program or service identified were calculated at the six schools in 

the sample. Each site had between one and four other supplemental services included in the 

analysis, and in addition to the Reading Partners programs at each school, there are 12 supplemental reading programs in total. The costs were aggregated at the site level rather than reported 

individually in order to protect school confidentiality. 

Personnel 

The Reading Partners site coordinator and outreach coordinator positions were staffed 

by AmeriCorps members. These individuals were young, recent college graduates with little to 

no additional experience. To price these positions, the national average earnings amount for 18­

to 24-year-olds with a bachelor’s degree was obtained from the U.S. Census Bureau, Current 

Population Survey, 2012 Annual Social and Economic Supplement. Benefits were estimated at 

31 percent based on the Bureau of Labor Statistics. Because this price may be higher than what 

the individuals received from AmeriCorps for their work, a sensitivity test was included that 

used the annual salaries and compensation information from the Reading Partners hiring 

postings that were available online. The hiring posts were specific to particular locations. Thus, 

the salaries were adjusted to national prices using the index of Regional Price Parities (2006­

2010) and then the salaries were averaged for a national average salary rate.2 

Based on the national statistics reported by the Bureau of Labor Statistics, the Department of Labor, and the National Center for Education Statistics, the hourly pay rate of school 

staff was transformed by dividing their annual salary and benefits by the number of hours in an 

academic year (that is, 1,440 hours from 180 days per year and 8 hours per day). 

Volunteer tutor time was valued at the same rate as paraeducators or teacher’s aids. This 

price was selected because the duties of the paraeducator best matched those of the volunteer 

tutor. Because Reading Partners relies on volunteers to serve students, upper and lower bounds 

were estimated for the value of volunteer time. The lower bound was estimated using the 

federal minimum wage. This wage was selected because the program advertises that its success 

is not dependent on the qualifications of the tutor. If all tutors were high school students, this 

price would represent the opportunity cost of the time contributed to the program. However, the 

program relies on tutors from various backgrounds. 

The upper-bound estimate used the tutor background information provided in Chapter 2 

(Figure 2.3). The time of any volunteer who was not enrolled in high school, college, or 

2

Aten, Figueroa, and Martin (2012). 

119 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

graduate school was valued as the average hourly wage of an individual with a college degree 

using the average of median hourly wages for a person with a bachelor’s degree across all age 

groups expressed in 2012 U.S. dollars.3 The other tutors, who were students, were valued at the 

paraeducator wage (which is very similar to the hourly wage of a high school graduate). While 

this estimate captures some of the variation in tutor education levels, the findings from the 

impact analysis in Chapter 4 find no clear indication that the background of a student’s primary 

tutor has any effect on reading abilities. 

The cost of transportation was included using the federal reimbursement rate per mile 

for driving or the price for a round trip on public transportation.4 The cost for public transportation is assumed to be $5 per round trip. 

Facilities 

In terms of school facilities, the construction cost of a new school building was adjusted 

to include the site, site development, furnishings, and interiors. The price per square foot was 

amortized over 30 years with an interest rate of 3 percent. This annual cost per square foot was 

further divided by the number of hours in an academic year (that is, 1,440 hours, or 180 days 

with 8 hours per day) to get the cost of school facilities per square foot per hour. The national 

median price for construction costs of elementary schools was collected from the 19th School 

Construction Report,

5 and the adjustment index is based on national statistics provided by the 

National Clearinghouse of Educational Facilities (2012).6 

Materials and Equipment 

The prices for all materials and equipment (for example, desks, chairs, computers, 

printers) were collected from publicly available sources and expressed in 2012 dollars. The 

Consumer Price Index was used as needed to adjust the prices to 2012 dollars. To reduce the 

burden of interviews, materials that varied in quantity with the number of students served — 

such as pens, Post-it® notes, bookmarks, student boxes, student folders, and so forth — were 

priced at $50 per student. Books used in the Reading Partners reading center and distributed to 

students were priced at the national average price of a children’s book used in public schools 

in 2012.7 

3

U.S. Census Bureau (2012). 4

Data on the federal reimbursement rate per mile came from the Internal Revenue Service (2011).

5

Abramson (2014).

6

See www.efc.gwu.edu. 7

SLJ (2013).
 

120 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

As described in Chapter 2, an important component of Reading Partners is that data inform the services provided to students, which is known as being “data-driven” in practice. The 

site-level ingredients included the Rigby PM Benchmark test kit, which is used three times per 

year to assess students’ reading levels and progress. The price of the Rigby PM Benchmark test 

kit in 2014 was transferred to 2012 dollars. During the interviews, the program managers 

reported that the Rigby kit was used for more than three years. The cost study assumes that the 

kit would require replacement after five years. The cost of the kit was amortized for five years 

using a discount rate of 3 percent. 

Calculating Costs 

The third step in applying the ingredients method is to estimate the total cost of the program and 

the cost per student. These costs can then be distributed across various entities or sponsors 

according to how they were financed. If an analysis includes cost-effectiveness, the fourth step 

is to calculate the cost-effectiveness ratios and examine the results comparatively. The cost 

study focused on estimating the cost of Reading Partners and the cost of the other supplemental 

reading services from the perspective of the school sites. A comparative cost-effectiveness 

analysis is not included because the study did not focus on the cost-effectiveness of other 

reading programs. 

Categories of cost estimation in this analysis include the value of total resources for all 

students served, cost per session, cost per program group student, and the total cost for the 

program group for each site. 

Limitations of the Cost Analysis 

As described above, the costs are sensitive to the valuation of volunteer time. Some limitations 

to the analyses should be noted. The main analysis used the same price ($15.14 per hour) for 

tutor time based on the national average wage of a paraeducator. By using the same price for all 

tutors, it could be possible that the true cost to replicate each site is not accurately estimated 

because the variability in tutors’ qualities, qualifications, experience, reason for tutoring, and so 

forth is not reflected. These qualitative characteristics of the tutors may be important for 

replication. However, as stated above, exploratory analysis did not find that tutor background 

was related to the students’ reading growth. It may be valuable in future analyses to collect 

more data on the volunteer tutors. 

Some sites used Federal Work-Study students from local colleges, in addition to community volunteers, to provide tutoring. The time spent tutoring by Federal Work-Study students 

was not easily distinguished from the time contributed by volunteers. Thus, the distribution 

analysis treated all tutors as volunteers. If data were available on the number of sessions 

121 
 

 

 

 

 

 

 

 

 

 

 

 

 

provided by each type of tutor, the distribution of the financial burden of the program costs 

could reflect the portion borne by the U.S. Department of Education. Additionally, the portion 

of the costs borne by the volunteers would drop slightly. 

The upper-bound estimate of the volunteer time used the average wage of an individual 

with a bachelor’s degree for all nonstudents and the average paraeducator wage for all student 

tutors. This analysis could be improved by using the national average wages for high school 

graduates, college enrollees, college graduates, and graduates with advanced degrees. These 

prices were not used because the data were not detailed enough to identify precisely which 

degree each tutor had obtained. Rather than introducing additional assumptions into the analysis, the paraeducator wage was used for all student tutors. 

The cost study prices the time contributed by teachers and principals as the national average wages earned by all individuals in those positions. Given the minimal involvement of 

these individuals, the prices used were not differentiated to reflect differences in qualifications 

and experience of the teachers and principals at each school site. Additionally, the cost study 

sought to minimize data collection to reduce the burden of interviews. As a result, the cost study 

did not inquire about the qualifications and experience of each teacher and the principals in the 

six school sites in the sample. 

Another potential source of data for the cost study was a biannual celebration of students’ progress in each site. However, the cost study could not fully investigate these two 

celebrations to identify who attended them. The celebrations were not deemed central to the 

successful implementation of the program and, therefore, in order to minimize the burden of 

data collection, the time spent by volunteers and possibly parents to attend the events was not 

included. 

The cost study team was able to collect observational data, which were very informative, for one site in the sample. However, this site visit occurred after many of the site interviews had been completed. Thus, some ingredients that may have been included in the implementation were not included in the cost study. For example, one site visit revealed that the 

reading center had a corner devoted to a book drive, from which students could take books 

home to build their own libraries. When the student returned a questionnaire about the book, the 

student received a star on a chart and another book. The books were donated to the program 

from community members and organizations. The cost study was not able to collect data on this 

aspect of the program. 

122 
 

 

 

 

 

 

 

 

 

Scale Test 

Many of the ingredients that were used to implement the Reading Partners program were 

needed in order to provide the program services, even if there were fewer students than could be 

served. Thus, the cost per student for each site is closely tied to the number of students served, 

which then causes variability in the cost per student. Based on the space required for the reading 

center and the program’s recommended furnishing of the space, the program is designed to 

serve a maximum of 80 students per reading center (eight students over five periods per day for 

two days each per week). The total number of students served at each site, including those who 

were not in the program group (for example, kindergarteners, first-graders, and students who 

enrolled at the school after study recruitment had been completed), varied from approximately 

40 to 80 students. To estimate the lower bound of the cost per student at each site, an analysis 

was included in which each site’s cost per student was calculated as though the site operated at 

full capacity serving 80 students. 

Additional assumptions that were required for the analysis include: 

•	 The workloads of the site coordinator, program manager, and regional executive director at each site do not change. 

•	 The maximum capacity of an outreach coordinator is assumed to be 600 

students. 

•	 The number of sessions per student received at each site is constant. 

•	 The volunteer body remains the same, but each tutor volunteers for additional sessions to meet the increased demand. 

•	 The working time of the school staff (teachers, principals, vice principals, 

and reading coordinator) remains constant. 

•	 The space required, which is designed to serve 80 students, remains constant. 

123 

Appendix E 

Additional Cost Findings 

 

 

 

 

 

 

 

 

 

 

Comparing Findings with Reading Partners Costs at Scale 

To investigate the lower bound of the cost per program student at each site, additional analyses 

were conducted as though the site operated at a full capacity of 80 students.1 As shown in 

Appendix Table E.1, under all of these assumptions the average cost per student drops considerably, from $3,610 to $2,390, with a range of $2,230 to $2,700 per student per site. These 

results equate to about a 34 percent reduction in costs, on average. As a school expands its scale 

to approach the maximum capacity, 

the fixed costs for the school, such as 

facilities and working time of Reading 

Partners staff and school staff, are 

shared by more students, leading to an 

increase in efficiency. However, it is 

unknown whether the expansion 

would affect the impact of the 

program. The total cost borne by the 

school is unchanged based on the 

assumptions; however, the portion of 

the total cost borne by the school 

drops dramatically as the increasing 

part of total cost is mainly borne by 

the volunteers and Reading Partners 

itself. 

This sensitivity test illustrates 

the variability in resource use among 

sites regardless of scale. While scale 

and resource use are likely related, 

this test shows how much of the sitelevel variability is driven by scale 

rather than by implementation differences in resource allocation, such 

as volunteer travel time, principal or 

school staff time, and the size of the 

facilities devoted to the reading 

center. It is possible, though, that an 

increase in the number of students 

Reading Partners Evaluation 

Appendix Table E.1 

Reading Partners Cost per Student: 

Main Study Findings and at Scale 

Cost per Student ($) 

Sitea 

Site A 

Main Findings 

3,450 

At Scaleb 

2,230 

Site B 3,420 2,550 

Site C 3,570 2,350 

Site D 

Site E 

5,190 

4,210 

2,240 

2,700 

Site F 2,740 2,270 

Pooled 3,610 2,390 

SOURCES: MDRC calculations from cost data. 

NOTES: Costs are measured in 2012 U.S. dollars and are 

rounded to the nearest 10 in the chapter tables and text. 

Averages have been weighted to account for the size 

of the program group at each site. 

All costs are those for the respondents in the program 

group and are adjusted for the larger dosage they 

received. a

To prevent identification of individual schools or 

programs, site letter identifiers are not consistent across 

analyses. 

b"At scale" assumes Reading Partners serves 80 

students per site. Additional assumptions are described in 

Appendix D. 

1

A full explanation of the assumptions used to estimate costs at scale appears in Appendix D. 

127 
 

 

 

 

 

 

 

 

 

 

 

 

would require an increase in school staff time to identify students and to collaborate with the 

program staff. 

Sensitivity of Reading Partners Cost to Changes in Valuations of 

Volunteer and AmeriCorps Members’ Time 

The research team also explored the sensitivity of the cost findings to differences in the value of 

personnel based on different likely scenarios.2 The results of this sensitivity test are provided in 

Appendix Table E.2. The average cost per program group student, as reported in Chapter 5, is 

$3,610 (with a range of $2,740 to $5,190). When volunteer time is valued at the rate of the 

federal minimum wage, a lower-bound estimate compared with the main analysis, the cost per 

student drops by about $600. The upper bound used a higher rate for volunteer time based on 

the assumption that all volunteers who were not students had bachelor’s degrees. This high 

estimate is about $300 higher than the main findings. 

An additional sensitivity test was included to test for uncertainty around the price used 

to value the AmeriCorps members who serve as Reading Partners site coordinators and outreach coordinators. The main analysis used the average earnings of recent college graduates as 

their annual salary. However, the actual salary and tuition benefits provided may have been 

lower than the national average wages. Thus, the sensitivity test used wage and tuition information that was available from the Reading Partners website to obtain an average salary. When 

this price is used, the cost per student drops by about $240, from $3,610 to $3,370. 

The main analyses use prices that reflect the broader marketplace and are likely closer 

to replication costs than the lower AmeriCorps member wage or minimum wage for the 

volunteers. These sensitivity analyses provide an estimate of how the range of the costs might 

vary. Because these tests focused on the uncertainty of the value of volunteers’ and AmeriCorps 

members’ time, the costs borne by the school were not changed. 

2

A full description of the sensitivity check appears in Appendix D. 

128 
 

 

 

 

 

 

 

 

 

 

 

Reading Partners Evaluation
 

Appendix Table E.2
 

Sensitivity Tests of Staff Prices for AmeriCorps
 

Members and Volunteers 

Cost per Student ($) 

Sitea Main Findings AmeriCorpsb Volunteer Lowc Volunteer Highd 

Site A 3,450 3,210 2,900 3,740 

Site B 3,420 3,220 2,790 3,640 

Site C 3,570 3,350 2,950 3,670 

Site D 5,190 4,840 4,550 5,690 

Site E 4,210 3,960 3,580 4,710 

Site F 2,740 2,570 2,210 3,150 

Pooled 3,610 3,370 3,010 3,930 

SOURCES: MDRC calculations from cost data. 

NOTES: Costs are measured in 2012 U.S. dollars and are rounded to the nearest 10 in the 

chapter tables and text. 

Averages have been weighted to account for the size of the program group at each 

site. 

All costs are those for the respondents in the program group and are adjusted for the 

larger dosage they received. aTo prevent identification of individual schools or programs, site letter identifiers are 

not consistent across analyses. 

bThe AmeriCorps sensitivity test uses the AmeriCorps stipend as the price of the site 

coordinators and outreach coordinator positions. c

The "Volunteer Low" sensitivity uses minimum wage as the price of volunteer time. 

dThe "Volunteer High" sensitivity uses U.S. census data on average earnings of 

individuals with bachelor's degrees as the cost of nonstudent volunteer time, and the cost 

of a paraeducator as the cost of student time. 

129 

 

 

 

 

 

 

References
 

AARP Experience Corps. 2014. “Frequently Asked Questions.” Website: 

www.aarp.org/experience-corps. 

Abramson, Paul. 2014. “New School Buildings Led Increase in Spending.” School Planning 

and Management: 19th Annual School Construction Report (February): 17-25. Website: 

www.webspm.com. 

Aten, Bettina H., Eric B. Figueroa, and Troy M. Martin. 2012. “Regional Price Parities for 

States and Metropolitan Areas, 2006-2010.” Survey of Current Business 92, 8: 229-242. 

Bayer, Amanda, Jean Baldwin Grossman, and David L. DuBois. 2013. “School-Based 

Mentoring Programs: Using Volunteers to Improve the Academic Outcomes of 

Underserved Students.” New York: MDRC. 

Benjamini, Yoav, and Yosef Hochberg. 1995. “Controlling the False Discovery Rate: A New 

and Powerful Approach to Multiple Testing.” Journal of the Royal Statistical Society 57, 1 

(Series B): 289-300. 

Borman, Geoffrey D., Robert E. Slavin, Alan C. K. Cheung, Anne M. Chamberlain, Nancy A. 

Madden, and Bette Chambers. 2007. “Final Reading Outcomes of the National 

Randomized Field Trial of Success for All.” American Educational Research Journal 44, 

3: 701-731. 

Bowden, Alyshia Brooks. 2014. “Estimating the Cost-Effectiveness of a National Program That 

Impacts High School Graduation and Postsecondary Enrollment.” Unpublished paper. New 

York: Columbia University. 

Chall, Jeanne S. 1983. Stages of Reading Development. New York: McGraw-Hill. 

Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences. Second Edition. 

Hillsdale, NJ: Lawrence Erlbaum Associates. 

Corporation for National and Community Service. 2014. “Our Programs: AmeriCorps.” 

Website: www.nationalservice.gov. 

Dane, Andrew V., and Barry H Schneider. 1998. “Program Integrity in Primary and Early 

Secondary Prevention: Are Implementation Effects Out of Control?” Clinical Psychology 

Review 18, 1: 23-45. 

Dhaliwal, Iqbal, Esther Duflo, Rachel Glennerster, and Caitlin Tulloch. 2012. “Comparative 

Cost-Effectiveness Analysis to Inform Policy in Developing Countries: A General 

Framework with Applications for Education.” Cambridge, MA: Abdul Latif Jameel 

Poverty Action Lab (J-PAL), Massachusetts Institute of Technology. 

131 
 

 

 

 

DuBois, David L, Bruce E. Holloway, Jeffrey C. Valentine, and Harris Cooper. 2002. 

“Effectiveness of Mentoring Programs for Youth: A Meta-Analytic Review.” American 

Journal of Community Psychology 30, 2: 157-197. 

Durlak, Joseph A., and Emily P. DuPre. 2008. “Implementation Matters: A Review of Research 

on the Influence of Implementation on Program Outcomes and the Factors Affecting 

Implementation.” American Journal of Community Psychology 41, 3-4: 327-350. 

Elbaum, Batya, Sharon Vaughn, Marie Tejero Hughes, and Sally Watson Moody. 2000. “How 

Effective Are One-to-One Tutoring Programs in Reading for Elementary Students at Risk 

for Reading Failure? A Meta-Analysis of the Intervention Research.” Journal of 

Educational Psychology 92, 4: 605-619. 

Fuchs, Douglas, and Lynn S. Fuchs. 2006. “Introduction to Response to Intervention: What, 

Why, and How Valid is It?” Reading Research Quarterly 41, 1: 93-99. 

Grossman, Jean Baldwin, and Kathryn Furano. 1999. “Making the Most of Volunteers.” Law 

and Contemporary Problems 62, 4: 1999-218. 

Hager, Mark A., and Jeffrey L. Brudney. 2004. “Balancing Act: The Challenges and Benefits of 

Volunteers.” Washington, DC: The Urban Institute. 

Harris, Douglas N. 2008. “Toward Policy-Relevant Benchmarks for Interpreting Effect Sizes: 

Combining Effects with Costs.” Educational Evaluation and Policy Analysis 31, 1: 3-29. 

Heinrich, Carolyn J., Robert H. Meyer, and Greg Whitten. 2010. “Supplemental Education 

Services under No Child Left Behind: Who Signs Up, and What Do They Gain?” 

Educational Evaluation and Policy Analysis 32, 2: 273-398. 

Hollands, Fiona, A. Brooks Bowden, Clive Belfield, Henry M. Levin, Henan Cheng, Robert 

Shand, Yilin Pan, and Barbara Hanisch-Cerda. 2014. “Cost-Effectiveness Analysis in 

Practice: Interventions to Improve High School Completion.” Educational Evaluation and 

Policy Analysis 36, 3: 307-326. 

Hollands, Fiona M., Yilin Pan, Robert Shand, Henan Cheng, Henry M. Levin, Clive R. Belfield, 

Michael Kieffer, A. Brooks Bowden, and Barbara Hanisch-Cerda. 2013. “Improving Early 

Literacy: Cost-Effectiveness Analysis of Effective Reading Programs.” New York: Center 

for Benefit-Cost Studies of Education, Teachers College, Columbia University. 

Hummel-Rossi, Barbara and Jane Ashdown. 2002. “The State of Cost-Benefit and CostEffectiveness Analyses in Education.” Review of Educational Research 72, 1: 1-30. 

Internal Revenue Service. 2011. “IRS Announces 2012 Standard Mileage Rates, Most Rates 

Are the Same as in July.” IR-2011-116. Website: www.irs.gov. 

Jacob, Robin Tepper, Thomas J. Smith, Jacklyn A. Willard, and Rachel E. Rifkin. 2014. 

“Reading Partners: The Implementation and Effectiveness of a One-on-One Tutoring 

Program Delivered by Community Volunteers.” Policy Brief. New York: MDRC. 

132 
 

 

 

 

 

Lee, Yung Soo, Nancy Morrow-Howell, Melissa Jonson-Reid, and Stacey McCrary. 2012. “The 

Effect of the Experience Corps® Program on Student Reading Outcomes.” Education and 

Urban Society 44, 1: 97-118. 

Levin, Henry M. 1975. “Cost-Effectiveness Analysis in Evaluation Research.” Pages 89-122 in 

Marcia Guttentag and Elmer L. Struening (eds.), Handbook of Evaluation Research: 

Volume 2. Beverly Hills, CA: Sage Publications. 

Levin, Henry M., and Clive R. Belfield. 2013. “Guiding the Development and Use of CostEffectiveness Analysis in Education.” New York: Center for Benefit-Cost Studies of 

Education, Teachers College Columbia University. 

Levin, Henry M., and Patrick J. McEwan. 2001. Cost-Effectiveness Analysis: Methods and 

Applications. Second Edition. Thousand Oaks, CA: Sage Publications. 

Levin, Henry M., Doran Catlin, and Alex Elson. 2007. “Costs of Implementing Adolescent 

Literacy Programs.” Pages 61-91 in Donald D. Deshler, Annemarie Sullivan Palincsar, 

Gina Biancarosa, and Marnie Nair (eds.), Informed Choices for Struggling Adolescent 

Readers: A Research-Based Guide to Instructional Programs and Practices. Newark, NJ: 

International Reading Association. 

Markovitz, Carrie E., Marc W. Hernandez., Eric C. Hedberg, and Benjamin Silberglitt. 2014. 

“Impact Evaluation of the Minnesota Reading Corps K-3 Program.” Chicago: NORC at the 

University of Chicago. 

May, Henry, Abigail Gray, Jessica N. Gillespie, Philip Sirinides, Cecile Sam, Heather 

Goldsworthy, Michael Armijo, and Namrata Tognatta. 2013. Evaluation of the i3 Scale-up 

of Reading Recovery: Year One Report, 2011-12. Philadelphia: Consortium for Policy 

Research in Education. 

McEwan, Patrick J. 2002. “Are Cost-Effectiveness Methods Used Correctly?” Pages 37-53 in 

Henry M. Levin and Patrick J. McEwan (eds.), Cost-Effectiveness and Educational Policy: 

Yearbook of the American Education Finance Association, 2002. Larchmont, NY: Eye on 

Education. 

Minnesota Reading Corps. 2014. “Guide to Applying to Minnesota Reading Corps 2014-15.” 

Website: www.minesotareadingcorps.org. 

National Council of State Legislatures (NCSL). 2014. “Common Core State Standards.” 

Website: www.ncsl.org. 

NCS Pearson, Inc. 2004. Stanford 10 Technical Data Report. Bloomington, MN: NCS Pearson, 

Inc. Website: www.pearsonassessments.com. 

O’Donnell, Carol L. 2008. “Defining, Conceptualizing, and Measuring Fidelity of 

Implementation and Its Relationship to Outcomes in K-12 Curriculum Intervention 

Research.” Review of Educational Research 78, 1: 33-84. 

Pearson Education, Inc. 2012. AIMSweb Technical Manual. Bloomington, MN: NCS Pearson, 

Inc. Website: www.aimsweb.com. 

133 
 

 

 

Quint, Janet, Rekha Balu, Micah DeLaurentis, Shelley Rappaport, Thomas J. Smith, and Pei 

Zhu, with Emma Alterman, Herbert Collado, and Emily Pramik. 2013. The Success for All 

Model of School Reform: Early Findings from the Investing in Innovation (i3) Scale-Up. 

New York: MDRC. 

Quint, Janet, Rekha Balu, Micah DeLaurentis, Shelley Rappaport, Thomas J. Smith, Pei Zhu, 

with Emma Alterman and Emily Pramik. 2014. Interim Findings from the Investing in 

Innovation (i3) Scale-Up: The Success for All Model of School Reform. New York: 

MDRC. 

Reading Partners. 2013. “Our History.” Website: www.readingpartners.org. 

Rice, Jennifer King. 1997. “Cost Analysis in Education: Paradox and Possibility.” Educational 

Evaluation and Policy Analysis 19, 4: 309-317. 

Ritter, Gary W., Joshua H. Barnett, George S. Denny, and Ginger R. Albin. 2009. “The 

Effectiveness of Volunteer Tutoring Programs for Elementary and Middle School 

Students: A Meta-Analysis.” Review of Educational Research 79, 1: 3-38. 

Rose, Stephanie. 2012. Third Grade Reading Policies. Denver: Education Commission of the 

States. 

Ross, John A., Khalid Barkaoui, and Garth Scott. 2007. “Evaluations that Consider the Cost of 

Educational Programs: The Contribution of High Quality Studies.” American Journal of 

Evaluation 28, 4: 477-492. 

Simon, Jessica. 2011. “A Cost-Effectiveness Analysis of Early Literacy Interventions.” 

Unpublished paper. New York: Columbia University. 

Slavin, Robert E., Cynthia Lake, Susan Davis, and Nancy A. Madden. 2011. “Effective 

Programs for Struggling Readers: A Best-Evidence Synthesis.” Educational Research 

Review 6, 1: 1-26. 

SLJ. 2013. “SLJ’s Average Book Prices 2013.” School Library Journal (June 11). Website: 

www.slj.com. 

Torgesen, Joseph K., Richard K. Wagner, and Carol A. Rashotte. 2012. TOWRE-2 Examiner’s 

Manual. Austin, TX: PRO-ED, Inc. 

U.S. Census Bureau. 2012. Current Population Survey Annual Social and Economic 

Supplement. Washington, DC: U.S. Census Bureau, Social, Economics, and Housing 

Statistics Division: Poverty. 

Website: www.census.gov/hhes/www/poverty/publications/pubs-cps.html. 

U.S. Department of Education. 2012. “Description of Supplemental Educational Services.” 

Website: www.ed.gov/nclbl. 

U.S. Department of Education. 2014. “School Improvement Grants.” 

Website: www2.ed.gov/programs/sif/index.html. 

134 
U.S. Department of Education, National Center	 for Education Statistics. 2013. “National 

Assessment of Educational Progress Reading: Grade 4 National Results.” Website: 

www.nationsreportcard.gov. 

Wasik, Barbara A. 1998. “Using Volunteers as Reading Tutors: Guidelines for Successful 

Practices.” The Reading Teacher 51, 7: 562-570. 

135 

 

 

 

 

 

About MDRC 

MDRC is a nonprofit, nonpartisan social and education policy research organization 

dedicated to learning what works to improve the well-being of low-income people. 

Through its research and the active communication of its findings, MDRC seeks to 

enhance the effectiveness of social and education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is 

best known for mounting rigorous, large-scale, real-world tests of new and existing 

policies and programs. Its projects are a mix of demonstrations (field tests of 

promising new program approaches) and evaluations of ongoing government and 

community initiatives. MDRC’s staff bring an unusual combination of research and 

organizational experience to their work, providing expertise on the latest in qualitative 

and quantitative methods and on program design, development, implementation, and 

management. MDRC seeks to learn not just whether a program is effective but also 

how and why the program’s effects occur. In addition, it tries to place each project’s 

findings in the broader context of related research — in order to build knowledge 

about what works across the social and education policy fields. MDRC’s findings, 

lessons, and best practices are proactively shared with a broad audience in the policy 

and practitioner community as well as with the general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range 

of policy areas and target populations. Once known primarily for evaluations 

of state welfare-to-work programs, today MDRC is also studying public school 

reforms, employment programs for ex-offenders and people with disabilities, and 

programs to help low-income students succeed in college. MDRC’s projects are 

organized into five areas: 

• Promoting Family Well-Being and Children’s Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the 

United Kingdom, MDRC conducts its projects in partnership with national, state, and 

local governments, public school systems, community organizations, and numerous 

private philanthropies. 
New York 

16 East 34th Street
�

New York, NY 10016
�

Tel: 212 532 3200
�

California 

475 14th Street
�

Oakland, CA 94612
�

Tel: 510 663 6372
�

www.mdrc.org 

mdrc MOBILIZING VOLUNTEER TUTORS TO IMPROVE STUDENT LITERACY Implementation, Impacts, and Costs of the Reading Partners Program MARCH 2015