Anchor

Scaling the Reading Corps intervention: 

Insights from the experiences of United 

Ways of Iowa

June 15, 2020

Christopher Jones, Daniel Friend, Mary Anne Anderson, Karen Needels, and Scott 

Richman

Submitted to:

Corporation for National and Community Service 

Office of Research and Evaluation 

250 E Street SW

Washington, DC 20525

Project Officer: Anthony Nerino 

Contract Number: GS10F0050L/CNSHQ16F0049

Submitted by:

Mathematica 

1100 First Street, NE, 12th Floor 

Washington, DC 20002-4221 

Phone: 202-484-9220 

Project Director: Scott Richman

Reference Number: 50332
This page intentionally blank for double-sided copying.
iii

CONTENTS 

EXECUTIVE SUMMARY .............................................................................................................................vii

Research questions, site selection, and data collection methods........................................................vii

Overview of the Reading Corps intervention.........................................................................................ix

Key findings from this case study report ...............................................................................................ix

I. INTRODUCTION................................................................................................................................... 1

A. Overview of the SEBM process study ........................................................................................... 1

B. Overview of the Reading Corps intervention................................................................................. 4

II. HOW DID UWI DEFINE AND OPERATIONALIZE SCALING? ............................................................ 7

A. Replicating and expanding Reading Corps ................................................................................... 7

B. Scaling while implementing Reading Corps with fidelity ............................................................... 9

III. HOW DID UWI SCALE READING CORPS? ...................................................................................... 13

A. Organizational workforce............................................................................................................. 13

1. Approach to structuring and training the workforce............................................................. 13

2. Facilitators and challenges to structuring and training the workforce ................................. 15

B. Systems to monitor implementation and facilitate communication ............................................. 19

1. Monitoring implementation .................................................................................................. 19

2. Communication systems ..................................................................................................... 22

C. Funding and other resources to support scaling ......................................................................... 24

1. Funding for Reading Corps scaling ..................................................................................... 24

2. Other resources ................................................................................................................... 27

D. Use of data systems and evaluation during scaling .................................................................... 28

1. Data systems ....................................................................................................................... 28

2. Use of evaluation................................................................................................................. 30

IV. SUMMARY OF KEY FINDINGS ......................................................................................................... 33

REFERENCES............................................................................................................................................ 37

APPENDIX: PROCESS STUDY METHODOLOGY ................................................................................... 41
This page intentionally blank for double-sided copying.
Scaling insights: United Ways of Iowa’s experience Mathematica

v 

EXHIBITS 

1 Personnel roles in Reading Corps intervention ............................................................................... 5

A.1 Scaling readiness framework developed under the SEBM project................................................ 43

A.2 Characteristics of those interviewed .............................................................................................. 47
This page intentionally blank for double-sided copying.
Scaling insights: United Ways of Iowa’s experience Mathematica

vii

EXECUTIVE SUMMARY

Scaling interventions that have demonstrated effectiveness can increase the likelihood that an 

organization’s activities will improve participants’ lives (National Implementation Research 

Network 2016). At the same time, scaling can be challenging because it goes beyond 

implementation—the focus is not only on implementing an intervention but also on reproducing 

the same effects for a larger or different population, in a new or different location, or perhaps 

while modifying some of the intervention’s components. 

The Corporation for National and Community Service (CNCS) has invested significant 

resources in supporting implementation of interventions designed to improve lives and 

strengthen communities through its AmeriCorps and Social Innovation Fund (SIF) programs. 

CNCS also invests in evaluating the effectiveness of these interventions and supporting the 

scaling of those that have evidence of being effective to serve new communities or populations. 

Recognizing that an increasing number of CNCS-funded grants were being used to scale 

interventions, CNCS contracted with Mathematica in 2016 to conduct the Scaling EvidenceBased Models (SEBM) project to deepen the agency’s understanding of the interventions and 

its knowledge base on scaling them. The project was also funded to generate systematic 

analysis on how the grantees planned to scale and what their experiences have been when 

doing so.

Using information gathered through the SEBM 

project’s process study, this report presents a case 

study of United Ways of Iowa (UWI), a CNCS 

grantee implementing the Reading Corps intervention 

in Iowa. Reading Corps is a standardized literacy 

intervention that provides one-on-one tutoring to 

students enrolled in pre-kindergarten through third 

grade to help them achieve reading proficiency. This 

case study provides insights about how UWI and its 

partners are scaling Reading Corps, as well as the 

factors that appear to facilitate and hinder scaling.

The intervention and 

grantee

This report describes the scaling of 

Reading Corps, a literacy 

intervention designed to improve 

reading proficiency, by United 

Ways of Iowa and its partners. 

Research questions, site selection, and data collection methods

More generally, the SEBM project’s process study examined how three organizations that 

received CNCS grant funding and that were selected for in-depth investigation scaled their 

evidence-based interventions. We define evidence-based interventions as those that have been 

demonstrated, through rigorous evaluation studies, to improve participant outcomes. The 

process study focused on how grantees viewed scaling, the actions they took when they scaled, 

and what factors appeared to facilitate or challenge scaling. The process study aimed to 

address two overarching research questions: 
Scaling insights: United Ways of Iowa’s experience Mathematica

viii

1. How did selected CNCS grantees define and 

operationalize scaling? For this research 

question, we describe the type of scaling that 

each grantee selected for the process study 

planned to undertake. The three types of scaling 

considered under the SEBM project are briefly 

defined in the box to the right. (See the appendix 

for more information about these definitions.)

Types of scaling

Expansion extends the 

intervention to more people in the 

same target population in the same 

location.

Replication extends the 

intervention to the same target 

population in a new location.

Adaptation extends the 

intervention to a different target 

population in either the same or 

different location or modifies the 

intervention for the same 

population in either the same or 

different location.

2. How did selected CNCS grantees scale 

evidence-based interventions? To answer this 

research question, we describe how the grantees 

selected for the process study used 

organizational and implementation supports—

including the organizational workforce,1 systems 

to monitor implementation and facilitate 

communication, funding and other resources, 

and use of data systems and evaluation—to 

facilitate scaling. We drew from the 

implementation science literature (see box 

below) to identify supports that are typically needed. In documenting the extent to which 

grantees drew upon organizational and implementation supports, the process study also 

identified factors that appeared to facilitate and hinder scaling. 

To answer these research questions, CNCS, with input from Mathematica, selected three 

grantees that were implementing evidence-based interventions. The grantees and the 

interventions they were implementing also demonstrated a higher degree of scaling readiness

than did other CNCS grantees. This meant that the grantees and interventions met the 

conditions expected to lead to successful scaling—that is, scaling the intervention while 

maintaining or exceeding the beneficial impacts documented in evidence about its 

effectiveness. 

Mathematica collected and analyzed data from four 

sources: (1) a review of documents relevant to each 

intervention and its scaling and supplied by the 

grantees or their partners, (2) a two-day visit during 

October 2018 to each grantee and local partners 

involved in scaling interventions, (3) brief telephone 

calls with grantee personnel shortly before and after 

the visits, and (4) follow-up telephone interviews 

conducted with grantee personnel about 12 months 

after the visits (that is, in September 2019). 

Information from these sources was compiled to 

identify insights about scaling that are particular to each grantee. 

What is implementation 

science?

Implementation science is the 

scientific investigation of factors 

associated with effective 

implementation of an evidencebased intervention or practice 

(Franks and Schroeder 2013).

1 We use the terms personnel and workforce to refer to organization and partner personnel as well as AmeriCorps 

members who deliver intervention services.
Scaling insights: United Ways of Iowa’s experience Mathematica

ix

Overview of the Reading Corps intervention

Reading Corps is a standardized, evidence-based literacy intervention that provides one-on-one 

tutoring to students enrolled in pre-kindergarten through third grade to help them achieve 

reading proficiency.2 It targets students who are reading below grade level, but not low enough 

to qualify for more intensive assistance. Reading Corps features 20 minutes of one-on-one, inschool reading tutoring each school day. Tutoring consists of a set of 10 scripted activities that 

target phonemic awareness, phonics, reading fluency, and comprehension skills. The specific 

set of activities used with each student is individualized based on their needs and progress. 

Students also receive materials and short literacy activities to take home to engage in with their 

families. Tutors track student progress by administering short weekly assessments. Students 

whose scores reach and remain above a specified threshold of reading proficiency exit the 

intervention. Each tutor is an AmeriCorps member3 serving at the school, who is overseen by an 

internal coach (an educator at the school) and a master coach (hired or contracted by the 

grantee for his or her literacy and coaching expertise). To ensure fidelity—the extent to which 

implementation of an intervention matches the intervention as designed—internal coaches 

observe tutors twice per month. Master coaches meet with tutors and internal coaches monthly 

to examine each student’s progress.

Reading Corps was originally developed in Minnesota by the state AmeriCorps commission, 

ServeMinnesota, and was first implemented in that state in 2003. Since then, Reading Corps 

has expanded to 12 states and the District of Columbia, serving more than 35,000 students 

during the 2017–2018 school year. It is currently overseen by Reading and Math, Inc., which we 

refer to as the developer in this report. The developer implements the Reading Corps 

intervention directly in some locations (primarily Minnesota) and licenses other organizations to 

replicate it in other locations, with the requirement that the intervention be implemented with 

fidelity. As a CNCS grantee, UWI, a statewide association of local United Way chapters,

receives CNCS grants to scale Reading Corps in Iowa. This report considers the full time frame 

of UWI’s implementation of Reading Corps, which began in 2012, but focuses on the period 

since it received a 2015 CNCS grant to scale the intervention to a larger number of schools in 

the state.

Key findings from this case study report

The key findings in this report pertain to two areas. First, we discuss the types of scaling that 

UWI and its partners have pursued while implementing Reading Corps in Iowa. Second, we 

discuss how UWI and its partners draw on organizational and implementation supports—

including the organizational workforce, systems to monitor implementation and facilitate 

communication, funding and other resources, and use of data systems and evaluation—to scale 

Reading Corps. We also discuss the facilitators and challenges that UWI and its partners have 

experienced while scaling.

2 See the Reading Corps websites for more information: http://www.readingcorps.org/ and

https://minnesotareadingcorps.org/. 

3 AmeriCorps members are individuals participating in local service programs funded by CNCS who commit their 

time to addressing critical community needs through engaging in national service. 
Scaling insights: United Ways of Iowa’s experience Mathematica

x 

Approaches to scaling. UWI has both replicated and expanded Reading Corps by partnering 

with school districts to implement the intervention in schools across Iowa. UWI originally 

replicated Reading Corps by licensing it from the developer in 2012, based on interest from 

Iowa policymakers, the state commission of volunteer service, and the local United Way 

chapters that formed UWI as their statewide association. After a planning year, UWI 

implemented Reading Corps during the 2013–2014 and 2014–2015 school years in about eight 

or nine schools, with about 10 to 15 tutors. UWI then used its 2015–2016 CNCS grant to 

expand Reading Corps to more schools in existing districts and to further replicate it in new 

districts in Iowa. Since 2015–2016, the intervention in Iowa has had about 65 to 70 AmeriCorps 

members serving as tutors in about 60 to 65 schools in as many as 30 districts. The specific 

locations at which Reading Corps is expanded and to which it is replicated depend heavily on 

local factors, most prominently school districts’ interest in participating and whether funding is 

available to cover the proportion of costs the CNCS grant does not cover. These factors have 

limited the extent to which Reading Corps is scalable in Iowa.

Reading Corps is an evidence-based intervention, and Reading and Math, Inc. requires 

organizations that license it to implement it with fidelity to the intervention. This means UWI 

avoids adapting the intervention. Instead, it adopts any updates to the intervention made by the 

developer, which also provides extensive assistance and supports to Iowa and other sites. 

These requirements are important to ensure that interventions that have shown evidence of 

effectiveness can continue to have those effects in new settings. UWI and other Reading Corps 

personnel reported being committed to fidelity and do not view the requirements as onerous or 

limiting. Nevertheless, UWI implements Reading Corps differently in a few ways, including by 

having tutors conduct additional literacy activities in the summer, although for the SEBM project 

we do not consider these to be modifications of the intervention.

Organizational workforce. UWI has two full-time Reading Corps personnel—a program 

director and a program manager. At the time of the site visit in October 2018, UWI’s only other 

employees were a contract bookkeeper, the UWI executive director, and a small number of 

personnel implementing a second, unrelated grant program that had ended as of September 

2019. The vast majority of those delivering Reading Corps in Iowa are not UWI personnel.

AmeriCorps members serve as tutors; school employees serve as internal coaches; and most of 

the 12 master coaches at the time of the site visit were employees of one of Iowa’s area 

education agencies (AEAs).4 These personnel take on Reading Corps duties for the schools 

they already serve for the AEA. UWI contracts for the remaining master coaches.

Reading Corps’ layered personnel approach involves roles that are well-defined, focused on 

specific aspects of implementation, and relate to other roles through a structure of monitoring 

and support. However, before implementing Reading Corps, UWI was a one-person 

organization, and UWI personnel reported a key challenge was having enough personnel to 

meet the intensive workload involved in scaling and administering the intervention at its current 

size. Because of this challenge, partners have been essential providing much of the 

workforce—internal coaches and master coaches—needed to deliver the intervention. Still, 

these arrangements can lead to challenges; for example, some coaches struggle to find the 

4 AEAs are funded by the state and provide a variety of services to school districts, including special education 

services. Each AEA serves a geographic area in the state.
Scaling insights: United Ways of Iowa’s experience Mathematica

xi

time needed for Reading Corps due to their other duties as educators. Also, as Reading Corps 

has expanded, it has been more difficult to recruit people to serve as tutors, due to improving 

economic conditions, competing national service programs in Iowa, and fewer candidates in 

rural areas.

Training is prescribed by the Reading Corps developer as part of the intervention. UWI holds an 

initial four-day training for tutors and internal coaches each September and two one-day 

trainings later in the fall, which the master coaches conduct.5 The developer supports the 

master coaches and UWI Reading Corps personnel, including through train-the-trainer 

meetings. Respondents described the Reading Corps training as thorough and said it prepares 

personnel to carry out their roles, facilitating successful implementation.

Systems to monitor implementation and facilitate communication. UWI uses the layered

structure and extensive processes built into the Reading Corps intervention to help monitor 

implementation fidelity. These include monitoring of tutors by internal coaches, including twicemonthly observations; monitoring of tutors and internal coaches by master coaches, including 

monthly meetings to review students’ progress; monitoring of master coaches by a lead master 

coach and developer personnel; and monitoring of other aspects of implementation by the lead 

master coach and UWI personnel, including around tutor caseloads and amounts of tutoring 

each student receives. UWI personnel also monitor members to ensure they are following 

AmeriCorps policies and procedures. Personnel also communicate outside of monitoring 

processes; for example, tutors and coaches meet regularly, and UWI personnel communicate 

changes in policies or procedures through mass emails. The developer also organizes separate 

monthly calls involving lead master coaches, program directors, and master coaches across 

sites; sends mass email updates; and communicates individually with the Iowa lead master 

coach and UWI program director, although more frequently with the former. 

Implementation monitoring and communication systems allow personnel to identify and address 

issues as they arise, including tutors struggling to deliver literacy activities, internal coaches not 

overseeing tutors with the prescribed frequency, and students not receiving the intended 

amount of tutoring per week. This facilitates implementing Reading Corps with fidelity. Reading 

Corps has many different personnel roles, and they need to coordinate, ask and answer 

questions, and monitor and support one another within the structure of the intervention. This 

became especially important when the intervention expanded and fewer opportunities existed 

for more personal forms of communication between some types of personnel, especially 

between leadership and frontline personnel. Personnel have also used frequent informal 

communication to supplement formal processes, often by email but also by phone or text 

message. However, as UWI has scaled Reading Corps and communication has become more 

complex, respondents reported challenges involving communication processes between 

leadership and frontline personnel, and challenges involving communication from the developer 

to Reading Corps personnel in Iowa. 

Funding and other resources to support scaling. Because CNCS requires that its funds 

cannot cover the entire cost of an intervention, matching funds must pay part of the cost of each 

5 For 2019–2020, per a change from the developer, returning internal coaches were excused from in-person training 

if they passed an online, open-book test.
Scaling insights: United Ways of Iowa’s experience Mathematica

xii

AmeriCorps member placed in a school. UWI is a statewide association of local United Way 

chapters, and each local chapter has sole authority (within the national organization) to 

fundraise in its geographic area. Thus, at the start of implementation in Iowa, UWI and 

stakeholders planned that local chapters would raise the matching funds. UWI is restricted from 

fundraising in those areas to avoid competing for funds with the local chapters. Although some 

chapters have raised the matching funds consistently, others have never provided funding or 

stopped doing so over time. UWI has tried to raise funds from other sources, but success has 

been limited. Most significantly, after initial implementation in Iowa, the state legislature came 

close to appropriating funding for a large expansion, but ultimately did not include it in the 

budget, and has not done so despite advocacy from UWI. As a result, most school districts 

provide the matching funding, but some school districts cannot afford to participate at all or with 

as many tutors as needed, which limits the scaling of Reading Corps in Iowa; respondents 

perceived that this also threatens the intervention’s sustainability in some districts. These 

challenges would be greater if not for the significant portions of the cost covered by partners. 

Part of this stems from the design of Reading Corps, as the internal coach is a school 

employee. Part is Iowa-specific, as the AEA whose personnel dedicate some of their time to 

serving as master coaches also continues to pay for their full salary and benefits, facilitating 

UWI’s capacity to serve more schools.

UWI is able to obtain most of the other resources needed for scaling from the developer and 

school partners. As part of licensing Reading Corps from the developer, each school year UWI 

purchases the physical materials needed for implementation. Reading Corps is highly structured 

and has extensive materials, including a program manual. Tutors work in schools, so most of 

Reading Corps’ needs for physical space are provided by the school. Schools also provide 

tutors with a computer. However, UWI personnel had to resolve basic infrastructural issues such 

as obtaining or upgrading sufficient computers, printers, and office space for themselves. Some 

of these challenges occurred because UWI is a small organization whose capacity needed to 

expand significantly to scale Reading Corps. 

Use of data systems and evaluation during scaling. UWI primarily uses an extensive data 

system built and managed by the developer. This system is critical for tracking the weekly 

progress assessments and other implementation processes, in particular for monitoring tutor 

caseloads, student-specific amounts of tutoring received, and internal coach observations of 

tutors. Personnel also use it to communicate via notes about students, and to view intervention 

materials. The developer also has adopted commercial software to enable UWI personnel to 

more easily produce a variety of reports from the data system, which they have used to better

identify and address monitoring and other implementation-related issues. Overall, personnel find 

the system easy to use and well-supported by the developer. Finally, UWI also uses other data 

systems to track AmeriCorps member hours, member recruitment, and school applications to 

participate in Reading Corps.

As required by its current CNCS grant, UWI is engaging with an external evaluator to conduct a 

formal evaluation during the 2019-2020 school year, using a quasi-experimental design study. 

UWI looks forward to this evaluation because evidence from formal evaluations has so far 

played a limited role in encouraging districts to participate. This is because those evaluations 

took place outside of Iowa. Iowa-specific data from the Reading Corps data system and reports 

from the developer have been more influential, and UWI expects that an evaluation in Iowa will 
Scaling insights: United Ways of Iowa’s experience Mathematica

xiii

be more appealing to Iowa districts as well. Outside of this planned evaluation, however, UWI

faces challenges to conducting Iowa-specific evaluations and data use. Because the developer

requires UWI to replicate Reading Corps with fidelity, and not to adapt it, UWI has limited ability

to consider and test out local changes and improvements to the intervention. UWI personnel

also have limited time and resources to conduct evaluations or use data outside of prescribed 

monitoring processes The developer has an extensive process to update Reading Corps,

although it is for the intervention as a whole and only considers site-specific circumstances and 

experiences to a limited extent.

Conclusion. UWI’s scaling of Reading Corps reveals both successes and challenges in 

replicating and expanding an evidence-based intervention. This report seeks to help 

stakeholders understand the factors that might facilitate and hinder scaling, based on insights

from the experiences of one CNCS grantee scaling an evidence-based intervention. Two

companion reports provide further insights on the scaling experiences of the other two CNCS

grantees included in the process study. One report presents a case study of Parent Possible’s

scaling of Home Instruction for Parents of Preschool Youngsters (HIPPY), a home visiting

intervention that seeks to help parents improve their young children’s development (Anderson et

al. 2020). The other report presents a case study of the Child Abuse Prevention Council’s

scaling of the Birth and Beyond intervention, which seeks to provide parenting education and

support to parents of children in order to reduce child maltreatment (Eddins et al. 2020). 

Additionally, a fourth report (Needels et al. 2020) presents a cross-grantee analysis of 

information collected from all three grantees; that report includes insights on the commonalities 

and differences in how grantees scaled evidence-based interventions, and the challenges and 

facilitators they faced while scaling. 
This page intentionally blank for double-sided copying.
Scaling insights: United Ways of Iowa’s experience Mathematica

1 

I. INTRODUCTION 

Scaling interventions that have demonstrated effectiveness can increase the likelihood that an 

organization’s activities will improve participants’ lives (National Implementation Research 

Network 2016). At the same time, scaling can be challenging because it goes beyond 

implementation—the focus is not only on implementing an intervention but also on reproducing 

the same effects for a larger or different population, in a new or different location, or perhaps 

while modifying some of the intervention’s components. 

The project

The Corporation for 

National and 

Community Service is 

deepening its 

understanding of how 

to scale interventions 

deemed to be effective 

through the Scaling 

Evidence-Based 

Models project. 

The Corporation for National and Community Service (CNCS) 

has invested significant resources in supporting implementation 

of interventions designed to improve lives and strengthen 

communities through its AmeriCorps and Social Innovation Fund 

(SIF) programs.6

 CNCS also invests in evaluating the 

effectiveness of these interventions and supporting the scaling of 

those that have evidence of being effective to serve new 

communities or populations. Although many of the grants that 

CNCS funds are for scaling interventions, little systematic 

analysis has been conducted on how the grantees have planned to 

scale and what their experiences have been when doing so. 

Recognizing this, CNCS contracted with Mathematica in 2016 to conduct the Scaling EvidenceBased Models (SEBM) project, to deepen the agency’s understanding of interventions and its 

knowledge base on scaling them. 

The intervention 

and grantee

This report describes 

the scaling of Reading 

Corps, a literacy 

intervention designed to 

improve reading 

proficiency, by United 

Ways of Iowa and its 

partners.

This report presents a case study, using information from the SEBM project’s process study, of 

United Ways of Iowa (UWI), a CNCS grantee implementing the 

Reading Corps intervention in Iowa (see box at left). Reading 

Corps is a standardized literacy intervention that provides oneon-one tutoring to students enrolled in pre-kindergarten through 

third grade to help them achieve reading proficiency. This case 

study provides insights about how UWI and its partners is 

scaling Reading Corps, and the factors that appear to facilitate 

and hinder scaling. 

A. Overview of the SEBM process study

As part of the SEBM project, Mathematica conducted a process 

study examining how organizations that received CNCS grant 

6 AmeriCorps supports a wide range of local service programs through grants to address critical community needs, 

such as those pertaining to education, public safety, health, and the environment. The Social Innovation Fund 

supported programs from 2010 through 2016. SIF grants were used to fund community-based programs to address 

challenging social problems communities face in the areas of economic opportunity, healthy futures and youth 

development. CNCS (2016) and CNCS (n.d.) provide a detailed description of these programs.
Scaling insights: United Ways of Iowa’s experience Mathematica

2 

funding scaled evidence-based interventions. We define evidence-based interventions as 

interventions that have been demonstrated, through rigorous evaluation studies, to improve 

participant outcomes. The process study focused on how these grantees view scaling, the actions 

they take when they scale, and what factors appeared to facilitate or challenge scaling. 

Specifically, the process study aimed to address two overarching research questions: 

1. How did selected CNCS grantees define and 

operationalize scaling? For this research 

question, we describe the type of scaling that each 

grantee planned to undertake. The three types of 

scaling considered under the SEBM project are 

briefly defined in the box to the right. (See the 

appendix for more information about these 

definitions.) 

Types of scaling

Expansion extends the intervention 

to more people in the same target 

population in the same location.

Replication extends the intervention 

to the same target population in a 

new location.

Adaptation extends the intervention 

to a different target population in 

either the same or different location 

or modifies the intervention for the 

same population in either the same 

or different location.

2. How did selected CNCS grantees scale 

evidence-based interventions? For this research 

question, we describe how the grantees selected 

for the process study used organizational and 

implementation supports to facilitate scaling. We 

drew from the implementation science literature 

(see box on the next page) to identify supports 

that are typically needed. In documenting the extent to which grantees drew upon 

organizational and implementation supports, the process study also identified factors that 

appeared to facilitate and hinder scaling.

To answer these research questions, CNCS, with input from Mathematica, selected three 

grantees that were implementing interventions with evidence of effectiveness, meaning that 

evaluation studies of those interventions used rigorous research designs and had consistently 

favorable findings. The grantees and the interventions they were implementing also, when 

compared to other CNCS grantees, demonstrated a higher degree of scaling readiness. This 

meant that the grantees and interventions met the conditions expected to lead to successful 

scaling—that is, scaling the intervention while maintaining or exceeding the beneficial impacts 

documented in evidence about its effectiveness. (See the appendix for details of the selection 

process and data collection). 

The grantees selected for the process study—and the interventions they were scaling—differed 

with respect to the size of the grantee implementing an evidence-based intervention, intervention 

focus areas,

7

 planned types of scaling, how long the grantee had been scaling the intervention, 

reported successes and challenges with their scaling experiences, and the extent to which the 

grantees had attempted to apply lessons learned in the past. Because CNCS did not intend for the 

7 Intervention focus area refers to the topics in which CNCS concentrates its funding: disaster services; economic 

opportunity; education; environmental stewardship; healthy futures (including physical and mental health, 

substance abuse, and nutrition); and veterans and military families. 
Scaling insights: United Ways of Iowa’s experience Mathematica

3 

grantees selected for the process study to offer interventions that were typical of all CNCS 

grantees, the insights from their scaling experiences might not apply to a broader set of CNCSfunded grantees or service providers. Still, because of their scaling readiness strengths, the 

findings from the process study can provide insights about scaling practices that can help 

stakeholders understand the conditions that might facilitate or hinder intervention scaling.

Mathematica collected and analyzed data from four sources: (1) a review of documents relevant 

to each intervention and its scaling supplied by the grantees or their partners, (2) a two-day visit 

during October 2018 to each grantee and local partners involved in scaling the interventions, (3) 

brief telephone calls with grantee personnel shortly before and after the visits, and (4) 12-month 

follow-up telephone interviews conducted with grantee personnel in September 2019.8

Information from these sources was compiled to identify insights about scaling particular to each 

grantee. (See the appendix for a more detailed discussion of the data sources.) 

This report presents a case study of one of three grantees 

included in the process study: United Ways of Iowa (UWI), 

implementing the Reading Corps intervention in Iowa. We 

adapted the process study’s research questions for UWI’s scaling 

of Reading Corps in Iowa. As the CNCS grantee, UWI partners 

with school districts to implement Reading Corps to eligible 

students in schools across Iowa. Under a 2015 CNCS grant, 

UWI scaled Reading Corps to a larger number of schools in the 

state. UWI initially scaled Reading Corps (via replication) in the 

state starting in 2012. This report considers the full time frame of 

UWI’s implementation of Reading Corps in Iowa (since 2012). 

However, this report focuses on the period since the 2015 CNCS 

grant. At the national level, Reading Corps’ scaling is supported 

and monitored by Reading and Math, Inc., located in Minnesota, 

where the intervention was developed and first implemented. 

What is 

implementation 

science?

Implementation science 

is the scientific 

investigation of factors 

associated with effective 

implementation of an

evidence-based 

intervention or practice 

(Franks and Schroeder 

2013).

In seeking to answer the process study’s research questions with a focus on this single grantee, 

this report describes how UWI is scaling an evidence-based intervention, providing an in-depth 

focus on the grantee’s implementation activities. This report aims to deepen understanding 

among funders, policymakers, and service providers on UWI’s efforts to scale an intervention 

with evidence of effectiveness. Two companion reports discuss findings from our case studies of 

the other two CNCS grantees included in the process study—Parent Possible, implementing the 

Home Instruction for Parents of Preschool Youngsters (HIPPY) intervention (Anderson et al. 

8 We use the terms personnel and workforce to refer to organization and partner personnel as well as AmeriCorps 

members who deliver intervention services. 
Scaling insights: United Ways of Iowa’s experience Mathematica

4 

2020), and the Child Abuse Prevention Council, implementing the Birth and Beyond intervention 

(Eddins et al. 2020).9

 

B. Overview of the Reading Corps intervention

Reading Corps is a standardized literacy intervention that provides one-on-one tutoring to 

students enrolled in pre-kindergarten through third grade to help them achieve reading 

proficiency.10 Reading Corps was originally developed in Minnesota by the state AmeriCorps 

commission, ServeMinnesota, and was first implemented in that state in 2003. The intervention 

seeks to engage students who are reading below grade level, but not low enough to qualify for 

more intensive assistance (for example, Title I reading assistance).

11 Intervention personnel 

identify eligible students using benchmark assessments that predict third-grade reading 

performance.

Reading Corps features 20 minutes of one-on-one, in-school reading tutoring each school day. 

Tutoring consists of a set of 10 scripted activities12 that target phonemic awareness, phonics, 

reading fluency, and comprehension skills. The specific set of activities used with each student is 

individualized based on their needs and progress; these activities range from blending and 

segmenting phonemes to reading text aloud multiple times to improve fluency. Students are 

removed from their regular classrooms, but not during core reading instruction, for tutoring. 

Students also receive materials and short literacy activities to take home to engage in with their 

families. Tutors track student progress by administering short weekly assessments. Students 

whose scores reach and remain above a specified threshold of reading proficiency exit the 

intervention. 

Tutors are AmeriCorps members—individuals participating in local service programs funded by 

CNCS who commit their time to addressing critical community needs through engaging in 

national service. Members engage in terms of service, which specify the number of hours that 

they are committed to serve.

13 Each tutor serves a caseload of 15 to 18 students in a school, and 

9 A previously published report (Needels et al. 2020) presented a cross-grantee analysis of information collected 

during the process study visits from all three grantees. The insights from this analysis pertain to two broad areas: 

(1) the approaches that grantees and their partners took to scaling—including how grantees viewed scaling and

their actions when the scaling was taking place; and (2) specific commonalities and differences in how they

scaled, and the challenges and facilitators they faced with these aspects of scaling. In contrast, the three case study

reports provide deeper insights into the scaling experiences of each of these grantees.

10 See the Reading Corps websites for more information: http://www.readingcorps.org/ and

https://minnesotareadingcorps.org/. 

11 However, according to a UWI respondent, starting in 2019–2020, the developer broadened the eligibility criteria 

to include all low-scoring students. Chapter II, Section B discusses this change in more detail.

12 Reading Corps uses different terminology than the SEBM process study and these three case study reports: 

Reading Corps uses the term “program” to refer to the overall Reading Corps intervention and “intervention” to 

refer to one or more of the 10 specific literacy activities delivered through Reading Corps tutoring.

13 This report refers to AmeriCorps members serving as Reading Corps tutors as “members” and “tutors” 

interchangeably.
Scaling insights: United Ways of Iowa’s experience Mathematica

5 

is overseen by an internal coach, who is a classroom teacher, reading specialist, or other educator 

at the school. Internal coaches observe tutors twice per month to monitor fidelity, which is the 

extent to which implementation of an intervention matches the intervention as designed. In this 

case, fidelity means that the tutors are delivering the literacy activities as designed and scripted. 

Internal coaches also handle most interactions with parents and students’ classroom teachers, 

including coordinating with teachers on when each student should leave the classroom for 

tutoring. 

Each tutor and internal coach are also overseen by a master coach, who is hired or contracted by 

the grantee based on the candidate’s literacy and coaching expertise. Master coaches meet with 

each tutor and internal coach monthly to conduct case reviews. During these reviews, this threeperson team examines each student’s progress based on the assessments, and uses the results to 

update the individualized approach to tutoring each student as needed. Exhibit 1 illustrates the 

personnel structure of the Reading Corps intervention. 

Exhibit 1. Personnel roles in Reading Corps intervention
Scaling insights: United Ways of Iowa’s experience Mathematica

6 

Reading Corps is an evidence-based intervention. The 10 scripted literacy activities it uses were 

derived from empirical studies of effective instructional practices in reading and literacy. The 

overall Reading Corps intervention has demonstrated evidence of effectiveness. A 2014 study, 

using a randomized controlled design, found positive effects on literacy assessment scores, with 

larger effects on younger students (kindergarten and first grade) and positive effects across 

different subgroups of students based on gender, race, dual language learner status, and lowincome status (Markovitz et al. 2014). 

According to our document review, since its initial implementation in Minnesota, the Reading 

Corps intervention has expanded to 12 states and the District of Columbia, serving more than 

35,000 students during the 2017–2018 school year. It is currently overseen by Reading and 

Math, Inc., which was created as a separate organization from the original developer 

(ServeMinnesota) to oversee the scaling of Reading Corps.14 Reading and Math, Inc. implements 

the Reading Corps intervention directly in some locations (primarily Minnesota) and licenses 

other organizations to replicate it in other locations, with the requirement that the intervention be 

implemented with fidelity. It also supports the other organizations implementing Reading Corps, 

such as by providing training and technical assistance, a data system, and intervention materials. 

The developer updates the Reading Corps intervention based on data and research from 

intervention locations. 

In the remainder of this report, we identify the types of scaling pursued by UWI (Chapter II), 

describe how it scales Reading Corps and discuss the factors that appear to facilitate and hinder 

scaling (Chapter III), and summarize our findings (Chapter IV). In the appendix, we describe the 

process study’s design and the methodologies used to collect and analyze data for this process 

study. 

14 This report refers to Reading and Math, Inc. as “the developer” because it oversees implementation of Reading 

Corps, including licensing it to other organizations, and maintains and updates the intervention.
Scaling insights: United Ways of Iowa’s experience Mathematica

7 

II. HOW DID UWI DEFINE AND OPERATIONALIZE SCALING? 

UWI, a statewide association of local United Way chapters, has received CNCS grants since 

2012 to scale Reading Corps in Iowa. UWI partners with local school districts to implement 

Reading Corps in schools. In scaling Reading Corps, UWI has both replicated and expanded

the intervention, implementing it with fidelity as required. 

A. Replicating and expanding Reading Corps

UWI originally replicated Reading Corps by licensing it from the current developer in 2012 to 

implement it in the state. UWI personnel identified three factors that led to the initial replication 

of Reading Corps in Iowa. First, the Iowa legislature had passed a law mandating grade retention 

for students not proficient in reading by the end of third grade, so policymakers focused on 

supporting grade-level reading. Although the mandatory 

grade retention was later removed before it could take effect, 

provisions in the law remained for schools to identify 

children at risk for not becoming proficient in reading and 

providing interventions to support literacy. Second, Iowa’s 

state commission on volunteer service identified Reading 

Corps as an intervention using AmeriCorps members that 

could benefit the state by improving student outcomes in 

reading. Third, the local United Way chapters that formed 

UWI as their statewide association wanted to support a 

reading intervention collectively through UWI. 

Scaling Reading 

Corps in Iowa

In Iowa, Reading Corps 

has been replicated

(brought to new locations 

for the same target 

population) and 

expanded (delivered to a 

larger population in 

existing locations).

After a planning year in 2012–2013, UWI implemented 

Reading Corps during the 2013–2014 and 2014–2015 school years in about eight or nine schools 

in five districts, with about 10 to 15 AmeriCorps members serving as tutors. For the 2015–2016 

school year, UWI received a larger grant from CNCS (this is the grant that was reviewed by the 

SEBM project and led to UWI’s selection for this process study). UWI used its 2015–2016 

CNCS grant to expand Reading Corps to more schools in existing districts and to further 

replicate it in new districts in Iowa. Since 2015–2016, the intervention in Iowa has had about 65 

to 70 members serving as tutors in about 60 to 65 schools in as many as 30 districts. Members 

have been supported by about 60 to 65 internal coaches (because there is usually one per school) 

and about 12 (at the time of the site visit) to 14 (at the time of follow-up calls) master coaches. 

During the 2018–2019 school year, about 1,700 students received Reading Corps tutoring.

UWI personnel explained that UWI would like to engage the highest-need schools and districts 

within the state for Reading Corps services. However, the specific locations to which Reading 

Corps is replicated and at which it is expanded depend heavily on local factors. Many, but not 
Scaling insights: United Ways of Iowa’s experience Mathematica

8 

all, of these factors involved similar considerations when replicating the intervention in new 

districts and expanding it in existing districts.15

• First, school districts have to agree to participate, and the primary considerations are their 

interest and whether funding is available. In some areas of the state, local United Way 

chapters have raised funds for the proportion of the cost of each tutor not funded by CNCS. 

In other areas, school districts must pay this proportion of the cost. (Chapter III, Section C 

explains this issue in more detail.) 

• Word of mouth and educator movement to new schools affected districts’ interest in 

participating. Respondents reported that more districts became interested as they heard from

teachers and principals who were satisfied with how their students had benefited, or as 

teachers and principals from a district with Reading Corps moved to a new district and 

wanted to bring the intervention there. At the same time, when a tutor performed poorly, 

word could spread and discourage districts and schools from participating. Respondents said 

this latter situation had only happened on a few occasions and did not indicate that it had 

been a major limitation.

• Some districts might incorrectly think they are not eligible to participate. During follow-up 

calls in September 2019, one respondent relayed that personnel in a newly participating 

district had said they previously thought the district was not eligible. This might have 

occurred because during the first two years, districts could only participate if their local 

United Way chapter raised the additional funding; however, after Reading Corps in Iowa 

expanded, the funding could come from any source. Although this was the only time UWI 

has heard about this misperception, the respondent said UWI plans to reach out to districts 

next year to correct any potential misperceptions about eligibility.

• Another local factor is AmeriCorps member recruitment. Although UWI recruits 

AmeriCorps members, members interview for a spot in a specific school. As a result, 

participation depends on whether UWI can find a satisfactory candidate to serve in a 

particular school. 

• Participation is a challenge with small schools. If not enough students in the target population 

fill one tutor’s caseload (15 to 18 students), assigning a tutor to that school is an inefficient 

use of resources.

• When working with districts to determine which schools will receive a tutor, UWI prioritizes 

higher-need schools to the extent possible. 

15 To the extent that respondents described scaling approaches, their definitions sometimes differed from those used 

by the SEBM project. They discussed expansion as increasing the size of implementation in Iowa and updating or 

maintaining operations at the larger size. In contrast, they discussed implementing Reading Corps with fidelity as 

replication, which they saw as necessary regardless of the size of implementation.
Scaling insights: United Ways of Iowa’s experience Mathematica

9 

Some of these factors have limited the extent to which Reading Corps is scalable in Iowa. UWI 

proposed in its 2015–2016 CNCS grant application for about 75 members to serve as tutors16, 

but the actual number has ended up slightly smaller because of these limitations, although as of 

September 2019 UWI anticipated getting closer to this number in 2019–2020 than in previous 

years. UWI has also not previously attempted formal outreach to districts, instead relying on 

districts to contact them to express interest; doing so could increase participation. UWI had 

previously planned to scale in more than half of the state’s districts; however, the funding from 

the state legislature to do so never materialized.17 State funding would have covered the 

proportion of costs for each tutor that districts or local United Way chapters currently must 

provide. 

B. Scaling while implementing Reading Corps with fidelity

Reading Corps is an evidence-based intervention, and 

Reading and Math, Inc. requires organizations that license 

it to implement it with fidelity. This means UWI avoids 

adapting the intervention. Instead, it adopts any updates to 

the intervention made by the developer. Developer 

personnel said they usually pilot test major updates in 

Minnesota first and that changes take effect at the start of a 

new school year. UWI is committed to fidelity and does 

not view the requirements as onerous or limiting. 

Respondents consistently noted the importance of fidelity 

so the intervention could be effective, as supported by past 

evidence. These comments included:

Implementation 

fidelity

UWI licenses Reading 

Corps from the 

organization that 

developed and oversees it. 

The license requires UWI 

to implement the 

intervention with fidelity.

• From a tutor: “Making sure that we’re highly trained in these interventions and that we’re 

doing them with fidelity I think is very 

critical.” 

• From an internal coach: “The fidelity 

check [of the tutor] is one of the most 

important parts... because when they 

stick with [implementing as intended] it’s 

proven through evidence that it works.” 

“The fidelity check [of the tutor] is one of the most 

important parts... because when they stick with 

[implementing as intended] it's proven through 

evidence that it works.”

– Internal coach

• From a master coach: “Our big responsibility is to ensure the fidelity of the program.” 

16 UWI proposed the equivalent of 75 tutors serving full time, but because a few tutors are not full time, the total 

number of tutors would have been slightly higher.

17 The legislature did add a section to the state administrative code about Reading Corps, which one respondent said 

encouraged school districts to participate in Reading Corps instead of pursuing alternative interventions or 

approaches.
Scaling insights: United Ways of Iowa’s experience Mathematica

10

The developer also provides extensive assistance and supports to Iowa and other sites, including 

intervention materials, a data system, and training and technical assistance for master coaches.

UWI implements Reading Corps differently in a few ways, although for the SEBM project we do 

not consider these to be modifications of the intervention because they do not alter any Reading 

Corps activities or other aspects of the intervention prescribed by the developer. First, UWI uses 

a modified definition for intervention completion, counting students who participate for at least 

12 weeks as completing Reading Corps. However, UWI uses this definition only to count 

performance benchmarks for CNCS, which requires grantees to define such benchmarks. UWI 

still follows the Reading Corps procedures, where students continue to receive tutoring until they 

consistently reach their target scores on assessments, regardless of their status at 12 weeks. 

Second, much of the evidence base for Reading Corps comes from evaluation of the developer’s 

implementation in Minnesota. In at least one evaluation (Markovitz et al. 2014), some (but not 

all) kindergarten students in Minnesota received a different tutoring approach, featuring doubled 

daily intensity (40 minutes instead of 20) and adding small-group activities instead of solely oneon-one tutoring. Students in other grades continued to receive the same approach of 20 minutes 

of one-on-one tutoring daily. However, at the time of the site visit in October 2018, the 

developer reported that this different approach remains confined to sites in Minnesota; the 

developer is still considering under what circumstances UWI and other implementing 

organizations could adopt the approach. In the meantime, UWI continues to use the 20-minute, 

fully one-on-one approach with kindergarten students. 

Third, UWI has tutors conduct additional literacy activities in the summer. Tutors choose or 

develop their own activities, which UWI personnel must approve. These do not have to consist 

of the scripted Reading Corps activities, and tutors do not administer any of the weekly 

assessments to monitor students’ reading progress. The summer activities do not even have to 

occur at the tutor’s school or serve the students who received tutoring during the school year. For 

example, one tutor helped conduct a summer reading program at her school, while another 

helped with a different summer reading program at a local library and set up a table there with 

themed books and activities. UWI added this component so AmeriCorps members could reach 

the required annual number of service hours for a full-time member (1,700 hours). The summer 

activities do not affect any aspect of the Reading Corps intervention or the activities that occur 

during the school year. 

For the 2019–2020 school year, Reading and Math, Inc. adapted the Reading Corps intervention 

by broadening the target population. However, for the SEBM project, we do not consider this to 

be a modification of the intervention by UWI because it continues to replicate the developer’s 

(now updated) model. Specifically, the developer broadened the eligibility criteria to include all 

students who previously would have scored too low to qualify for Reading Corps services (under 

the logic that they were eligible for more intensive assistance). One UWI respondent attributed 

the change to feedback from schools that when these children did participate, their literacy skills 

were improving, even if they had not yet reached proficiency. This respondent also explained 
Scaling insights: United Ways of Iowa’s experience Mathematica

11

that the change was feasible because the developer had shifted its performance goals from 

focusing on levels of proficiency to emphasizing student growth. Unlike measures based on 

reading proficiency, measures based on growth are potentially within reach for all students, even 

the lowest-scoring ones. 
This page intentionally blank for double-sided copying.
Scaling insights: United Ways of Iowa’s experience Mathematica

13

III. HOW DID UWI SCALE READING CORPS? 

To understand how UWI supported scaling of the Reading Corps intervention in Iowa, we 

describe aspects of implementation that are identified as having key roles in scaling 

interventions. Each of these components is shown to help organizations scale interventions while 

they seek to generate the same beneficial participant outcomes that occurred before scaling 

(National Implementation Research Network n.d). Namely, we examined the following: 

• How the workforce helped to carry out Reading Corps implementation 

• How grantee and partner personnel used monitoring and communication systems to support 

implementation as intended 

• The sufficiency of funding and other resources, such as materials and physical space 

• The use of data systems to monitor ongoing implementation and inform any changes that 

might need to be made, and evaluation to assess whether a scaled intervention is still 

producing the same outcomes observed in prior research

A. Organizational workforce

Engaging supportive leadership and sufficient personnel members, who have been appropriately 

trained in their duties, can support intervention scaling. Strong leaders can provide creative 

solutions to implementation problems as well as other meaningful implementation supports 

during scaling (Bernfeld 2006). Implementation science literature also suggests that specifying 

workforce characteristics, such as requirements around the types of education and experience 

that personnel should have, supports strong implementation (Fixsen et al. 2005, 2013).

Additionally, procedures to train personnel have been shown to facilitate scaling the intervention 

with fidelity, meaning the extent to which implementation of an intervention matches the 

intervention as designed (Breitenstein et al. 2010; National Implementation Research Network 

n.d.).

1. Approach to structuring and training the workforce

According to the document review and interviews with respondents, UWI has two full-time 

Reading Corps personnel: a program director and a program manager. The current program 

director was hired when UWI received the CNCS grant in 2015–2016 to expand and further 

replicate Reading Corps in Iowa. Since 2015–2016, three people have filled the role of program 

manager; the most recent was hired in summer 2019. UWI also contracts with a bookkeeper to 

handle fiscal tasks for Reading Corps, such as payroll and grant reporting; she also performs a 

small amount of non-Reading Corps work for UWI. UWI is a small organization; outside of 

these three personnel, at the time of the site visit in October 2018, UWI’s only other employees 

were a small number of personnel implementing a second, unrelated grant program (which had 

ended by September 2019), and the UWI executive director. 
Scaling insights: United Ways of Iowa’s experience Mathematica

14

The executive director oversees UWI’s budget and finances for Reading Corps and manages 

UWI’s Reading Corps personnel, using an informal approach to supervision. She also

communicates and promotes the intervention with a range of stakeholders, including local United 

Ways chapters, businesses, and state legislators and policymakers. Her primary duties are outside 

of Reading Corps and relate to UWI’s status as a statewide association of local United Ways 

chapters. 

The vast majority of those delivering Reading Corps in Iowa are not UWI personnel. UWI 

recruits people to join AmeriCorps and serve as tutors in specific schools, primarily using a 

major job search website, supplemented by initiatives such as Service Year Alliance. 

Participating schools provide an employee to fill the internal coach role. This role is expected to

encompass 6 to 9 hours each month, although more time is involved in the beginning of the 

school year. One of Iowa’s area education agencies (AEAs) designates its personnel who already 

work with schools to take on the master coach role for schools that participate in Reading 

Corps.18 At the time of the site visit, 9 of the 12 master coaches, including the lead master coach, 

were AEA personnel. They each support a relatively small number of schools and tutors because 

they only serve the schools where they already provide other services. The remaining three 

master coaches are contracted directly by UWI, and each serves more schools (one works nearly 

full time as a master coach). 

Training is prescribed by the Reading Corps developer as part of the intervention. UWI holds an 

initial four-day training in September of each year, shortly after the school year starts. In 

Minnesota, the initial Reading Corps training is held before the school year begins, but in Iowa 

teachers return to work closer to the start of the school year, which does not leave enough time 

before the year begins for the Reading Corps training. This training covers the following: 

• AmeriCorps, including member regulations, policies, and expectations 

• Reading Corps, including its background and history, principles of literacy and reading 

instruction, and delivering Reading Corps, including practice delivering the literacy activities

To avoid overwhelming trainees during the initial training, two additional, one-day trainings take 

place later: 

• One in October of each year to cover the Reading Corps data system 

• One in November that covers how to deliver the most complex of the 10 literacy activities 

Trainings are attended by AmeriCorps members and internal coaches, and conducted by master 

coaches who have at least one year of experience and have attended a two-day train-the-trainer 

meeting held by the developer. The developer also supports the master coaches who serve as 

trainers through co-training support during their first training and observations of the training. 

18 AEAs are funded by the state and provide a variety of services to school districts, including special education 

services. Each AEA serves a geographic area in the state.
Scaling insights: United Ways of Iowa’s experience Mathematica

15

The developer provides additional training and support, including (1) training for new master 

coaches, (2) customized training for each organization implementing Reading Corps (including 

UWI) on the Reading Corps data reporting system, and (3) an annual national meeting in 

Minneapolis for personnel from all implementing organizations. 

Whereas the developer provides a single statewide training for implementing Reading Corps in 

Minnesota, UWI started holding trainings in two locations in Iowa after expanding to the current 

number of members and schools. This is primarily to limit travel times. It also has the benefit of 

exposing some members and internal coaches to a different master coach than the one who will 

work with them during the school year. One respondent noted in September 2019 that the size of 

Reading Corps in Iowa was almost large enough to add a third training location. 

For the 2019–2020 school year, Reading and Math, Inc. excused returning internal coaches from

in-person training if they passed an online test. One UWI respondent noted this led to more work 

for UWI personnel who had to review the test results and explain any areas of concern to each 

coach. However, the respondent added that the online test process might better prepare internal 

coaches because it requires them to review the Reading Corps material more closely to pass. This 

respondent also expressed a desire to conduct more training remotely especially as the 

intervention is further expanded and replicated in Iowa, to ease the burden of travel for personnel 

and the cost to the intervention. 

2. Facilitators and challenges to structuring and training the workforce

Partners have been essential in providing the workforce needed to deliver Reading Corps.

This support has helped UWI with each type of personnel delivering or overseeing tutoring. 

• The AEA’s designation of its personnel to serve as master coaches means UWI does not need 

to recruit and hire as many people for this role. 

• School personnel serving as internal coaches is an inherent 

part of Reading Corps. However, respondents said that, as 

part of participating districts’ and schools’ strong support 

for Reading Corps, they have largely selected and 

supported internal coaches who effectively carry out their 

intervention duties. For example, the district and school 

officials interviewed described deliberate processes of 

identifying the most appropriate people and asking for 

their agreement to serve as the internal coach. In one 

district, these were reading specialists who already had the 

most relevant expertise among school personnel; in one 

school in another district, the principal identified the 

person viewed as the best fit based on the skills needed. 

Promising practice 

about the workforce

Having internal coaches 

(due to the design of 

Reading Corps) and master 

coaches (due to in-kind 

support from an area 

education association’s 

personnel) who already 

work in the participating 

schools facilitates having

the personnel needed to 

deliver Reading Corps.

• As Reading Corps has expanded, districts and schools have 

helped recruit AmeriCorps members. Respondents mentioned that some districts use their 
Scaling insights: United Ways of Iowa’s experience Mathematica

16

knowledge of their area to reach out to groups of candidates, such as people who moved 

there recently and want to become teachers, or people with children who want to return to the 

workforce. One respondent added that district and school participation in recruiting can 

increase their commitment to the intervention. However, this respondent said most districts 

still need help with recruiting and hiring candidates, in part because of all the other demands 

on their time. To meet this need, some local United Way chapters have also helped recruit 

members.

“...the best training I’ve 

ever received for any job.” 

– Tutor

Training was cited as a facilitator of the intervention’s success. Respondents consistently 

praised the training as thorough and helpful to the tutors and internal coaches, with one tutor 

calling it “the best training I’ve ever received for any job.” Tutors 

explained this was important because the first few months of 

delivering the intervention could feel overwhelming simply 

because it requires intensive effort and attention to detail. One 

tutor explained, “If you just read the scripts and do it like they tell 

you to do it then you’re okay. But you’re just overwhelmed 

because this is the first time you’ve ever done it.” They said both high quality training and 

experience delivering Reading Corps are needed to become comfortable in their role. Tutors 

mentioned a couple additional training topics that might be useful, including on organizing their 

Reading Corps materials to facilitate efficient service delivery and on dealing with children who 

have behavioral difficulties.

Lesson learned 

about the workforce

A large increase in 

capacity is difficult to build

in a short period of time, 

especially for a small 

organization with limited 

initial capacity.

One key challenge is having enough personnel to oversee 

and manage the intervention. Respondents reported the 

number of UWI personnel has not been enough to handle the 

workload of implementing Reading Corps at its current scale. 

One respondent said another program manager and an 

administrative assistant were needed. This workload issue stems 

in part from how UWI structured its budget to implement

Reading Corps. According to one respondent, at the start of the 

larger 2015–2016 CNCS grant, UWI had set the level of 

matching funding per tutor too low to afford a third UWI

workforce member dedicated to the Reading Corps intervention. During the follow-up calls in 

September 2019, this respondent explained that UWI could not increase the matching funding 

level because schools would have too much difficulty providing a higher amount, given the 

challenges meeting the existing level (as discussed in Section C of this chapter). Because UWI is 

so small, it has limited capacity to support the intervention workforce’s implementation efforts. 

One respondent noted that “we wrote a great organizational capacity plan that said we’re going 

to do this and we’re going to do that,” but then struggled to find the time needed to carry out the 

proposed plan. As of September 2019, UWI had updated procedures for a couple of timeintensive administrative tasks, including bringing in the contract bookkeeper for more support, 

which had freed up some time for the program director to work on other tasks.
Scaling insights: United Ways of Iowa’s experience Mathematica

17

Personnel from partners serving as coaches can also lead to capacity-related challenges.

Similar to workload challenges UWI personnel face, some internal coaches struggle to find the 

time needed for Reading Corps, but in this case it is because of their other duties as educators. As 

one master coach said, “They feel a huge commitment to the students they serve [in their regular 

role], so it’s hard for them.” Master coaches address the issue in part through what one described 

as “gentle reminders” that the school agreed to commit the internal coach’s time to devote to 

Reading Corps, along with suggestions on how to fit in their Reading Corps activities. If needed, 

master coaches or UWI personnel will follow up with the school principal. Master coach 

respondents added that the AEA-provided master coaches also struggle at times to balance their 

Reading Corps role with their other duties. 

“At the end of the day there’s only 

so many people who want to work 

for [the stipend amount].” 

– UWI respondent

As Reading Corps has expanded, recruiting people to serve as tutors has become more 

difficult. Respondents attributed this in part to improving economic conditions, which decrease 

the appeal of the AmeriCorps stipend amount. Iowa had an especially strong economy at the time 

of the site visit in October 2018, with unemployment among 

the lowest in the nation.

19 Strong economic conditions can 

also make it harder to retain members, although as of late

September 2019, only two members had left since the 2019–

2020 year had started, in late August 2019. One UWI 

respondent said members are unlikely to leave after the first 

couple months of the school year. The respondent added that 

UWI is generally able to replace members who leave early in the year because they can be 

caught up to speed at the trainings in the fall. Some members end up leaving after the school 

year, before fulfilling the summer component. 

Iowa also has a large set of national service programs, which effectively compete for people 

willing to serve as AmeriCorps members. As one UWI respondent said, this means “flooding the 

market with national service opportunities in Iowa... all of us struggle to some degree or other to 

fill our grants, because at the end of the day there’re only so many people who want to work for 

[the stipend amount].” Finally, recruiting members in rural areas has been more challenging, 

because there are fewer candidates and, in some cases, schools ruled out a local candidate who 

was already known to school staff for reasons unrelated to the candidate’s potential tutoring 

ability. According to one UWI respondent, efforts by CNCS to help increase the visibility and 

prominence of national service—making it “part of the national culture”—could facilitate local 

recruiting. The respondent mentioned a previous series of panels on national service held by the 

Department of Defense and CNCS, including one held in Iowa, as an example, but had not heard 

about any follow-up or similar efforts since then. 

UWI personnel said one challenge was not providing enough additional developmental and 

growth opportunities for AmeriCorps members beyond direct Reading Corps training. For 

example, these could cover broader issues around literacy and education, diversity and 

19 See state unemployment rates from the Bureau of Labor Statistics for 2018: https://www.bls.gov/lau/lastrk18.htm. 
Scaling insights: United Ways of Iowa’s experience Mathematica

18

communication in the workplace, or the communities they are serving, or targeted workforce 

skills such as writing resumes and responding to interview questions. Respondents noted these 

had been an important part of other AmeriCorps programs they managed or participated in. 

During the October 2018 site visit, respondents indicated that it is challenging to find time for

such additional developmental and growth opportunities because AmeriCorps members spend 

their time tutoring students, and UWI personnel do not want to lose tutoring time by pulling 

members out of schools to attend them. Although they did not describe this challenge as 

affecting implementation of Reading Corps, respondents were concerned about how it would 

affect what members learned from their AmeriCorps experiences. As of September 2019, UWI 

had developed more of these opportunities, spurred by a large number of snow days the previous 

winter that led to a need for members to make up lost hours. UWI structured these opportunities 

by having members read material on their own and then participate in online discussions at times 

when they were not tutoring, such as after school or on a weekend. 

After the intervention scaled up to its current size in Iowa, UWI had to more clearly define 

policies and procedures, particularly those regarding member supervision. Respondents 

noted that policies and procedures in place when there were fewer members allowed for more 

flexibility that would not be sustainable at a larger scale. One respondent explained, with fewer 

members “you can make the one-off exception [to a supervision or human resources-related 

policy for members] and say, ‘Oh yeah, that’ll be okay’ because you know you’re not going to 

have 30 other people potentially asking you to do exactly the same thing.” As another example, 

UWI limited members’ flexibility in choosing summer activities to reduce the amount of time 

UWI personnel would spend reviewing and approving them. 

Changes in CNCS policies and procedures for enrolling and exiting AmeriCorps members 

have implications for UWI personnel’s time managing members. One respondent said CNCS 

policies and procedures were not burdensome overall but described some changes that led to 

UWI personnel spending more time on these tasks. For example, this respondent explained that 

CNCS had removed the paper option for the member enrollment and exit forms, and that some 

members struggled to use the online version, requiring UWI personnel to travel to meet them inperson or otherwise arrange for help with filling out the forms. However, the respondent also 

praised changes that had reduced the burden on UWI personnel. For example, the CNCS 

citizenship verification process has been much prompter than in the past. The respondent also 

explained that CNCS had recently allowed organizations implementing interventions with 

AmeriCorps members to use a vendor to help manage the fingerprinting process for Federal 

Bureau of Investigation background checks. This process was more efficient and saved 

considerable time for UWI personnel; the respondent said it was well-suited for organizations 

operating at a larger scale. 
Scaling insights: United Ways of Iowa’s experience Mathematica

19

B. Systems to monitor implementation and facilitate communication

In studying the conditions under which evidence-based interventions are implemented, research 

on implementation science has identified specific supports that can help to ensure an 

intervention’s fidelity, which is important to scaling. Lack of fidelity can be a reason why 

interventions might produce good outcomes when initially implemented but then fail to yield the 

same outcomes when scaled (Breitenstein et al. 2010). Robust systems that track measures 

related to fidelity and have processes in place to address challenges that arise can help ensure 

that an intervention maintains the beneficial outcomes that it produced before scaling. 

According to implementation science research, a system to foster communication among 

organizational personnel, as well as personnel from partner agencies, can be another critical 

support for fidelity during scaling. Frequent communication should be maintained so that leaders 

can constructively intervene, address challenges, and strengthen implementation supports on an 

ongoing basis (Nord and Tucker 1987). Researchers have found that better adherence to 

intervention components might be related to implementing agencies demonstrating high quality 

communication between stakeholders, including well-specified channels of communication, 

common goals, and clear lines of authority (Mihalic and Irwin 2003, Fagan et al. 2008, Fagan 

and Mihalic 2003). 

1. Monitoring implementation

a. Approach to monitoring implementation 

UWI uses the extensive structure and processes built into the Reading Corps intervention to help 

monitor implementation fidelity. As one master coach noted, “The beautiful thing about the 

program is that it is really built in for us to have the best practices regarding fidelity.” These 

processes include the following: 

• Internal coaches have primary responsibility for monitoring AmeriCorps members and how 

they tutor students. While observing tutors, internal coaches must use a checklist to verify the 

tutor is delivering the literacy activities with fidelity, and immediately correct any mistakes 

the tutor makes. Internal coaches ensure that tutors deliver literacy activities as scripted but 

also broadly monitor how tutors interact with students. 

• Master coaches monitor internal coaches and tutors as well as students’ progress in the 

intervention. In addition to the monthly meetings to review students’ progress, master 

coaches observe internal coaches and tutors to ensure they are fulfilling their roles with 

fidelity. Several respondents said the master coach role was an important strength of Reading 

Corps, especially because master coaches’ expertise about the literacy activities freed up 

UWI personnel to monitor other aspects of implementation.

• The lead master coach and developer personnel also monitor master coaches. The lead master 

coach observes each master coach from one to three times per year, depending on his or her 

tenure as a master coach. The lead master coach also monitors other aspects of 
Scaling insights: United Ways of Iowa’s experience Mathematica

20

implementation, such as caseloads and amounts of tutoring received. Originally, the 

developer conducted more of these observations in Iowa but has shifted this work to the lead 

master coach, building the coach’s capacity in these duties through training and technical 

assistance. 

• UWI personnel also conduct monitoring, primarily to ensure members are following 

AmeriCorps policies and procedures. This includes having members fill out timesheets and 

tracking that they are on pace to reach their required number of service hours. However, 

UWI personnel also monitor aspects of implementation such as tutor caseloads and amounts 

of tutoring each student receives, tutors’ entry of assessment results in the data system, and 

internal coaches’ observations of tutors. 

b. Facilitators and challenges to monitoring implementation

Respondents consistently indicated that monitoring 

processes are being followed with fidelity to the Reading 

Corps intervention. In addition to following the prescribed 

processes, internal coaches also meet with tutors outside of 

observations; tutors reported that their coach formally meets 

with them anywhere from weekly to monthly to as needed, but 

that informal contact is much more frequent, often daily. Also, 

internal coaches, master coaches, and the lead master coach 

explained that when they notice during observations or other 

monitoring (such as data system reports) that a tutor or coach is 

struggling with his or her duties, they conduct extra observations 

to provide that person with more oversight and support. Tutors, 

internal coaches, and master coaches all expressed appreciation for one another and said they 

work well together. The only indication of deviating from these processes came from one 

internal coach who mentioned observing their tutors at least once or twice per month, when the 

intervention calls for twice per month. 

Promising practice 

with monitoring

Having workforce roles 

and processes for 

monitoring—among 

other implementation 

supports—built into 

Reading Corps helps 

ensure the intervention is 

implemented with fidelity.

Respondents described using monitoring processes to identify and address several 

implementation issues as they arise. These processes, including those built into Reading Corps, 

cover each intervention activity and personnel delivering them. The issues include: 

• Tutors struggling to deliver the literacy activities with fidelity or not meeting expectations for 

their role (such as arriving to school on time, or entering assessment data). When tutors 

struggle with fidelity, internal and master coaches help them improve their service delivery. 

The coaches also follow up with tutors about other Reading Corps activities, such as entering 

data, while UWI personnel address human resources-related issues such as tutor attendance.

• Internal coaches not overseeing tutors with the prescribed frequency. During the follow-up 

interviews, one UWI respondent explained that use of a new report from the intervention data 

system uncovered that some internal coaches were not observing tutors regularly but were 
Scaling insights: United Ways of Iowa’s experience Mathematica

21

waiting and then observing several times in a short window of time. UWI personnel 

discussed the issue with master coaches so they could address the issue. 

• Schools attempting to serve students with levels of reading proficiency too low for Reading 

Corps and who instead need more intensive support. During the October 2018 site visit, one 

respondent said that master coaches usually notice this issue, although UWI personnel and 

tutors have also identified when it occurs. If so, UWI personnel and master coaches follow 

up with schools about serving the appropriate target population. However, as of September 

2019, this was no longer an issue because the intervention developer expanded the eligibility 

criteria to include these students.

• Students not receiving enough of the intended amount of 100 minutes per week. Some of this 

shortfall is unavoidable due to student absences, but it sometimes occurs when teachers do 

not allow the student to leave the class for tutoring. Respondents said this usually subsides 

when teachers see how students were benefiting from the tutoring, and internal coaches 

usually resolve issues with the teachers. In a few cases, UWI personnel had to intervene with 

internal coaches and principals, and had to stop implementing Reading Corps in one school 

when this support was not forthcoming. 

Respondents described a couple of potential monitoring challenges stemming from using

personnel from partners as coaches. Respondents said that, in a few rare cases, the school 

assigned a teacher as the internal coach without first getting the teacher’s input, which was a 

concern because that teacher was less likely to be committed to his or her role of monitoring 

tutoring. One respondent also expressed concern that the AEA-funded master coaches’ preexisting relationships with their schools meant they might place slightly less emphasis on fidelity 

to avoid hurting their relationships. However, this respondent indicated this was not a serious 

problem. This respondent also noted that the pre-existing relationships were a strength because 

the master coach and school personnel can draw on their experience working together. 

The small initial size of Reading Corps in Iowa made it slightly harder for master coaches 

to acclimate to their roles. At the start of implementation in Iowa, each master coach only 

served a few members and did not need to spend much time on intervention activities. Even at 

the current, larger scale, this can still be an issue for the AEA-provided master coaches because

they serve fewer schools. However, respondents indicated this was not a major issue. Monitoring 

and communication processes help master coaches carry out their duties, even when they are 

starting out and have less experience in the role. 
Scaling insights: United Ways of Iowa’s experience Mathematica

22

2. Communication systems

a. Approach to using communication systems 

The Reading Corps intervention defines several communicationrelated activities and processes. Some are part of monitoring, 

such as observations of tutors and monthly master coach 

meetings. Internal coaches’ duties include communicating with 

students’ classroom teachers and parents. Personnel in Iowa 

have additional formal meetings outside of those prescribed by 

the intervention. For example, additional meetings between 

members and coaches occur anywhere from weekly to monthly 

as needed. In addition, personnel from UWI and the lead master 

coach meet monthly to review reports from the Reading Corps 

data system and to discuss issues. More broadly, UWI personnel 

communicate changes in policies or procedures through mass 

emails to members and coaches, and respond to questions from them, especially at the beginning 

of the school year when members and coaches have the most questions. 

Promising practice 

in communication

Personnel supplement 

formal, prescribed 

meetings with frequent 

informal communication 

to ask questions, keep 

processes consistent, 

and maintain 

relationships.

There is also frequent communication between the Iowa site and the developer, although more 

intensively through the lead master coach than the UWI personnel. The developer organizes 

monthly meetings of all of the lead master coaches across Reading Corps sites, and all of the 

program directors across sites. The developer also sends monthly email updates to all state 

partners. Meetings involve presentations by the developer and discussions among participants; 

the topics discussed are based on feedback from participants. The lead master coach is also in 

frequent individual contact with the coaching lead at the developer; individual contact between 

developer and UWI personnel appears to be less frequent. 

In addition to communication involving Iowa site leaders, the developer holds three monthly 

calls involving the Iowa master coaches: (1) one with master coaches across all of Reading 

Corps who have more than two years of experience, (2) one for master coaches across all of 

Reading Corps in their first or second year (at the time of the site visit, only one coach in Iowa 

was in this group), and (3) a supplemental one specifically for master coaches in Iowa.20 Calls 

cover specific issues members encounter when delivering the literacy activities and how master 

coaches can support them. The developer also uses these calls to discuss how to monitor data 

completeness and accuracy and to answer questions from the master coaches. Because the calls 

involve master coaches with different experiences, the content varies across calls. 

Finally, UWI personnel said they would like to organize a periodic internal call with all of the 

Iowa master coaches but are reluctant to add this type of call. This reluctance is because most 

master coaches are AEA employees who are already setting aside some of their time for Reading 

20 The developer does not conduct the third call for every state or other location where Reading Corps is 

implemented, but the developer does so in Iowa because the master coaches are a larger group and the AEA 

coaches each serve fewer schools and get less practice in their role. 
Scaling insights: United Ways of Iowa’s experience Mathematica

23

Corps. UWI personnel would also like to supplement email updates to members and internal 

coaches, for example, by regularly spotlighting members or their summer literacy activities, but 

as of September 2019 had not had time to go beyond communicating essential updates. 

b. Facilitators and challenges involving communication systems

Personnel use informal communication to supplement formal processes. Members and 

internal coaches reported informal communication that is much more frequent than formal 

communication, often occurring daily. Internal and master coaches described frequent 

communication outside of the monthly meetings, 

often by email but also by phone or text message. 

One internal coach explained the value of this 

communication, saying, “Any questions that come 

up, being [able] to email or otherwise contact [the 

master coach] and get a quick response back has 

been an immense help.” Outside of the formal, 

developer-arranged calls, the lead master coach is 

in frequent communication with the master coaches to send updates and reminders and 

informally answer questions as they arise. The program director and program manager from 

UWI communicate constantly. The program director also sends individual emails to master 

coaches to informally check in with them. 

“Any questions that come up, being 

[able] to email or otherwise contact [the 

master coach] and get a quick 

response back has been an immense 

help.”

– Internal coach 

Communication processes between leadership and frontline personnel have been 

challenging. When the intervention in Iowa was smaller, the program director spoke with each 

member weekly. However, the program director explained that since the expansion, “I’m lucky 

to see them twice a year, other than training.” UWI respondents 

also noted that the expansion made it difficult to check in with 

members after their first 30 days, as required for AmeriCorps, 

mainly due to the larger number of members and constraints on 

the capacity of UWI personnel. One respondent added that, 

although mass emails are used to communicate changes, 

members and internal coaches do not always thoroughly read the 

emails and understand the changes. UWI personnel view part of 

the internal coach role as ensuring their members understand 

these changes. However, this has been challenging because 

internal coaches do not think of themselves as supervisors whose 

role includes oversight outside of the formal monitoring 

activities. One UWI respondent noted that because internal 

coaches already communicate regularly with master coaches around Reading Corps, they tend to 

also ask master coaches about administrative issues that should be asked of UWI personnel, and 

that master coaches try to help the internal coach directly. This leads to delays in UWI personnel 

finding out about administrative issues. 

Lesson learned on 

communication

After the intervention in 

Iowa increased in size, 

leadership did not have 

enough time to 

communicate 

individually with the 

larger number of 

frontline personnel.
Scaling insights: United Ways of Iowa’s experience Mathematica

24

Feedback indicated that personnel in Iowa would benefit from having more consistent and 

timely communication from the developer in some cases, to help foster smooth 

implementation. In the time since Reading Corps was first implemented in Iowa, Reading and 

Math, Inc. has shifted from a model with one contact for UWI to one in which different contacts 

are responsible for different aspects of the intervention such as master coaching, data, and 

materials. One respondent said that although the previous model was preferred, the new model 

was understandable and likely reflected the developer’s own attempts to scale its oversight of 

Reading Corps. However, feedback suggested that the new model posed some communication 

challenges. For example, some information from the developer could have been communicated 

more consistently with both UWI personnel and master coaches, and some changes could be 

communicated with more advance notice to allow UWI to provide input or to implement the 

changes as prescribed. 

C. Funding and other resources to support scaling

Providing sufficient and sustainable funding as well as other nonfinancial resources can be 

critical to intervention scaling. According to findings from implementation science research, 

providing adequate resources might be one of the most significant factors influencing 

implementation of an intervention (Wenter et al. 2002). Resources can include a range of 

supports such as funding, physical space, and intervention materials (Klingner et al. 2001, 2003; 

Coolbaugh and Hansel 2000). Organizations might want to ensure the availability of such 

resources well before implementation begins so that they can develop and put into place any 

needed space, equipment, and other supports (Metz and Albers 2014). 

1. Funding for Reading Corps scaling

a. Approach to funding intervention scaling

The primary source of funding for Reading Corps in Iowa is UWI’s CNCS grant. Because CNCS 

requires that its funds cannot cover the entire cost of an intervention, matching funds must pay 

part of the cost of each AmeriCorps member placed in a school. UWI is a statewide association 

of local United Way chapters, and each local chapter has sole authority (within the national 

organization) to fundraise in its geographic area, so UWI and stakeholders planned that local 

chapters would raise the matching funds. UWI is restricted from fundraising in those areas to 

avoid competing for funds with the local chapters. UWI has also pursued fundraising in areas 

without a local chapter, as well as state funding (the largest potential source) and funding from 

other grants. School districts can also provide the matching funding, and in most cases this is 

what occurs.

Reading Corps funding becomes more efficient—the cost per tutor decreases—as its size 

increases, according to one respondent. This is primarily because the fixed costs—mainly those 

involving UWI personnel—can be spread out over a larger number of tutors. However, one 

respondent noted that these efficiencies are limited because most costs also increase 

proportionally when scaling. For example, costs related to master coaches and travel are related 
Scaling insights: United Ways of Iowa’s experience Mathematica

25

to the numbers of tutors and schools. Nevertheless, different approaches to personnel and other 

aspects of implementation can affect these costs; the same respondent noted that UWI was 

considering using more local master coaches in future years, which would reduce the amount of 

travel to schools and save some money. 

b. Facilitators and challenges involving funding for intervention scaling

Partners cover significant portions of the cost of the intervention in Iowa, facilitating 

UWI’s capacity to implement it. Districts and schools cover the costs of the internal coach; the 

coach devotes part of their time to Reading Corps and the school or district continues to pay the 

full salary and benefits of the school employee serving in this role. Schools also provide funding 

needed for the internal coach to attend Reading Corps training. One UWI respondent explained 

that this is a relatively small amount, particularly because UWI structures trainings to limit costs 

such as overnight travel. In Iowa, the AEA whose 

personnel dedicate some of their time to serving as master 

coaches also continues to pay their full salary and benefits. 

Respondents noted that without this in-kind contribution, 

Reading Corps would not have the capacity to serve as 

many schools. As one respondent said of the AEA, “We 

would never be able to do this program without them.” The respondent also noted that some of 

the other AEAs, although not providing their personnel, have encouraged the districts and 

schools in their areas to participate in Reading Corps. 

“We would never be able to do 

this program without [the AEA].” 

– Respondent

Fundraising challenges limit the size of Reading Corps in 

Iowa. Although the planned approach was for local chapters to

raise the matching funding and some chapters have done so 

consistently, other chapters have never provided funding or 

stopped doing so over time. One respondent expressed 

frustration over this situation, noting many areas had local 

donors who would likely provide funding if UWI were not 

restricted from asking. Although UWI is less restricted in 

locations without a local United Way chapter, respondents 

reported that local chapters remain opposed to local fundraising 

even in those areas.21

Lesson learned on 

funding

Limitations on UWI’s 

ability to fundraise have 

meant school districts 

must supply matching 

funding to participate, 

which has limited UWI’s 

ability to serve new 

districts and schools and 

could affect the 

sustainability of Reading 

Corps in some districts 

in Iowa.

Respondents also reported that opportunities for state-level or 

other funding have been limited. After the initial 

implementation of Reading Corps in Iowa, the state legislature 

came close to appropriating funding for a large expansion, but 

ultimately did not include it in the budget. Since then, UWI has been unsuccessful in advocating 

for funding, despite efforts such as inviting the governor and legislators to visit participating 

21 In addition, those areas are already more challenging for fundraising because areas without a local chapter, such 

as the southwestern portion of the state, tend to have weaker economic conditions.
Scaling insights: United Ways of Iowa’s experience Mathematica

26

schools to observe Reading Corps activities and meeting with the director of the Iowa 

Department of Education, which the UWI executive director has led. One respondent noted that 

“there’s simply more statewide resources in Minnesota compared to Iowa,” explaining that in 

Minnesota, Reading Corps receives funding from the state legislature and from the foundation of 

a large corporation headquartered in the state. Respondents also attributed lack of funding to 

changing stakeholder priorities. One respondent noted that Iowa policymakers have shifted the 

focus of state education policy from reading programs to teacher professional development. 

These challenges have limited the scaling of Reading Corps in Iowa because they leave districts 

and schools to supply the matching funding in most cases. Some districts or schools have enough 

money in their budgets or have foundations that can raise the funds, but despite their interest 

many cannot afford to participate at all or with as many tutors as they need, especially without 

more state funding for districts and schools. One UWI respondent added that in some cases, 

schools do not apply to external sources that could pay for their matching funding, often because 

they are not used to doing so or are reluctant to rely on a funding source that might end up being 

short-term. However, in September 2019 this respondent also said that the Iowa Department of 

Education planned to start giving UWI a list of schools that had unspent funds from state Early 

Literacy Implementation (ELI) grants, which could be used to pay for the matching funding. The 

respondent said that with the list, UWI “can go talk to them and say, ‘Look, your third-grade 

reading scores are at 68 percent and you have unspent ELI money...we could start making a dent 

in this.’” 

Respondents perceived that these funding challenges also threaten the sustainability of Reading 

Corps in some districts in Iowa. One UWI respondent said budget pressures facing districts and 

schools could lead more of them to drop out of the intervention, despite their desire to keep 

participating. This respondent added that UWI cannot depend on always receiving CNCS 

funding, citing two risks: (1) that CNCS might not continue funding Reading Corps in Iowa if it 

cannot meet CNCS standards for evidence-based programs using evaluation results and (2) that 

all CNCS funding will be eliminated from the federal budget.22 Another UWI respondent said 

that without CNCS funding, UWI would not have the resources to continue implementing 

Reading Corps. From the participating school districts’ perspective, sustainability would be very 

challenging if districts had to pay for the full cost of the intervention. One district official said 

that sustaining the intervention in the district’s highest-need schools might be possible, but not 

anywhere else. Another official said the district could likely could not afford to continue the 

intervention without external funding. 

22 As of September 2019, the entire CNCS budget had been proposed for elimination from the federal budget 

multiple times, although these proposals had not been implemented at the time.
Scaling insights: United Ways of Iowa’s experience Mathematica

27

2. Other resources

a. Approach to leveraging other resources 

As part of licensing Reading Corps from the developer, each school year UWI purchases the 

physical materials needed for implementation. Reading Corps is a highly structured intervention 

and the developer has created extensive implementation materials, including a program manual, 

calendar, workbook, and templates for meeting agendas and forms. The developer updates 

materials every year. A materials coordinator from the developer works with UWI and 

organizations in other locations to deliver the materials they need. UWI prints out other 

materials, such as training handouts and human resources documents, some of which are made 

available by the developer through a web portal.

Reading Corps is delivered in schools, which provide most of the physical space needed for 

implementation. Most tutors travel around the school and work with children in or near their 

classrooms. Tutors also need a small office space in the school for their work outside of directly 

serving children, primarily entering assessment results into the data system. Office space for the 

UWI personnel managing Reading Corps is paid under the CNCS grant. Most of UWI’s office 

space is only needed because of Reading Corps; UWI’s only current employee not directly

implementing Reading Corps is the executive director, whose salary and office space are funded 

by dues from local United Ways chapters. 

b. Facilitators and challenges to leveraging other resources 

School partners provide many of the additional resources needed for implementation. With 

tutors working in schools, most of Reading Corps’ needs for physical space are addressed. 

Respondents said finding space could be challenging for some schools, although they are able to 

come up with solutions such as finding a spot in the school library. Schools also provide the 

computer for the tutor to use while in the school. Although schools could usually provide one on 

hand, one district official explained, “I’ve had to purchase 

some of those out of Title I funds if the building didn’t 

have one that they thought they could spare.”23 Lesson learned about 

other resources

UWI needed to address 

resource needs not only for 

direct delivery of intervention 

services but also for its 

personnel managing the 

intervention; the latter proved 

to be challenging.

UWI experienced some initial issues with 

infrastructure resources. The computers and 

printer/scanner UWI personnel used during the first few 

years of implementation were older and did not have 

enough capacity to meet their needs. When UWI received 

funding to expand Reading Corps, personnel used the 

larger budget to purchase better equipment. UWI does not 

have its own network server, which it is considering 

purchasing. The program director initially worked out of a public library and added, “I didn’t 

23 The U.S. Department of Education provides Title I grants to schools and districts to improve the academic 

achievement of economically disadvantaged students.
Scaling insights: United Ways of Iowa’s experience Mathematica

28

have office furniture for the first few months, because we weren’t sure we had money.” Some of 

these challenges occurred because UWI is a small organization whose capacity needed to expand 

significantly to implement Reading Corps. One respondent noted that some issues could have 

been avoided with better planning and execution of UWI’s initial Reading Corps budget. UWI 

has moved to a larger, more sufficient space since it began implementing Reading Corps. 

D. Use of data systems and evaluation during scaling

Data systems can be used to track, measure, and store information about program 

implementation. These systems typically include a financial data collection and reporting system 

as well as a management information system to record the processes and outcomes of the 

intervention’s core components. According to findings from implementation science research, 

using data systems is critical to monitoring an intervention’s implementation because these 

systems can alert personnel as to whether changes are necessary to improve the intervention’s 

effectiveness or efficiency (National Implementation Research Network n.d.). 

In addition to drawing on data and data systems to monitor scaling in an ongoing way, 

intervention developers and organizations scaling those interventions can use evaluation to 

assess whether scaled evidence-based interventions are maintaining their intended effects—that 

is, producing the same beneficial outcomes that were produced before scaling (Bangser 2014). 

Furthermore, multiple studies of the same intervention in different scaling contexts can be used 

to generate evidence about whether the intervention can produce similarly positive results while 

being scaled across new or adapted settings and populations. 

1. Data systems

a. Approach to using data systems

Data systems are a critical feature in delivering Reading Corps. Key aspects of providing and 

monitoring services in the intervention, such as the weekly progress assessments, require a data 

system. UWI primarily uses the Reading Corps Data Management System (RCDMS), an 

extensive system built and managed by the developer. RCDMS also plays a key role in other 

implementation processes. For example, it is used to monitor whether tutors have entered their 

assessment data and whether internal coaches have conducted their required observations of 

members. It can also serve as a means for communication, as the tutors and coaches can send 

notes to one another about specific students. In addition, the system has data on tutor attendance, 

tutor caseloads, and student-specific amounts of tutoring received, with codes for reasons why a 

student did not receive tutoring on a particular day. It also contains materials such as Reading 

Corps training documents, video examples of delivering the scripted literacy activities, and 

additional reading passages that tutors can use.

The state requires schools to use formative assessments, which are the same ones Reading Corps

uses, allowing personnel to use data already being collected. Some schools choose to have 

teachers or other school personnel conduct those assessments and report the results to the tutor 
Scaling insights: United Ways of Iowa’s experience Mathematica

29

(who enters them in RCDMS), to avoid administering the same assessment twice to those 

students. Other schools decide to have the tutor conduct the assessments for students 

participating in Reading Corps. As a result, some internal coaches also reported reviewing data 

through the FastBridge assessment system used by Iowa districts and schools. One internal coach 

reported using the FastBridge system more than RCDMS to review data, although another 

internal coach reported using RCDMS more. 

Although data systems are primarily used in Reading Corps to track student progress on reading 

assessments and monitor implementation processes, UWI personnel use other data systems as 

well. They track member hours using a system that many AmeriCorps programs use, to ensure 

members are on pace to reach their required number of service hours. UWI personnel also use a 

data system to track AmeriCorps member recruitment, and an online system that schools can use 

when applying to participate in Reading Corps. 

b. Facilitators and challenges involving use of data systems 

Promising practice 

in data systems

Using software to 

facilitate better reporting 

of data from the data 

system expands UWI’s 

ability to use it for 

monitoring and 

improvement and allows 

more efficient operations 

by reducing the need for 

the developer to generate 

reports.

Respondents reported that the data system is easy to use and 

well supported by the developer. They said RCDMS is helpful 

for implementing the intervention and increases workforce 

efficiency. Respondents described how the developer has made 

minor changes to the system, based on user feedback, that have 

made it easier to use. The developer also has adopted 

commercial software (Microsoft Power BI) to enable better 

reporting of data from RCDMS. Respondents noted that UWI 

personnel previously had to ask developer personnel to generate 

reports for them. With this software in place, UWI personnel can 

produce the reports they need at any time. The developer has 

also set up the software so UWI can generate a greater variety of 

reports, including reporting data on service delivery by tutor, receipt of tutoring by all of the 

students at each school, and more details on internal coaches’ observations of members. 

However, one respondent said it would help “to be able to do more of the things that I know the 

[developer] can do with the data,” such as comparing schools or grade levels. Personnel have 

used the reports to more easily identify and address implementation issues, including those 

described in the monitoring section earlier. 

Respondents described how their frequent and intensive use of the system would uncover 

errors. Although there are no formal procedures to validate data accuracy, master coaches 

explained that because each student receives weekly assessments, a mistakenly entered score 

would stand out compared to other scores. More broadly, the frequent observations of tutors by 

coaches ensures tutors are administering the assessments as intended, meaning the data generated 

are high quality. One master coach summarized the issue as, “it’s really not difficult to know if 

the data is accurate. The struggle can be making sure that the tutors are entering it in a timely 
Scaling insights: United Ways of Iowa’s experience Mathematica

30

manner.” Members are required to enter data weekly, although some said they enter their data at 

the end of each day. 

Implementing Reading Corps without access to the data system was briefly a challenge. 

Before RCDMS, UWI used a system provided by the developer, edSpring, which crashed in 

2015–2016, the first school year after UWI received the larger CNCS grant to expand Reading 

Corps in Iowa. Although the developer accelerated the already-planned rollout of RCDMS in 

response, during a six-week period UWI and frontline personnel were limited to paper forms and 

Microsoft Excel spreadsheets to work with data, which one respondent described as challenging. 

2. Use of evaluation 

a. Approach to using evaluation

Both UWI and Reading and Math, Inc. use evaluation when scaling Reading Corps. UWI is 

required by its current CNCS grant to conduct a formal evaluation. At the time of the site visit in 

October 2018, the plan was to engage an external evaluator to conduct a quasi-experimental 

design study in which students participating in Reading Corps are matched with similar students 

(primarily within participating schools and, only if needed, from other, similar schools) who are 

not participating. UWI planned that, after a test run in the 2018–2019 school year, the evaluator 

would conduct the evaluation during the 2019–2020 school year. As of September 2019, UWI 

was very close to contracting with the evaluator, and the planned time frame for the evaluation 

was still during 2019–2020. 

One UWI respondent expressed excitement for the opportunity from the upcoming evaluation, 

because having Iowa-specific evidence of effectiveness could help increase district participation 

and stakeholder support. This respondent also noted that previously, evidence of effectiveness 

from evaluations had been less important for decisions by districts and schools to participate, 

compared to awareness of the intervention itself and then word of mouth from the districts and 

schools initially implementing it. On the few occasions when needed, UWI cites the evaluation 

that took place in Minnesota (Markovitz et al. 2014). However, another UWI respondent 

mentioned that the decision about which intervention to initially replicate in Iowa had 

emphasized choosing one with evidence of effectiveness. The first respondent thought that an 

evaluation from Iowa would be more effective at increasing interest from districts, especially if 

UWI were to cite evaluation evidence while reaching out to districts that have not yet discussed 

the possibility of participating in Reading Corps. 

More broadly, developer respondents noted that randomized controlled trials had taken place in 

three locations implementing Reading Corps, building the evidence base around the intervention. 

The developer has used those studies to consider changes to the intervention, and developer 

personnel said they would use results from the Iowa evaluation after they became available. The 

developer also helps other organizations with evaluation logistics; in Iowa, the developer 

personnel helped UWI prepare its evaluation proposal. 
Scaling insights: United Ways of Iowa’s experience Mathematica

31

Apart from formal evaluations, the developer conducts extensive improvement activities on 

Reading Corps. Developer personnel described how they examine intervention data and gather 

feedback from the organizations licensing the intervention (including UWI). One UWI 

respondent said tutors, internal coaches, school teachers and principals, and master coaches are 

surveyed annually by the developer, although UWI personnel are not. The developer then 

considers and develops updates, and pilot tests them—usually in Minnesota—before rolling them 

out more broadly. One example is the changes to tutoring for kindergarten students discussed in 

Chapter II: doubling the amount of daily tutoring and using small-group activities to supplement 

one-on-one tutoring. As another example, the overall data have shown that first-grade students 

do not progress as much compared to other students. As a result, the developer is working with a 

team of experts to develop and pilot a new scripted literacy activity that would replace one of the 

existing activities. One developer respondent said, “We’re always looking for ways to improve” 

Reading Corps. 

Most of the developer’s evaluation is conducted across all locations and does not specifically 

focus on Iowa or any other location. Developer personnel said that although they have 

approximate targets for tutor caseload sizes and amounts of tutoring students receive, they do not 

set targets for student growth on assessments because 

growth rates are sensitive to many other factors besides 

Reading Corps. They also do not compare results across 

locations because context can vary widely by location; 

instead, the developer examines trends in each location’s 

results over time. The developer does produce year-end 

reports using Reading Corps data for each location, including Iowa. The report includes data on 

numbers of students served, amounts of tutoring received, fidelity of assessments given and 

observations conducted, and results on the assessments and student growth. The report also lists 

areas of strength and areas for improvement for the organization implementing Reading Corps in 

that location to consider. One UWI respondent said this report was helpful, with “great, great 

graphs” that can be shown to partners and stakeholders. Another UWI respondent said that the 

type of data from these reports, especially from the initial years of implementation in Iowa, had 

helped increase districts’ and schools’ interest in participating and had helped UWI further 

replicate and expand Reading Corps to its current size. 

“We’re always looking for ways to 

improve [Reading Corps].”

– Developer respondent 
Scaling insights: United Ways of Iowa’s experience Mathematica

32

b. Facilitators and challenges involving use of evaluation 

Some evaluation and data use is built into Reading Corps 

and is facilitated using results from the Reading Corps data 

system. On a day-to-day basis, the data system (RCDMS) is 

primarily used by coaches and tutors to record and monitor 

students’ progress on literacy assessments and service receipt. 

However, data also serve to identify issues with tutors and 

schools; data are also helpful for broader purposes such as 

identifying the effectiveness of each of the 10 scripted literacy 

activities. Because frequent assessment is a key part of Reading 

Corps and the intervention uses an extensive data system with 

strong reporting capabilities, personnel can draw on the 

detailed data and reports for evaluation and improvement. 

Promising practice in 

evaluation

Having an extensive data 

system and incorporating 

frequent data collection 

into the intervention 

allows the developer to 

use these data to 

evaluate how the 

intervention is working 

and to consider 

improvements.

UWI faces challenges to conducting Iowa-specific 

evaluations and data use. First, because UWI must implement the intervention with fidelity, it 

is not allowed to make changes to the intervention and test how well those changes work. More 

importantly, UWI personnel have limited time and resources to conduct evaluations or use data 

outside of prescribed monitoring processes. UWI personnel described a couple of initiatives they 

would like to explore if they had more time. For example, they would like to generate new 

school-level and grade-level data reports that would enable them to consider targeted 

improvements. One respondent also mentioned using data more to help recruit districts and 

schools by showing administrators and principals the benefits for students. 
Scaling insights: United Ways of Iowa’s experience Mathematica

33

IV. SUMMARY OF KEY FINDINGS 

In Iowa, UWI is scaling an evidence-based literacy tutoring intervention—Reading Corps—by 

both replicating it in new locations and expanding it to serve more schools in existing districts. 

While scaling Reading Corps, UWI and its partners and stakeholders have drawn upon supports 

such as organizational workforces, implementation monitoring and communication systems, 

funding and other resources, and data systems and evaluation to varying degrees. In this chapter, 

we highlight key findings in light of what we learned about the scaling of Reading Corps by 

UWI. 

• A highly structured intervention facilitates replication and expansion. A key facilitator 

of UWI’s implementation of Reading Corps in Iowa is the extent to which the intervention is 

highly structured. Reading Corps has a layered personnel approach where each role—

particularly tutors, internal coaches, and master coaches—is well defined and focused on 

specific aspects of implementation, including activities and assessments that are detailed and 

scripted. The intervention has strong training that prepares personnel to carry out their roles, 

and each role relates to other roles through a structure of monitoring and support. 

Implementation monitoring and communication systems allow personnel to identify and 

address issues as they arise, and the data system is central for service delivery, monitoring, 

and evaluation. The developer also provides extensive assistance, ranging from training and 

technical assistance to infrastructural elements such as the data system and materials. 

• Replication and fidelity requirements facilitate the implementation of an effective 

intervention but limit consideration of changes and improvements. The developer 

requires UWI and other implementing organizations to replicate Reading Corps with fidelity 

and not to adapt it. These requirements are important to ensure that interventions that have 

shown evidence of effectiveness can continue to have those effects in new settings. 

Respondents consistently expressed the importance of fidelity and their commitment to 

implementing the intervention as designed. However, these restrictions also limit UWI’s 

ability to consider local changes and improvements to Reading Corps. For example, such 

restrictions affect UWI’s use of Iowa data and evaluations to consider intervention 

improvements. The developer has a process to update Reading Corps, although it is for the 

intervention as a whole and only considers site-specific circumstances and experiences to a 

limited extent. 

• Communication and collaboration help facilitate scaling, although ensuring 

communication remains strong after scaling can be a challenge. As with other 

implementation supports, many formal communication processes are built into Reading 

Corps. Reading Corps has many different personnel roles—members, internal coaches, 

master coaches, and the UWI workforce primarily, plus district and school administrators and 

developer personnel. Personnel in these roles work for several different organizations and 

many rarely meet in person. They need to coordinate, ask and answer questions, and monitor 

and support each other based on how their roles are structured relative to each other within 
Scaling insights: United Ways of Iowa’s experience Mathematica

34

the intervention. This became especially important when the intervention became larger and 

fewer opportunities existed for more personal forms of communication between some types 

of personnel. In addition to the extensive formal communication—much of it built into the 

intervention or arranged by the developer—personnel engage in frequent informal 

communication and collaboration. Ensuring communication remains responsive and 

consistent after scaling has proved somewhat challenging. For example, personnel might not 

communicate consistent messages to others or might not communicate needed information to 

the right people. Despite these challenges, respondents said support from people in other 

roles was very helpful, and that others were committed and bought into the goals of the 

intervention. 

• Limited personnel size and organizational infrastructure have been a key challenge.

Before implementing Reading Corps, UWI was a one-person organization and found it 

challenging to increase its capacity to meet the demands of implementation, especially after 

expanding to a much larger scale after the first two years of implementation in Iowa. UWI 

personnel had to deal with basic infrastructural issues such as obtaining or upgrading 

sufficient computers, printers, and office space for themselves. The developer’s supports 

have helped in many areas. Still, the UWI intervention workforce would have benefited from 

having others in the organization to draw on for support, such as human resources personnel 

to advise on recruiting and hiring members. The UWI personnel who led the effort to scale 

Reading Corps starting with the 2015–2016 school year reported having difficulty meeting 

the intensive workload involved in administering the intervention. Some of these issues 

would be less pressing if UWI had budgeted to have another employee, but this would have 

increased the matching funding amount needed to place a tutor in a school, making it even 

more challenging for districts and schools to participate. 

• Stakeholder and partner support has contributed to some of the intervention’s greatest 

strengths and challenges. Because UWI’s capacity has been limited, support from 

stakeholders and partners has been crucial. The design of Reading Corps requires 

considerable involvement from participating schools; this in-kind support bolsters the 

intervention and encourages school commitment but is also challenging for several reasons. 

First, schools usually have to find funding for part of the cost of the intervention. Second, 

school personnel have to balance their intervention activities with the rest of their educational 

mission. Third, some personnel involved in implementing the intervention are not subject to 

the authority of the grantee. In Iowa, an AEA’s provision of master coaches has had similar 

benefits and challenges. Support from local United Ways chapters was also critical for initial 

implementation in Iowa but has dropped off since then, and lack of state funding and United 

Way-imposed limits on UWI’s ability to fundraise have made scaling very challenging. 

Finally, UWI’s implementation of Reading Corps can influence the surrounding context as 

word spreads between schools and districts about their intervention experiences. 

Occasionally this is problematic when a school has a member who performs poorly, but this 

is largely positive because most schools have positive experiences and teachers see the 

benefits of the tutoring. 
Scaling insights: United Ways of Iowa’s experience Mathematica

35

UWI’s scaling of Reading Corps reveals both successes and challenges in replicating and 

expanding an evidence-based intervention. This case study report, along with two companion 

case study reports about other CNCS-funded grantees, was intended to illustrate the various 

experiences that organizations attempting scaling might face (Anderson et al. 2020; Eddins et al. 

2020). Using an implementation science lens, this report sought to help stakeholders understand 

the factors that might facilitate and hinder scaling, and the lessons learned by one particular 

grantee scaling an evidence-based intervention. 
This page intentionally blank for double-sided copying.
Scaling insights: United Ways of Iowa’s experience Mathematica

37

REFERENCES 

Anderson, Mary Anne, Katie Eddins, Karen Needels, and Scott Richman. “Scaling the Home 

Instruction for Parents of Preschool Youngsters (HIPPY) intervention: Insights from the 

experiences of Parent Possible.” Report submitted to the Corporation for National and 

Community Service. Washington, DC: Mathematica, June 2020. 

Bangser, Michael. “A Funder's Guide to Using Evidence of Program Effectiveness in Scale-Up 

Decisions.” New York: MDRC and the Growth Philanthropy Network, 2014. 

Bernfeld, G. A. “The Struggle for Treatment Integrity in a ‘Dis-integrated’ Service Delivery 

System.” The Behavior Analyst Today, vol. 7, no. 2, 2006, pp. 188–205. 

doi:10.1037/h0100086. 

Blase, Karen, and Dean Fixsen. “Core Intervention Components: Identifying and 

Operationalizing What Makes Programs Work.” Washington, DC: Office of the Assistant 

Secretary for Planning and Evaluation, Office of Human Services Policy, U.S. Department 

of Health and Human Services, February 2013. 

Breitenstein, S. M., D. Gross, C. Garvey, C. Hill, L. Fogg, and B. Resnick. “Implementation 

Fidelity in Community-Based Interventions.” Research in Nursing & Health, vol. 33, no. 2, 

2010, pp. 164–173. doi:10.1002/nur.20373. 

Coolbaugh, Kathleen, and Cynthia Hansel. “The Comprehensive Strategy: Lessons Learned from 

the Pilot Sites.” OJJDP Juvenile Justice Bulletin. Washington, DC: U.S. Department of 

Justice, Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention, 

March 2000. 

Corporation for National and Community Service. “AmeriCorps State and National Policy 

Frequently Asked Questions (FAQs).” n.d. Available at

https://www.nationalservice.gov/sites/default/files/documents/AmeriCorps_State_National_

Policy_FAQs.pdf. Accessed September 5, 2019. 

Corporation for National and Community Service. “Metadata Glossary.” 2016. Available at

https://www.nationalservice.gov/sites/default/files/

documents/GLOSSARY-Metadata%20Categories%20Definitions%20for% 

20Publication.pdf. Accessed July 31, 2019. 

Eddins, Katie, Mary Anne Anderson, Karen Needels, and Scott Richman. “Scaling the Birth and 

Beyond (B&B) intervention: Insights from the experiences of the Child Abuse Prevention 

Council (CAPC).” Report submitted to the Corporation for National and Community 

Service. Washington, DC: Mathematica, June 2020. 

Fagan, A. A., K. Hanson, J.D. Hawkins, and M. W. Arthur. “Bridging Science to Practice: 

Achieving Prevention Program Implementation Fidelity in the Community Youth 

Development Study.” American Journal of Community Psychology, vol. 41, no. 3-4, June 

2008, pp. 235–249. 
Scaling insights: United Ways of Iowa’s experience Mathematica

38

Fagan, A., and S. Mihalic. “Strategies for Enhancing the Adoption of School-Based Prevention 

Programs: Lessons Learned from the Blueprints for Violence Prevention Replications of the 

Life Skills Training Program.” Journal of Community Psychology, vol. 31, no. 3, 2003, pp. 

235–253. 

Fixsen, D. L., K. Blase, K., A. Metz, and M. Van Dyke. “Statewide Implementation of EvidenceBased Programs.” Exceptional Children, vol. 79, no. 2, 2013, pp. 213–230. 

Fixsen, Dean, Sandra Naoom, Karen Blase, Robert Friedman, and Frances Wallace. 

“Implementation Research: A Synthesis of the Literature.” Tampa, FL: University of South 

Florida, 2005. 

Franks, Robert, and Jennifer Schroeder. “Implementation Science: What Do We Know and 

Where Do We Go from Here?” In Applying Implementation Science in Early Childhood 

Programs and Systems, edited by Tamara Halle, Allison Metz, and Ivelisse Martinez-Beck. 

Baltimore, MD: Paul H. Brookes Publishing Company, 2013. 

Klingner, J., S. Ahwee, P. Pilonieta, and R. Menendez. “Barriers and Facilitators in Scaling Up 

Research-Based Practices.” Exceptional Children, vol. 69, no. 4, 2003, pp. 411–429. 

Klingner, J. K., M. E. Arguelles, M. T. Hughes, and S. Vaughn. “Examining the Schoolwide 

‘Spread’ of Research-Based Practices.” Learning Disability Quarterly, vol. 24, no. 4, 2001, 

pp. 2221–2234. 

Markovitz, C., M. Hernandez, E. Hedberg, and B. Silberglitt. “Impact Evaluation of the 

Minnesota Reading Corps K-3 Program.” Chicago, IL: NORC at the University of Chicago, 

2014. 

Maxwell, N., and S.B. Richman. “Scaling Programs with Research Evidence and Effectiveness 

(SPREE).” The Foundation Review, vol. 11, no. 3, 2019, pp. 55-67. 

Metz, A., and B. Albers. “What Does It Take? How Federal Initiatives Can Support the 

Implementation of Evidence-Based Programs to Improve Outcomes for Adolescents.” 

Journal of Adolescent Health, vol. 54, no. 3, supplement, March 2014, pp. S92–S96. 

Mihalic, S., and K. Irwin. “Blueprints for Violence Prevention: From Research to Real-World 

Settings—Factors Influencing the Successful Replication of Model Programs.” Youth 

Violence and Juvenile Justice, vol. 1, no. 1, January 2003, pp. 1–23. 

National Implementation Research Network. “Active Implementation Science and Practice.” 

Chapel Hill, NC: National Implementation Research Network, October 2016. Available at 

https://nirn.fpg.unc.edu/resources/brief-1-active-implementation-practice-and-science. 

Accessed November 14, 2019. 

National Implementation Research Network. (n.d.). “Implementation Drivers.” Available at 

https://nirn.fpg.unc.edu/module-2. Accessed September 19, 2019. 

Necoechea, Denise Marie. “Children At-Risk for Poor School Readiness: The Effect of an Early 

Intervention Home Visiting Program on Children and Parents.” Riverside, CA: University of 

California, Riverside, 2007. 
Scaling insights: United Ways of Iowa’s experience Mathematica

39

Needels, Karen, Daniel Friend, Scott Richman, Katie Eddins, and Christopher Jones. “Agency 

Guide for: Scaling Evidence-Based Interventions: Insights from the Experiences of Three 

Grantees.” Report submitted to the Corporation for National and Community Service. 

Oakland, CA: Mathematica, June 2020. 

Nord, W. R., and S. Tucker. Implementing Routine and Radical Innovations. Lexington, MA: D. 

C. Heath and Company, 1987.

Wenter, D. L., S. T. Ennett, K. M. Ribisl, A. A. Vincus, L. Rohrbach, C. L. Ringwalt, et al. 

“Comprehensiveness of Substance Use Prevention Programs in U.S. Middle Schools.” 

Journal of Adolescent Health, vol. 30, 2002, pp. 455–446. 
This page intentionally blank for double-sided copying.
Scaling insights: United Ways of Iowa’s experience Mathematica

41

APPENDIX: PROCESS STUDY METHODOLOGY 

A. Overview of the process study 

The Corporation for National and Community Service (CNCS) contracted with Mathematica in 

2016 to conduct the Scaling Evidence-Based Models (SEBM) project. The project is intended to 

deepen the agency’s understanding of evidence-based interventions and its knowledge base on 

scaling them. Through the SEBM project, Mathematica has (1) reviewed and evaluated research 

evidence on the effectiveness of AmeriCorps interventions that were funded in 2015 and 2016 

and Social Innovation Fund (SIF) interventions that were funded in 2010 and 2011,24 (2) 

assessed grantees’ plans for scaling, and (3) evaluated the readiness for scaling of CNCS-funded 

interventions that showed research evidence of effectiveness and recommended for further study 

the grantees and interventions that showed evidence of readiness for scaling. 

As part of the SEBM project, Mathematica conducted a process study examining how three 

CNCS-funded grantees implementing interventions that Mathematica assessed to have evidence 

of effectiveness and to be ready to scale actually scaled their interventions. This process study 

was structured to identify the types of scaling that grantees undertook and describe how grantees 

drew upon organizational and implementation supports to facilitate scaling. This report presents 

a case study of one of three grantees included in the process study: United Ways of Iowa (UWI) 

implementing the Reading Corps intervention in Iowa. 

This appendix details the scaling definitions used for the project and the scaling readiness 

framework that informed grantee selection (Section B); the methods used to conduct the process 

study, including the grantee selection process (Section C); and the methodologies used to collect 

and analyze data for the process study (Section D). 

B. Defining scaling and the SEBM scaling readiness framework

To better understand how funders like CNCS and other stakeholders can foster the scaling of 

evidence-based interventions, Mathematica first operationalized the concept of scaling by 

identifying three types of scaling that can be pursued:25

24 AmeriCorps supports a wide range of local service programs through grants to address critical community needs, 

such as those pertaining to education, public safety, health, and the environment. SIF supported programs from 

2010 through 2016. SIF grants were used to fund community-based programs to address challenging social 

problems communities face in the areas of economic opportunity, healthy futures, and youth development. CNCS 

(2016, n.d.) provides a detailed description of these CNCS programs.

25 Mathematica adapted these definitions from Fixsen et al. (2005), a synthesis of implementation research 

published by the National Implementation Research Network. For example, Fixsen et al. (2005) defines 

“adaptation of the program” as “modifications that are made in a program to accommodate the context and 

requirements at an implementation site” and defines “replication” is defined as the implementation of an 

intervention in new sites. Given these existing definitions, Mathematica defined “expansion” as the 

implementation of an intervention in the same site, with the same population, but serving more people. 
Scaling insights: United Ways of Iowa’s experience Mathematica

42

• Expansion extends the intervention to more people in the same target population in the same 

location. It requires that the intervention and the organization serve a larger number of 

participants with the same service quality and in a consistent manner with the intervention’s 

design. An example of expansion would be increasing the number of unemployed adults 

served at a work center by hiring five more job search specialists who will each serve 20 

more adults. 

• Replication extends the intervention for the same target population to a new location. It 

requires the intervention and the organization maintain service quality and fidelity to the 

intervention in the new location. An example of replication would be implementing a reading 

program designed for 5th graders in a new school district, city, and state, but serving the 

same target population of 5th graders. 

• Adaptation extends the intervention to a new target population. It requires that the 

organization adapt the intervention in a way that maintains service quality. An example of 

adaptation would be modifying a parent training curriculum designed for mothers to include 

language that is more inclusive of fathers. 

Scaling is considered to be successful when the intervention (1) is replicated, expanded, and/or 

adapted, and (2) maintains or surpasses its beneficial impacts for participants after the scaling 

has occurred. Drawing on these definitions as well as research from implementation science, 

Mathematica then developed for the SEBM project a framework that identifies five conditions 

that indicate whether an intervention and the organization implementing it are ready for scaling 

(Exhibit A.1). For example, the framework specifies that an intervention might be ready for 

scaling if it is well specified. In the implementation science literature, this means that the core 

elements, or set of activities that is critical for achieving beneficial outcomes for the 

intervention’s participants, are made clear and that for each core intervention element, a 

description exists of the dimensions necessary to produce the intended outcomes (Blase and 

Fixsen 2013). (A comprehensive synthesis of the implementation science literature that supports 

the scaling readiness framework is available in Maxwell and Richman 2019). 
Scaling insights: United Ways of Iowa’s experience Mathematica

43

Exhibit A.1. Scaling readiness framework developed under the SEBM project

The first three conditions indicate whether an intervention might be ready to be scaled: 

• A well-specified intervention, consisting of a description of the content, mode of service 

delivery, intensity, workforce needs,26 and setting for each core element. A well-specified 

intervention also includes a definition of participation in and completion of the intervention. 

• A well-defined target population, consisting of a description of the population for which 

the intervention was found to be effective. 

• Implementation supports, consisting of a description of supports that can help ensure 

fidelity, such as an implementation monitoring team and performance benchmarks. 

Implementation supports also include a description of the procedures for putting the supports 

into action, such as describing the processes the monitoring team follows and a process for 

measuring performance benchmarks. 

The final two conditions indicate that an organization might be ready to scale an intervention: 

• Enabling context, consisting of a description of the presence of organizational and partner 

agency leadership and culture that supports the scaling effort. Enabling context is 

demonstrated with examples of ways that the organization is innovative and has improved 

upon past interventions, particularly in the face of implementation challenges. 

26 We use the terms personnel and workforce to refer to organization and partner personnel as well as AmeriCorps 

members who deliver intervention services.
Scaling insights: United Ways of Iowa’s experience Mathematica

44

• Implementation infrastructure, consisting of a description of the organizational 

infrastructure, such as the workforce, materials, and physical space that support 

implementation.

The scaling framework was used to inform selection of the three grantees that participated in the 

process study, because CNCS and Mathematica sought to include interventions with 

implementing grantees that appeared to be ready to scale. We also collected data from the 

grantees included the process study using questions that were informed by the framework. This 

helped us understand whether the requirements for readiness for scaling were indeed present and 

sustained during implementation of each intervention. 

C. Grantee selection for the process study

Mathematica and CNCS used a multistage process to select the interventions and the grantees 

scaling them for inclusion in the process study. In the first stage, Mathematica reviewed the 

evaluation studies that grantees submitted to demonstrate evidence of their intervention’s 

effectiveness and grantees’ plans for scaling those interventions. Grantees submitted these 

documents to CNCS in 2015 and 2016 for AmeriCorps grantees and in 2010 and 2011 for SIF 

grantees. Mathematica used those documents to identify 17 interventions that CNCS grantees 

were scaling that demonstrated evidence of effectiveness. Mathematica identified these 

interventions by assessing whether the evaluation studies used rigorous research designs and had 

consistently favorable findings, and whether the intervention upon which the evidence was based

aligned with the proposed plans for the intervention during scaling. 

In the second stage, Mathematica developed and applied a scoring system to rank the 

interventions, and the grantee(s) scaling them, according to their readiness to scale. The scoring 

system used condition-level scores to operationalize each of the five conditions in the scaling 

readiness framework (see Section B). Mathematica identified eight interventions, associated with 

10 grantees27 that had relatively high scores and represented a mix of scaling types and 

intervention focus areas.28 

In the third stage, CNCS staff in the Office of Research and Evaluation (ORE) conducted 

outreach and telephone screening interviews to learn more information about the eight 

interventions and 10 grantees. ORE staff reached out to the CNCS program officers, AmeriCorps 

State Commission administrative staff, and AmeriCorps National Direct staff who oversaw or 

interacted with each of the 10 identified grantees to understand any grant management issues or 

grantee capacity concerns that might preclude them from participating in the process study. 

Then, ORE staff contacted administrative personnel from the grantees via telephone, using a 

protocol to collect information on the grantees’ reported progress toward their intervention 

scaling objectives, barriers and successes they had encountered when scaling, and their interest 

27 One recommended intervention was being scaled by three different grantees; other recommended interventions 

were being scaled by one grantee each.

28 Intervention focus areas are the topics in which CNCS concentrates its funding: disaster services; economic 

opportunity; education; environmental stewardship; healthy futures (including physical and mental health, 

substance abuse, and nutrition); and veterans and military families.
Scaling insights: United Ways of Iowa’s experience Mathematica

45

and ability to participate in the process study. One of the 10 grantees asked not to be included in 

the process study, indicating that it could not fulfill the necessary data collection activities 

associated with the process study. 

In the final stage, Mathematica used the information CNCS collected to develop criteria to 

identify three grantees as candidates for the process study. The selection criteria included grantee

size and project age, geographic location, intervention focus areas, types of scaling, reported 

successes and challenges with their scaling experiences, and reported efforts to date to codify 

lessons learned. After applying these criteria to the remaining nine grantees, Mathematica 

recommended three grantees, each implementing a different intervention, as candidates for the 

process study. Based on Mathematica’s recommendations and application of the criteria, CNCS 

ultimately selected three grantees to include in the process study, all of which agreed to 

participate in the study.

29 

At the time of selection into the process study, the grantees varied in the extent to which they 

appeared to fully operationalize the conditions of the scaling framework. The selected sample 

included grantees that had reported, during their initial screening interviews with CNCS, both 

successes and challenges in their scaling execution processes, resource planning issues, 

successes and difficulties generating community support, grant management concerns, and 

successful, mixed, and poor results on various aspects of scaling. Because CNCS did not select 

the grantees at random, and they were not representative of all CNCS grantees, the insights from 

the experiences of the three process study grantees and their partners cannot be interpreted as 

applicable to a broader set of CNCS-funded grantees or service providers. However, the 

grantees—and the interventions they were scaling—were considered to be some of the strongest 

in terms of readiness to scale, and ranged in features such as geographic location, intervention 

focus areas, types of scaling, and the length of time they had been scaling their interventions. 

Because of their scaling readiness strength and range of experiences, the findings from the 

process study allow us to draw lessons learned and illustrative practices that can help 

stakeholders understand the conditions that might facilitate intervention scaling. 

D. Methods for collecting and analyzing data for the process study

1. Methods for collecting data for the process study

Mathematica staff collected data from all three grantees selected for the process study during 

summer and fall 2018 and fall 2019. In summer 2018, Mathematica held pre-visit telephone calls 

with grantee personnel from all three grantees and reviewed grantees’ program documents. 

Mathematica staff then conducted two-day visits during October 2018 to each grantee as well as 

any partner organizations involved in scaling activities, and brief follow-up telephone interviews 

after each visit. In September 2019, Mathematica staff conducted 12-month follow-up telephone 

interviews with grantee personnel. 

29 One of the grantees ultimately selected for inclusion in the process study differed from Mathematica’s 

recommendations. Based on its internal conversations, CNCS selected this grantee in light of its own research and 

funding priorities.
Scaling insights: United Ways of Iowa’s experience Mathematica

46

During the visits to each grantee, members of the process study team conducted one-on-one or 

small group interviews with the following types of personnel: (1) the program manager who 

oversaw implementation of the intervention being scaled; (2) grantee executives, such as the 

executive director and deputy directors; (3) data managers from grantees; (4) any other personnel 

from grantees or partners directly involved in supporting scaling activities, including supervisors 

and managers of frontline personnel; (5) frontline personnel, including AmeriCorps members, 

who directly delivered the intervention being scaled; and (6) for this grantee and one of the other 

two grantees visited, personnel from the developer of the intervention being scaled. (The third 

grantee visited was scaling an intervention it developed itself.) 

Both individual and group interviews during the process study visits generally lasted 30 minutes 

to two hours. The study team held the most comprehensive interviews with the program manager

for each grantee. These interviews covered all topics related to the five conditions in the scaling 

framework (that is, the presence of a well-specified intervention, a well-defined target 

population, implementation supports, an enabling context, and an implementation infrastructure); 

the type of scaling conducted by the grantee and its partners; and the factors that appeared to 

facilitate and challenge implementation and scaling. Interviews with other types of respondents 

were more limited in scope. During interviews with grantee executives, we focused on topics 

related to planning and funding for scaling, and the use of evidence of intervention effectiveness 

in planning scaling efforts. When speaking with other types of grantee personnel, we focused on 

topics related to their specific function—for example, we concentrated on data systems and 

evaluation efforts when interviewing data managers. When interviewing personnel more closely 

aligned to frontline operations, in both individual and group settings, we focused on topics 

related to direct service provision, implementation supports (such as training, communication 

systems, data systems, and implementation and performance monitoring), use of evaluation, and 

other factors that might facilitate or hinder scaling. When interviewing intervention developers, 

we focused on topics related to implementation supports, evaluation efforts, plans to innovate or 

improve the intervention, scaling efforts beyond the specific grantee visited, and other factors 

that might facilitate or hinder scaling.

The study team collected additional data from the grantees during the follow-up telephone 

interviews held in September 2019. The study team held these interviews with the grantee 

program directors and executives. Across the grantees, these interviews focused on changes that 

had occurred in scaling since the visit in October 2018, any successes or challenges associated 

with scaling that the grantees experienced, and clarifications about information collected during 

the October 2018 visits. 

2. Information used for the analysis of UWI’s scaling of Reading Corps 

We based the analysis for this report on information collected from UWI and its partners, as 

summarized in the previous section. First, study team members reviewed documents that could 

shed light on how UWI executed scaling of the Reading Corps intervention and the supports the 

grantee had in place to scale it. Examples of such documents include personnel manuals and 
Scaling insights: United Ways of Iowa’s experience Mathematica

47

intervention materials, such as calendars, literacy activity scripts, and assessment instructions. 

The study team summarized the contents of these documents in a detailed write-up that also 

included notes from the data collection that took place during the process study visit (discussed 

below). 

The study team then visited the grantee in October 2018. During and shortly after the site visit, 

we conducted one-on-one or small group interviews with the following personnel: (1) the UWI

program director who oversaw implementation of Reading Corps; (2) the UWI program manager 

who assisted with implementation; (3) a former UWI program manager who had left the 

organization; (4) the UWI executive director; (5) the UWI fiscal specialist; (6) four of the 

AmeriCorps members who served as tutors; (7) three of the teachers or other school personnel 

who served as internal coaches; (8) three of the principals or administrators from schools and 

districts participating in Reading Corps; (9) two of the master coaches, one of whom is also the 

lead master coach; and (10) three personnel from Reading and Math, Inc., including the director 

of national replication, master coaching lead, and data and evaluation manager. We did not visit 

UWI’s offices or any schools; instead, we interviewed personnel at an off-site location or by 

phone. The tutors, internal coaches, principals and administrators, and master coaches we 

interviewed had been asked to participate by the program director. The program director avoided 

asking personnel in their first year of involvement with Reading Corps because they would only 

have had a couple of months of intervention experience to draw on. In total, we interviewed 20 

respondents during and shortly after the visit.30 Exhibit A.2 details the characteristics of the 20 

interview respondents. 

Exhibit A.2. Characteristics of those interviewed 

Respondent characteristic Response category

Number of 

respondents

Type of positiona Grantee program manager 1 

Grantee executive 1 

Other grantee personnel 3 

Partner administrator 3 

Frontline supervisor 5 

Frontline personnel 4 

Developer personnel 3 

Type of personnel AmeriCorps member 4 

Paid organizational personnel (from the 

grantee, the developer, or local partners) 16

30 The interview with the program director took place over multiple blocks during the site visit, with the combined 

total significantly exceeding two hours. The program director also attended the other interviews (other than with 

the intervention developer) but did not actively engage in the interviews. Three interviews (the master coaches, 

former program manager, and intervention developer) took place by phone shortly after the site visit, in November 

2018.
Scaling insights: United Ways of Iowa’s experience Mathematica

48

Respondent characteristic Response category

Number of 

respondents

Experience in current position Fewer than 12 months 1 

1 to 2 years 4 

3 to 5 years 7 

More than 5 years 8 

Experience with organization Fewer than 12 months 0 

1 to 2 years 4 

3 to 5 years 9 

More than 5 years 7 

Experience in the same type of work Fewer than 12 months 0 

1 to 2 years 3 

3 to 5 years 6 

More than 5 years 11

Highest level of education Less than high school degree 0 

High school degree (including equivalency) 2 

Some college, no degree 1 

Associate’s degree 0 

Bachelor’s degree 8 

Master’s degree or above 9 

Gender Female 16

Male 4 

Race/ethnicity White 20

All other 0 

a For this characteristic, we categorized interview respondents as follows: the UWI program director as 

grantee program manager; the UWI executive director as grantee executive; the current and former UWI 

program managers and the fiscal specialist as other grantee personnel; the principals and district 

administrators as partner administrators; the two master coaches and three internal coaches as frontline 

supervisors; and the tutors as frontline personnel. 

Finally, the study team conducted follow-up telephone interviews with UWI personnel in 

September and early October 2019. The study team held a three-hour interview (split into two 

parts) with the UWI program director, and one 30-minute interview with the UWI executive 

director.

3. Analysis of collected data 

We ensured accuracy and thoroughness of data collection in the following ways: (1) preparing 

standardized protocols that were tailored to each respondent type and to the intervention and 

grantee; (2) having both a senior and junior researcher conduct the visits and telephone 
Scaling insights: United Ways of Iowa’s experience Mathematica

49

interviews so that one team member could take notes while the other conducted interviews; (3) 

audio recording interviews and taking detailed, near-verbatim notes during interviews; and (4)

having multiple study team members review and provide feedback on the interview notes as well 

as ask for clarifications on content when necessary and appropriate. 

After finalizing the site visit interview notes, study team members synthesized those notes into a 

detailed write-up based on a standardized template. The template grouped information according 

to (1) scaling readiness conditions; (2) the contextual factors that affect implementation and 

scaling; (3) the accomplishments, challenges, successes, and facilitators of scaling; and (4) the 

sustainability of scaling efforts. The write-up also included a checklist that summarized the 

intervention’s readiness for scaling in each of the five conditions of the scaling framework. 

Members of the study team reviewed these write-ups for completeness, thoroughness, and 

accuracy. Before visiting, the research team also developed detailed descriptions of each 

intervention and each grantee’s scaling of it and asked the respective program managers to 

review and correct the description if needed. Study team members synthesized the follow-up 

telephone interview notes by organizing the notes by respondent and by the topics that aligned 

with the chapters and sub-sections of each of the case study reports. 

To conduct the analysis across all three grantees, the study team reviewed the grantee-specific 

write-ups to synthesize data according to the data collection topics of interest. (These topics 

related to the research questions and the ways that grantees were approaching aspects of the five 

conditions that indicate scaling readiness.) Because the study team conducted this analysis before 

the September 2019 follow-up telephone interviews, it and the resulting report (Needels et al. 

2020) were based on a more limited set of information than the analysis for the case study 

reports. 

For this report, the authors assessed the topic-specific information relevant to only UWI from the 

analyses conducted with the data from the site visit and telephone interviews. The authors used 

these data to identify insights and takeaway conclusions that have the potential to be broadly 

applicable as CNCS seeks to support its grantees in their scaling efforts. Quotes from interview 

respondents also provided illustrative insights. A similar approach was used to analyze data for 

the two companion reports (Anderson et al. 2020; Eddins et al. 2020), each of which provides indepth insights about scaling using data from the two other grantees in the process study.
This page intentionally blank for double-sided copying.
This page intentionally blank for double-sided copying.
Mathematica

Princeton, NJ • Ann Arbor, MI • Cambridge, MA 

Chicago, IL • Oakland, CA • Seattle, WA

Tucson, AZ • Woodlawn, MD • Washington, DC 

EDI Global, a Mathematica Company

Bukoba, Tanzania • High Wycombe, United Kingdom

mathematica.org