Anchor

Scaling Evidence-Based Interventions: 

Insights from the Experiences of Three

Grantees

June 30, 2020

Karen Needels, Daniel Friend, Scott Richman, Katie Eddins, and Christopher Jones 

Submitted to:

Corporation for National and Community Service

Office of Research and Evaluation

250 E Street SW

Washington, DC 20525

Project Officer: Anthony Nerino

Contract Number: GS10F0050L/CNSHQ16F0049

Submitted by:

Mathematica 

111 E. Wacker Drive Suite 3000

Chicago, IL 60601

Telephone: (312) 994-1002

Facsimile: (312) 994-1003

Project Director: Scott Richman

Reference Number: 50332
This page has been left blank for double-sided copying.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

iii

ACKNOWLEDGMENTS

The authors would like to acknowledge several people who contributed to and supported this 

report funded by the Corporation for National and Community Service (CNCS) under contract 

GS10F0050L/CNSHQ16F0049. At CNCS, Anthony Nerino, Lily Zandniapour, Roshni Menon, 

and Mary Hyde provided advice and direction that greatly enhanced our ability to make this 

report usable. We appreciate their thoughtful insights into CNCS programs and their comments 

on our products. At Mathematica, Diane Herz provided detailed feedback that greatly enhanced 

the report. Dale Anderson provided editorial assistance and Sharon Clark and Colleen Fitts 

helped prepare it.
This page has been left blank for double-sided copying.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

v 

CONTENTS

EXECUTIVE SUMMARY.....................................................................................................ix

A. Overview of scaling and components necessary for successful scaling ........................2

B. Overview of the process study ......................................................................................3

1. Process study research questions..........................................................................3

2. Selection of the three grantees included in the process study ................................4

3. Overview of the three process study grantees and their interventions ....................4

4. Data collection and cross-site analysis methods ....................................................6

C. Ways that organizations define and operationalize scaling............................................6

D. Ways that the organizations scale successful interventions ..........................................9

1. Implementation infrastructure needed to successfully scale interventions ..............9

2. Activities and considerations to ensure fidelity to the intervention when scaling ...15

3. Organizational factors that facilitate or hinder scaling...........................................19

4. The role that evaluation and data (process, outcome, or impact data) play in 

scaling efforts.......................................................................................................20

References ...............................................................................................................................22

APPENDIX A DETAILS OF THE PROCESS STUDY GRANTEES, THEIR PARTNERS, AND 

THEIR INTERVENTIONS ..........................................................................................................................A.1
This page has been left blank for double-sided copying.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

vii

TABLES

1. The three process study grantees and their interventions............................................................... 5

FIGURES

ES.1. Necessary conditions for successful scaling of interventions ..........................................................ix

1. Necessary conditions for successful scaling of interventions .......................................................... 3
This page has been left blank for double-sided copying.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

ix

EXECUTIVE SUMMARY

The Corporation for National and Community Service (CNCS) invests significant 

resources in interventions designed to improve lives and strengthen communities, in 

evaluating the effectiveness of these interventions, and in supporting the scaling of 

effective interventions to serve new communities or populations. CNCS is interested in 

learning about factors that facilitate or impede such scaling. Therefore, CNCS 

contracted with Mathematica in 2016 to conduct a project, entitled Scaling EvidenceBased Models (SEBM), that aims to deepen the agency’s understanding of effective 

interventions and its knowledge base on scaling them.

Although many CNCS grantees have plans for scaling their interventions, little 

information is available regarding their scaling efforts. Mathematica, in consultation with 

CNCS, developed a scaling framework to inform the work on this project. Using it as a 

guide, Mathematica is conducting a process study of three CNCS-funded grantees and 

their partners to learn how they scaled their interventions, what factors facilitated or 

hindered scaling, and the conditions required for successful scaling (that is, what is 

needed for interventions to maintain their effectiveness after scaling). This report 

presents insights from a cross-site analysis of information collected during process 

study site visits with these grantees.

The scaling framework. Mathematica’s scaling framework identifies five conditions 

necessary to successfully scale an intervention (Figure ES.1). The first three conditions 

indicate whether an intervention is ready to be successfully scaled:

Figure ES.1. Necessary conditions for successful scaling of interventions 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

x 

1. A well-specified intervention consists of a description of the content, the mode of 

service delivery, the intensity, workforce needs, and setting for each core element.

2. A well-defined target population means a description of the population for which 

the intervention was found to be effective.

3. Implementation supports include a description of the processes and supports in 

place to ensure the intervention can be implemented with fidelity to its model, such

as a team that monitors implementation, continuous quality improvement processes, 

and pre-service and in-service workforce training requirements.

The final two conditions indicate that an organization might be ready to scale an 

intervention:

1. Enabling context consists of a description of the presence of organizational 

leadership and culture that supports innovation, learning, and improvement for an 

intervention.

2. Implementation infrastructure includes a description of the organizational 

infrastructure, such as a human resources system, workforce, funding, materials, 

and physical space, which supports implementation.

Scaling an intervention enables more people to participate in it, and scaling successfully 

means that the beneficial impacts of the intervention on participants that have been 

found through rigorous scientific research have been maintained or surpassed. CNCSfunded grantees have used three approaches to scale their interventions: expansion, 

replication, and adaptation. Expansion extends the intervention to more people in the 

same target population in the same location. Replication extends the intervention to the 

same target population in a new location. Adaptation extends the intervention to a new 

target population or implements it in a new 

setting (type of location).

Process study research questions, grantees, 

and data collection and analysis methods.

The cross-site process analysis presented in 

this report was designed to address two 

overarching research questions: (1) how does 

the organization define and operationalize 

scaling? and (2) how do organizations scale 

successful interventions? Our insights on these 

issues focus primarily on the approaches the 

three process study grantees have taken and 

the experiences they have had when scaling. As 

a result, they are more broadly applicable to a 

range of interventions and organizations. 

Additionally, while the insights are derived from 

Three process study grantees 

and their scaled interventions

Parent Possible, implementing the 

Home Instruction for Parents of 

Preschool Youngers (HIPPY) 

program in Colorado

The Child Abuse Prevention 

Council (CAPC), implementing the 

Birth and Beyond (B&B) Home 

Visitation Program in Sacramento 

County, California

United Ways of Iowa,

implementing the Reading Corps 

program in Iowa
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

xi 

the grantees’ experiences during scaling, they may also be more broadly applicable to 

implementation experiences other than scaling.

The three grantees in the process study were selected by CNCS from among a broader 

group of 2015 and 2016 AmeriCorps grantees and 2010 and 2011 Social Innovation 

Fund grantees. The selected grantees demonstrated an organizational readiness to 

scale and were operating interventions that demonstrated evidence of effectiveness. 

They were selected to vary on several characteristics, such as size, intervention focus 

areas, types of scaling, the successes and challenges with their scaling experiences, 

and their efforts to codify lessons learned. At the time of their selection, all three 

grantees were replicating their interventions at new sites, two were expanding services 

at existing sites, and two were adapting services to either serve a new target population 

or better serve an existing one. Two of the grantees were scaling interventions created 

by an external developer; the other was scaling a complex intervention that included 

both activities based on a curriculum from an external developer and activities 

developed by the grantee.

The analysis is based on information from two-day visits to each grantee during October 

2018; follow-up telephone interviews with one grantee in November 2018; and a review 

of program documents, such as recruitment materials and manuals for personnel. The 

visits typically consisted of one-on-one or small group interviews with: (1) the program 

manager who oversees the intervention; (2) a grantee executive; (3) one or more other 

grantee administrators; and (4) frontline staff and AmeriCorps members who are 

responsible for delivering the intervention. Site visitors also met with personnel from 

partner organizations. To conduct the analysis, we developed a site-specific template to 

code information according to topics of interest, assessed the topic-specific information 

to identify themes and insights, and used specific examples from the grantees to 

illustrate insights for this report. 

Insights from the process study cross-site analysis. The insights from the analysis 

pertain to two broad areas: (1) the approaches that grantees and their partners took to 

scaling—including how grantees viewed scaling and their actions when the scaling was 

taking place; and (2) specific aspects of how they scaled—including the organizational 

resources needed to scale, implementation supports, the organizations’ enabling 

contexts, and the use of evaluations in scaling efforts—and the challenges and 

facilitators they faced with these aspects of scaling.

Approaches to scaling. The three grantees identified and pursued opportunities 

through which scaling could address community needs, but the availability of funding 

and stakeholder interest could influence where, when, and how scaling occurred. 

Funding could be available to scale services in one part of a geographic area but not 

another, for example. In addition, contractual arrangements for using an intervention 

model could make replication and expansion more feasible than adaptation because 

intervention developers external to grantees had prescriptive rules guiding 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

xii

implementation of their intervention models. More broadly speaking, though, site visitors 

perceived that grantees do not seem to view scaling as an activity fundamentally 

different from the other implementation activities they pursue (and that also align with 

the program’s mission and the grantee’s desire to serve the community). Scaling could 

be viewed, at least partly, as business as usual for the grantees as they strove to 

identify and implement innovative ways to provide services.

Implementation infrastructure. Some resources that are part of an organization’s 

implementation infrastructure for scaling were easily obtained but others posed more 

challenges. Grantees were easily able to acquire intervention materials and physical 

space to implement the interventions—likely due, in large part, to the use of externallydeveloped interventions and the ability of personnel to meet participants at home for two 

of the interventions and at school for the third. However, approaches to personnel and 

human resource systems while scaling the interventions were more complicated and 

raised three areas of concern. First, when replicating an intervention at new sites, 

grantees needed to strike a balance between having enough personnel at the new sites 

to implement the intervention effectively and avoiding inefficiencies that would arise as 

procedures at the new sites were developed and refined. Second, administrators at the 

two grantees that adapted their interventions to serve a new target population, or better 

serve an existing one, needed to consider the linguistic and cultural needs of the 

personnel best suited to serve members of the target population. Third, administrators 

from all three grantees reported difficulty recruiting and retaining the needed number of 

AmeriCorps members. Although this could pose a challenge for grantees regardless of 

whether or not they are scaling and could suggest a need nationwide to address 

recruitment challenges (such as through a national recruitment campaign), scaling 

exacerbated the issue by increasing the needs for AmeriCorps members.

In addition, funding sources facilitated scaling, but also presented some challenges. 

Administrators at two of the three grantees highlighted the precarious nature of their 

intervention activities due to the short-term nature of grants and the need to continually 

search for new funding. This led to use of a mix of funding sources, would could be 

inefficient because each source had its own eligibility or reporting requirements. 

Different funders might also have different time frames or interests that could pressure 

grantees to adapt their activities to suit the funder.

Implementation supports. Several implementation supports, such as training and 

communications, were impacted by the additional personnel needed for scaling. For 

example, scaling meant the need to adapt training logistics to accommodate a larger 

number of personnel across additional locations. As a result, grantees for all three 

interventions conducted more decentralized trainings, and they developed site-specific 

materials to help personnel better apply the information obtained through training to 

their distinct circumstances. In addition, scaling meant that face-to-face communication 

was less feasible, so all three grantees relied on technology (such as email or video 

chat) and small-group meetings to convey important information. Furthermore, the 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

xiii

combination of scaling activities and day-to-day program operations left little time for 

grantee personnel to develop or conduct a continuous quality improvement process to 

create and test strategies to address implementation challenges and, as a result, 

improve the program.

Enabling context. Grantees and partners typically had support from internal and 

community leaders, as well as other stakeholders, to scale their interventions. Across 

the grantees, personnel implementing the intervention reported feeling supported in 

their positions by their organizational leaders, and they fostered a collegial and 

collaborative environment. Despite this high level of support, program scaling was 

sometimes impeded because of organizational turnover, policies, or other challenges. 

For example, some personnel at one grantee organization discussed how a transition of 

organizational leadership led to a decrease in the responsiveness of leadership to 

frontline personnel. For another grantee, organizational policies that limited where and 

how they could fundraise presented barriers to successful scaling. Furthermore, 

although all three grantees valued partners’ contributions, they noted that a partner’s 

goals might not perfectly align with the grantee’s goals. This could limit scaling, such as 

by restricting the potential for serving a new target population.

Use of evaluations in scaling efforts. In general, when grantees and partners used 

data, they did so to monitor performance and report to funders. They were not 

conducting internal evaluations related to the effect of the interventions on their program 

participants. Despite this, administrators from two grantees felt that scaling might 

facilitate the development of additional evidence about their intervention’s effectiveness, 

which could foster further scaling.

Next steps. This report, which is based on a cross-site analysis of information primarily 

from site visits to three grantees, presents insights into how the grantees and their 

partners approached scaling their interventions and the organizational and 

implementation aspects of how they scaled. It is one of four documents that 

Mathematica has provided to CNCS as part of the process study component of the 

SEBM project. The study team also prepared three site-specific reports, one for each of 

the three grantees (Anderson et al. 2020; Eddins et al. 2020; Jones et al. 2020). Those 

reports are based on the information gathered from the October 2018 site visits and 

information gathered through telephone calls with site personnel conducted about 12 

months later. The discussions that occurred about 12 months later focused on progress 

toward scaling since the site visits, changes to the interventions that occurred during the 

additional scaling, and challenges that have arisen with scaling.
This page has been left blank for double-sided copying.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

1 

The Corporation for National and Community Service (CNCS) has invested significant resources 

in interventions designed to improve lives and strengthen communities through its AmeriCorps 

and Social Innovation Fund (SIF) programs, as well as other programs.1

 Both CNCS and the 

organizations it funds also have invested resources in evaluating the effectiveness of these 

interventions. CNCS would like to support the scaling of effective interventions it funds and is 

interested in learning about the factors that facilitate or impede such scaling.

In 2016, CNCS contracted with Mathematica to conduct a project, entitled Scaling EvidenceBased Models (SEBM), to deepen the agency’s understanding of the most effective program 

innovations and its knowledge base on scaling them. Although many of the organizations whose 

interventions CNCS has funded have plans for scaling them, little systematic analysis has been 

conducted on their plans for scaling interventions.

The SEBM project’s process study examines three CNCS-funded grantees and their partners to 

learn how they scaled their interventions, what factors facilitated or hindered scaling, and the 

conditions required for successful scaling (that is, what is needed for interventions to maintain 

their effectiveness after scaling). This report is based on a cross-site analysis of the information 

collected from the three grantees visited by Mathematica study team members for the process 

study. The study team also prepared three site-specific reports, one for each of the three grantees 

(Anderson et al. 2020; Eddins et al. 2020; Jones et al. 2020). These reports, and the project 

overall, are intended to support CNCS’s efforts to develop a strategic approach for supporting 

scaling of evidence-based models. 

This report contains four main sections. In the first we describe the scaling framework developed 

for the project and which guides the process study. In the second we provide an overview of the 

process study, including the research questions it is answering, the method used to select three 

CNCS-funded grantees for inclusion in the process study, an overview of these grantees and their 

interventions, and the analysis methods used. In the third section we present insights about how 

the grantees defined and operationalized scaling. In the fourth section we offer insights about 

grantees’ efforts to scale their successful interventions, including (1) the resources they needed, 

(2) their considerations to ensure fidelity to the interventions, (3) the organizational factors that 

facilitated or hindered scaling, and (4) the role that evaluations and data played. In an appendix 

to the report, we describe in greater detail the methodology used for selecting the grantees for the 

process study and more information on their interventions, service delivery models, target 

populations for services, and organizational characteristics.

1 CNCS (2016) and CNCS (n.d.a) provide a description of CNCS programs. AmeriCorps supports a wide range of 

local service programs through grants to address critical community needs, such as those pertaining to education, 

public safety, health, and the environment. From 2010 to 2016, SIF grants were used to fund community-based 

programs to address challenging social problems communities face in the areas of economic opportunity, healthy

futures and youth development.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

2 

A. Overview of scaling and components necessary for successful scaling

CNCS-funded organizations have used three approaches 

to scale their interventions so more people could receive 

the beneficial impacts of an intervention: expansion, 

replication, and adaptation (see the box for an 

explanation of each option).2 Successful scaling uses at 

least one of these approaches and maintains or surpasses 

the beneficial impacts of the intervention on participants 

that have been found through rigorous scientific research. 

Drawing upon research from implementation science, 

Mathematica developed a framework that identifies five 

conditions necessary to successfully scale an intervention 

(Figure 1; Maxwell and Richman 2019). 

The first three conditions indicate whether an 

intervention might be ready to be successfully scaled:

Types of scaling

Expansion extends the 

intervention to more people in 

the same target population in 

the same location.

Replication extends the 

intervention to the same target 

population in a new location.

Adaptation extends the 

intervention to a new target 

population or implements it in a 

new setting (type of location).

• A well-specified intervention consists of a description of the content, the mode of service 

delivery, the intensity, workforce needs, and setting for each core element.

• A well-defined target population means a description of the population for which the 

intervention was found to be effective. 

• Implementation supports include a description of the processes and supports in place to 

ensure the intervention can be implemented with fidelity to its model, such as a team that 

monitors implementation, continuous quality improvement processes, and pre-service and inservice workforce training requirements.

The final two conditions indicate that an organization might be ready to scale an intervention: 

• Enabling context consists of a description of the presence of organizational leadership and 

culture that supports innovation, learning, and improvement for an intervention. 

• Implementation infrastructure includes a description of the organizational infrastructure, 

such as a human resources system, workforce, funding, materials, and physical space, which 

supports implementation. 

2 Strategies other than scaling that CNCS-funded organizations have used to continue the implementation of their

interventions include (1) sustaining services—that is, planning to serve the same population in the same location 

without making purposeful changes to the intervention; and (2) deepening services—that is, serving the same target 

population in the same location with enhanced services of the same intervention.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

3 

Figure 1. Necessary conditions for successful scaling of interventions

B. Overview of the process study

The process study was designed to collect rich information from three CNCS-funded grantees 

and their partners about their experiences scaling evidence-based interventions. In this section 

we present (1) the study’s research questions, (2) the methods used to select three grantees for 

inclusion in the study, (3) overviews of each grantee and their interventions, (4) the data that 

were collected, and (5) the cross-site analysis conducted for this report.

1. Process study research questions

To help deepen CNCS’s understanding of how to support the scaling of effective interventions, 

the process study was designed to address two overarching research questions: 

1. How does the organization define and operationalize scaling? 

2. How do organizations scale successful interventions? 

In this report’s cross-site analysis, we present emerging themes from the information collected 

from the process study grantees. These themes address the overarching research questions and 

offer insights into how grantees approached scaling, the actions they took when they scaled, and 

the challenges and facilitating factors they faced when scaling. Although we include descriptions 

of the interventions used by the three grantees and their partners, we focus primarily on the 

approaches the grantees have taken and the experiences they have had when scaling so that our 

insights are more broadly applicable to a range of interventions and organizations. Additionally, 

while the themes discussed in this report are derived from the grantees’ experiences during 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

4 

scaling, they may also be more broadly applicable to implementation through methods other than 

scaling.

2. Selection of the three grantees included in the process study

A multistage process was used to select the interventions and the grantees scaling them for 

inclusion in the process study. After reviewing documents submitted to CNCS by its 2015 and 

2016 AmeriCorps grantees and 2010 and 2011 SIF grantees, Mathematica identified 17 

interventions that demonstrated evidence of effectiveness and determined which of these, and 

their implementing grantees, demonstrated a 

readiness to scale based on five conditions shown 

in Figure 1. After Mathematica provided a 

preliminary list of recommended 8 interventions 

and the 17 grantees scaling them for CNCS to 

consider for the process study, CNCS conducted 

an extensive effort to gather information from 

these grantees and the CNCS stakeholders who 

work with them. Based upon the information 

obtained, CNCS chose three grantees and their 

interventions to include in the process study. The 

three grantees were selected to vary on several 

characteristics, such as size, intervention focus 

areas, types of scaling, the successes and 

challenges with their scaling experiences, and their 

efforts to codify lessons learned. Importantly, they 

were not selected at random, and they are not 

representative of all CNCS grantees. Therefore the 

insights from the experiences of the three process study grantees and their partners cannot be 

interpreted as applicable to a broader set of CNCS-funded grantees or service providers. 

Three process study grantees 

and their scaled interventions

Parent Possible, implementing the 

Home Instruction for Parents of 

Preschool Youngers (HIPPY) 

program in Colorado 

The Child Abuse Prevention 

Council (CAPC), implementing the 

Birth and Beyond (B&B) Home 

Visitation Program in Sacramento 

County, California

United Ways of Iowa, 

implementing the Reading Corps 

program in Iowa

3. Overview of the three process study grantees and their interventions

For the process study, Mathematica visited the following grantees to gain insights on the 

organizations themselves and the interventions they are scaling. Table 1 provides brief 

descriptions of the grantees and their interventions, while Appendix A provides more detailed 

information. 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

5 

Table 1. The three process study grantees and their interventions

Program Description Scaling activities

Parent Possible

Parent Possible—formerly the Colorado Parent and Child Foundation—

which is a state-level office in Colorado, received a CNCS grant in 2015 

to implement the Home Instruction for Parents of Preschool Youngers 

(HIPPY) program, a standardized program and home visiting curriculum 

licensed from HIPPY USA. HIPPY is implemented in a mix of urban and 

rural locations in nine Colorado counties. The intervention is intended to 

increase the early language and literacy skills of young children (ages 3 

to 5) to improve their school readiness, school attendance, classroom 

behavior, and academic performance. It is also designed to boost 

parents’ involvement in their children’s learning and their own selfesteem as educators. The intervention consists of (1) home visits with 

parents of 3- to 5-year old children, which include instructional exercises, 

discussion, and feedback to parents; (2) activities for parents to conduct 

using a standardized curriculum and learning materials; and (3) monthly 

group meetings, which include presentations to parents by guest 

speakers, enrichment activities, and themed discussions to reinforce the 

home visits. AmeriCorps members serve as HIPPY home visitors.

The grantee has been scaling the 

program in two different ways. First, in 

the two years prior to the site visit, the 

grantee replicated HIPPY in four new 

sites in the state. One of these was a 

reopening of a site with a new partner 

organization after the site had previously 

ended operations with a different partner 

organization. Second, the grantee has 

been making adaptations by adding 

activities to home visits as funding 

permits and in a pilot-like fashion. 

Examples of topics for add-on activities 

include health, nutrition, and housing, 

with the topics chosen during particular 

visits depending on a parent’s interest.

The Child Abuse Prevention Council (CAPC)

CAPC received a 2016 CNCS grant to implement, in conjunction with 

partners, the Birth and Beyond (B&B) program. Based in Sacramento 

County, California, the CAPC coordinates a countywide collaborative 

network of partners (called the Collaborative) that provide B&B services 

that seek to reduce child maltreatment. The intervention consists of: (1) 

home visits to parents of children between 0 to 5 years old, (2) group 

parent education classes conducted at the Family Resource Centers 

administered by Collaborative partners, (3) crisis intervention services, 

and (4) referrals to and the provision of enhanced services and supports 

that help to address families’ other needs. The curriculum used for the 

home visits and classes is called the Nurturing Parent Program (NPP). 

AmeriCorps members serve as B&B home visitors or parent educators.

As of the site visit, the Collaborative was 

scaling B&B in four ways: (1) replicating 

the intervention at a new site; (2) 

expanding services at an existing site to 

serve more families; (3) adapting 

services from families with children ages 

0–5 to families with children ages 0–17; 

and (4) adapting services and NPP 

materials to be more attuned to the 

cultures of certain demographic groups.

The United Ways of Iowaa

The United Ways of Iowa, a state-level association of Iowa’s local United 

Way organizations, has been implementing Reading Corps since the 

2013–2014 school year. Reading Corps is a standardized literacy 

program that is licensed by Reading and Math, Inc. (formerly 

ServeMinnesota Action Network) for implementation in states such as 

Iowa. It involves the provision of one-on-one, in-school literacy tutoring 

to students who struggle to read, with the goal of boosting their reading 

skills. Students who participate in the program meet with tutors, who are 

AmeriCorps members, for 20 minutes each school day until they 

consistently achieve a target level of reading performance. Tutors use a 

set of 10 scripted interventions that target critical phonemic awareness, 

phonics, reading fluency, and comprehension skills.

Starting during the 2015–2016 school 

year, the grantee has been scaling the 

literacy-focused activities in the two 

different ways. It expanded Reading 

Corps’ implementation in some school 

districts and replicated it in other districts 

that did not previously have the 

intervention. In total for Iowa, Reading 

Corps went from being implemented in 

fewer than 10 schools to about 65 

schools and from involving about 10–15 

AmeriCorps members to about 65–70 

members. In addition, it incorporated a 

summer component to supplement the 

Reading Corps activities, which take 

place during the school year. However, 

the new summer activities sponsored by 

the grant are not considered a formal 

adaptation of Reading Corps activities 

because they do not affect any part of 

the developer’s program model.

aThe grantee name, as a state association of local chapters, is “United Ways of Iowa”, whereas specific local 

chapters are referred to as a “United Way.” 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

6 

4. Data collection and cross-site analysis methods 

We based this report on our cross-site analysis of information collected during two-day visits to 

each of the three process study grantees during October 2018 and follow-up telephone interviews 

with one grantee in November 2018.3 Although the details of the visits were tailored to the 

unique features of the interventions, grantees, and their partners, the visits typically consisted of 

one-on-one or small group interviews with (1) the program manager who oversees the 

intervention; (2) an executive of the grantee; (3) one or more other grantee administrators; and 

(4) frontline staff and AmeriCorps members who are responsible for delivering the intervention. 

Site visitors also met with administrators and frontline personnel from partner organizations. 

Interviews generally lasted between 30 minutes and two hours. Interview topics related to the 

five scaling conditions discussed earlier, the ways in which the grantee and partners have been 

scaling, and implementation and scaling challenges and facilitators. Study team members also 

reviewed documents that could shed light on grantee and partner fidelity to the intervention 

model and the supports they had in place to successfully scale it. Examples of such documents 

include recruitment materials, personnel manuals, human resource and communication protocols, 

and results of participant satisfaction surveys. 

To conduct the cross-site analysis, we first developed a site-specific template for coding the site 

visit interview notes and the information from related documentation according to the data 

collection topics of interest—that is, topics related to the research questions and the ways that 

grantees are approaching aspects of the five conditions necessary for successful scaling 

interventions. We then assessed the topic-specific information across the three sites to identify 

insights and takeaway conclusions that have the potential to be broadly applicable as CNCS 

seeks to support its grantees in their scaling efforts We use specific examples from the process 

study grantees to illustrate these ideas from the analysis.

While this report provides insights from our cross-site analysis of the information gathered to 

date, the study team supplemented this report with three site-specific reports that provide an indepth account of how each grantee is scaling its intervention (Anderson et al. 2020; Eddins et al. 

2020; Jones et al. 2020). Those reports are based on the information gathered from the October 

2018 site visits and information gathered through telephone calls with site personnel conducted 

about 12 months later. The discussions that occurred about 12 months later focused on progress 

toward scaling since the site visits, changes to the interventions that occurred during scaling, and 

challenges that have arisen with scaling.

C. Ways that organizations define and operationalize scaling

As described in Section B.3, the three CNCS grantees and their partners had several scaling 

activities underway when the study team conducted site visits. All three grantees and their 

partners replicated their interventions to new locations; the United Ways of Iowa’s scaling of 

Reading Corps was especially extensive, with about a fivefold increase in schools from one 

3 For convenience, we generally refer to the information as being provided through the site visits and from site visit 

interviewees, even though we obtained some information through telephone interviews.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

7 

school year to the next. In addition, CAPC expanded B&B services to serve more participants at 

an existing site and was in the process of both adapting services to families with children in a 

broader age range and making cultural adaptations to services and materials. Parent Possible also 

adapted its services by covering a greater breadth of new topics during home visits with families 

in addition to the HIPPY intervention content. 

To gain insights about the ways that organizations define and operationalize scaling, the study 

team asked grantees and their partners about the types of scaling they were pursuing, their 

rationale for doing so, how they adapted or modified the intervention when they scaled it, and the 

perceived effects of any changes on the intervention’s effectiveness. We reviewed and 

synthesized the information and drew several conclusions. 

Scaling activities were influenced by a mix of intentional planning, funding availability, and 

stakeholder interest.

4 Generally speaking, the three grantees strove to identify and pursue 

opportunities through which scaling could address specific community needs, but they also tried 

to capitalize on available funding and stakeholder interest. The study team heard several 

examples from grantees (all of which had been chosen because the programs they were striving 

to scale had evidence of effectiveness) to support this conclusion. 

For example, CAPC personnel opened B&B at a new site and expanded services at an existing 

site because the Collaborative identified these locations as having a high rate of infant mortality 

and a high call volume to the CPS hotline. The new site had been shut down due to a lack of 

funding (the Collective had chosen to close this site rather than cut back services at all their 

locations), and CAPC was able to reopen it when funding subsequently increased. The location’s 

previous closure meant that the Collaborative needed to redevelop trust between community 

members and the program, even though the program model had not changed. 

Parent Possible personnel targeted new sites for HIPPY services partly based on a review of 

demographic data and a needs assessment and partly on self-identification by the community. 

They identified potential areas with target populations that had little access to other pre-K 

services. (Some areas, such as rural ones, might have few or no other similar services.) However, 

an administrator also reported that sites often self-identify to receive HIPPY services, thus 

spurring the grantee to fundraise to facilitate expansion to those areas. Some sites already have 

funding available and reach out to Parent Possible for other support, such as support with the 

application to HIPPY USA.5

Intervention characteristics, such as contractual arrangements and complexity of 

interventions, might constrain the type of scaling done. The type of scaling that is feasible can 

depend heavily on whether or not the grantee is the developer of the intervention. Personnel 

4 Issues related to the funding of scaling activities are discussed more fully in Section D.

5 HIPPY USA personnel supported a view that some opportunities to scale arise due to initiative by others besides 

the HIPPY organization. They reported that efforts to adapt the program to serve children with autism were spurred 

when a state agency that serves families affected by autism inquired about the potential to serve a child with autism 

who was in a family with another child already receiving HIPPY services.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

8 

implementing Reading Corps and HIPPY reported needing to maintain fidelity to the 

intervention because of contractual requirements imposed on them by the intervention 

developers, although they also valued doing so because they were aware that the interventions 

had evidence of effectiveness. 

• The licensing agreement for the United Ways 

of Iowa required strict adherence to the 

Reading Corps model, which has very specific 

requirements for service delivery. Reading 

Corps materials used for tutoring are 

structured, and an important component of 

program implementation is monitoring tutors’ 

adherence to the program’s model and 

students’ progress by both internal coaches 

and master coaches.

• Sites implementing HIPPY must go through 

an accreditation process every three years, and 

an accreditation worksheet provided by 

HIPPY USA specifies what a site needs to do 

to implement the model as designed. 

In these contexts, it is likely more feasible to scale 

through expansion or replication than through 

adaptations. 

Spotlight on United Ways of Iowa

Personnel at the United Ways of Iowa 

desired to scale Reading Corps at Iowa 

schools with the greatest need (that is, 

those having the lowest reading scores), 

but the grantee needed to work within the 

context of the United Way’s broader 

requirements for operations. As a statelevel United Way entity, the grantee cannot 

raise funds through efforts that might 

compete with fundraising efforts of local 

United Ways. Therefore, the matching 

funds, which are required to place 

AmeriCorps members in schools as 

Reading Corps tutors, must be generated 

locally—in practice, this means either from 

local United Ways or schools. If a local 

area was unable to generate the matching 

funds, operating Reading Corps in the 

area’s schools was not feasible. Hence, in 

addition to school personnel’s desire for the 

school to have Reading Corps, a driving 

factor determining which schools 

participate in Reading Corps is the ability to 

obtain the match funding.

Although CAPC did not face these contractual 

constraints, the complexity of the B&B intervention posed other constraints on scaling. The 

array of B&B services (described in Section A.3) might hinder the ability of an organization to 

replicate the intervention in locations outside of Sacramento County, where it is currently 

implemented. Administrators and frontline personnel view all four of the B&B intervention 

components as important to the program’s effectiveness for participants. Several components, 

including the crisis intervention services and referrals to and provision of enhanced services, do 

not follow a structure or curriculum that could facilitate replication by another organization. 

Additionally, each Family Resource Center (FRC) in the Collaborative implements the crisis 

intervention and enhanced services differently according to the needs of its target population, 

which may be difficult for others to replicate. Administrators identified this complexity as one 

reason the intervention has been scaled only within Sacramento County. 

Scaling can be viewed, at least partly, as “business as usual” for organizations that 

continuously strive to identify and implement innovative ways to provide services to people. 

Administrative personnel across all three grantees reported that they continually looked for new 

opportunities to: expand their services to areas that are not already receiving those services; 

enhance services in new ways to assist the people they serve; and make appropriate 

modifications in response to the requirements of new funding, while keeping in mind the 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

9 

evidence of effectiveness of the original intervention. Within this context, site visitors perceived 

that grantees do not view scaling as an activity fundamentally different from the other 

implementation activities they pursue. 

Key Finding

To many grantee personnel, 

activities we identified as 

scaling were part of normal 

program operations to 

respond to community needs 

and improve the success of 

their intervention. As a result, 

they did not appear to view 

scaling as different from 

business as usual.

During the three site visits, the study team heard of numerous ideas that grantees and partner 

personnel had to improve what they were doing; some 

pertained to scaling while others were aimed at improving 

implementation supports more generally. For example, 

Reading Corps program personnel described an idea to 

partner with private schools to overcome funding 

constraints. Another idea was to offer more professional 

development training for AmeriCorps members. Topics to 

be covered in this training could include challenges with 

fostering literacy (including for children with behavioral 

issues), developing a deeper understanding of resources in 

the community, and fostering a greater sense of national 

service.

While implementation science research has identified different types of scaling and what is 

necessary for successful scaling (Fixsen et al. 2005), it appeared to site visitors that program 

administrators and frontline personnel were not bounded by these definitions. Their goals were to 

broaden and strengthen their services to meet individuals’ and families’ needs in whatever ways 

they could, regardless of whether that involved scaling or other strategies. Grantee personnel 

focused on implementing evidence-based interventions with fidelity because they expected that 

doing so would help their program participants. 

D. Ways that the organizations scale successful interventions

In this section, we present insights about the ways that grantees scaled successful interventions. 

We discuss the infrastructure resources needed, considerations for ensuring fidelity to the 

interventions, organizational factors that facilitated or hindered scaling, and the roles played by 

evaluations and data.

1. Implementation infrastructure needed to successfully scale interventions

As specified by our framework (Figure 1), successful scaling requires an implementation 

infrastructure. This infrastructure includes four key components: (1) human resource systems and 

dedicated intervention personnel, (2) materials to implement the intervention, (3) physical space, 

and (4) funding or financial resources. During the site visits, the study team learned from grantee 

and partner personnel about how they marshaled these resources.

Although we discuss each of these infrastructure components needed for scaling, the importance 

of each can depend on details of the intervention itself. For example, a home visiting program 

might have little need for physical space but great needs for personnel training. With this in 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

10

mind, we identified some cross-cutting insights about the organizational infrastructure needed for 

scaling based on the information collected from the three grantees.

a. Workforce and human resources 

We identified four insights about workforce needs for scaling, the last of which pertains 

specifically to the use of AmeriCorps members given the distinctive nature of that program. 

When replicating to new sites, administrators need to carefully assess how a site’s initial 

personnel size influences resource efficiency. An administrator pointed out different 

considerations that could influence the desired initial size of a new site. On the one hand, there is 

a goal of having a new site with enough personnel to operate efficiently. The fixed costs of 

running a site, such as leasing building space and 

providing utilities, might be about the same 

regardless of the number of personnel or participants 

at the site; based on experience, an administrator for 

one grantee cautioned against having any site with 

so few personnel or participants that service delivery 

is not cost-efficient. On the other hand, there can be 

a goal of working out the challenges that arise with 

scaling before bringing on a larger team. Another 

administrator cautioned against trying to achieve too 

large of a scaling effort in a single site because of 

inefficiencies that would arise given the need to 

debug the processes during scaling; these processes 

could include recruiting participants, receiving 

referrals from partners, or providing services in a 

new physical space. For example, guidance from 

HIPPY USA is to start with three or fewer home 

visitors at a new site. 

Spotlight on HIPPY

The HIPPY intervention had 

personnel rules that facilitated 

scaling. The HIPPY program 

model includes a hierarchical 

supervisory structure, with both a 

coordinator and a supervisor of 

that coordinator at each site. 

Eventually, a site might get so 

large that an additional layer of 

personnel is needed. HIPPY 

program rules specify that an 

assistant coordinator is needed if 

a site serves more than 180 

children. Adherence to this rule is 

required to maintain accreditation.

Scaling can have significant effects on supervisory and frontline personnel. Scaling may make 

supervision easier. United Ways of Iowa administrators explicitly stated that implementation of 

Reading Corps in more schools (some of which were in new school districts and some of which 

were in districts in which the program had already been operating) led to a more efficient 

oversight system because master coaches became located closer to the tutors and internal 

coaches they supervise. Before scaling the intervention, there were few master coaches and the 

schools each one oversaw were spread out, making travel to them difficult. With scaling, there 

were more schools and master coaches, who were located closer to their schools, cutting master 

coaches’ travel time. Expanding to new schools also shortened the learning curve for master 

coaches. With scaling, master coaches were assigned to work with more tutors and internal 

coaches than before, which means that they more quickly became experienced with their tasks of 

observing and monitoring tutors and internal coaches. 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

11

Furthermore, as a result of scaling, administrators from United Ways of Iowa needed to develop 

formal policies and procedures related to workforce and human resources (and other aspects of 

managing the implementation of Reading Corps outside the scope of the developer’s 

requirements) when a more informal approach had been sufficient previously. The decentralized 

nature of the intervention—with tutors and internal coaches spread across a state—contributed to 

the need for formal policies and procedures after expansion to new schools. 

When striving to serve a new target population or better serve an existing one, administrators 

valued personnel whose language and culture aligned with those of the target population and 

noted challenges. Through the site visits, we learned of 

efforts to better serve participants with distinctive linguistic 

or cultural needs. HIPPY administrative personnel reported 

that a barrier to serving a target population can be a 

mismatch between the languages spoken by personnel and 

by participants, especially those in communities with a high 

concentration of refugees or immigrants. In addition, HIPPY 

personnel valued having staff and AmeriCorps members 

who come from the communities they served. Personnel 

involved in the B&B program also emphasized the 

importance of having a cultural fit when they were providing 

services to African Americans. Generally, program 

personnel described taking into account both a wide breadth 

of cultural experiences of target populations and the needs of personnel with similar cultural 

backgrounds to successfully implement an intervention. For example, Parent Possible has home 

visitors who speak Spanish so that the HIPPY program can serve Spanish-speaking families, but 

some of these personnel are not fluent in English. 

HIPPY USA provides materials for families in 

both Spanish and English, but the training, forms, 

and materials for home visitors had not yet been 

translated into Spanish. The grantee is translating 

these supporting materials for the home visitors to 

address this issue, but they had not finished their 

efforts at the time of the site visit.

Key Finding

Organizational leadership 

might take into consideration 

the characteristics and 

backgrounds of their 

personnel when adapting to 

serve a new target 

population. Employing those 

with similar characteristics to 

those they intend to serve 

may help to reach that 

population.

Spotlight on Birth and Beyond

During our site visit, CAPC personnel 

described their satisfaction after adapting 

B&B to be more culturally responsive to 

African Americans. Through an analysis 

of data about retention rates of families in 

the program, CAPC personnel learned 

that African American families were not 

being retained at the same rate as 

families in other demographic groups. 

Efforts to make the program more 

culturally responsive to this population 

included changing personnel training, 

hiring different types of personnel, and 

altering strategies to recruit participants. 

After the changes, the retention rate of 

African American families had become 

comparable to those of other families.

Serving new types of communities can be 

especially tricky when there are multiple 

components of an intervention. For example, 

personnel might communicate effectively when 

they work one-on-one with participants during 

home visitations, but additional personnel or 

accommodations might be needed when groups of 

participants who speak different languages meet. 

For example, when an intervention includes group 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

12

meetings in addition to home visits, it can be more difficult to schedule the meetings if the bulk 

of the meeting content is discussed in English but translators need to be in attendance to serve 

the needs of non-English speakers. 

Administrators from all three grantees reported difficulty recruiting and retaining AmeriCorps 

members—regardless of whether or not the program is scaling. One administrator reported that 

new AmeriCorps program rules make it challenging to replace AmeriCorps members who end 

their service agreements shortly after they start their service with the grantee.6

 According to the 

administrator, there used to be about four weeks as a time window at the start of an AmeriCorps 

member’s service that could be used for that member to assess the suitability of the position. 

Under the changed rules, the time window has been shortened to about one week. The 

administrator thought that this shorter amount of time is not enough for someone to develop a 

good understanding of what the program really is about. 

Furthermore, some potential AmeriCorps members might not understand that AmeriCorps is 

community service with a living allowance rather than a job. According to administrators, the 

living allowance is not much compared to what regular jobs can pay, and some potential 

members do not often see the value of community service. One administrator pointed out that it 

is especially challenging to recruit AmeriCorps members when the economy is strong, given how 

the stipend compares to what could be earned through a job. This administrator thought it 

especially important to find people who understand that the AmeriCorps position is community 

service rather than a job. Although this is may be a challenge for some programs regardless of 

whether they are scaling, scaling exacerbated the issue by increasing personnel needs. 

Furthermore, expansion into some types of areas might make finding and retaining AmeriCorps 

members interested in community service and in the compensation offered even more 

challenging than it would otherwise be. Examples of such areas that we heard about during the 

site visits were rural ones and ones with low unemployment rates. 

Given that all three grantees reported challenges recruiting and retaining AmeriCorps members 

to at least some extent, it might be valuable for CNCS to consider a broad effort, such as a 

national campaign, to help address these challenges.

b. Materials

Whether or not the intervention was created by a separate program developer influenced 

grantees’ access to and use of materials. The two grantees that replicated program developers’ 

intervention models—Parent Possible (HIPPY) and the United Ways of Iowa (Reading Corps)—

also were required to use the developers’ materials for service delivery. They reported easily 

acquiring the materials needed through purchasing copies or printing electronic versions. For 

example, HIPPY USA provides all of their accredited sites with the materials they need for 

successful implementation. They provide all of the copyrighted intervention materials, including 

what home visitors and coordinators need; as well as online information to help coordinators 

6 The administrator did not mention the AmeriCorps policy that can be used under some circumstances to refill slots 

that have been vacated by AmeriCorps members. More information on this policy can be found at CNCS (n.d.).
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

13

supervise and support home visitors on a weekly basis. In addition, during the training for 

coordinators, HIPPY USA provides an extensive set of information about topics such as 

workforce needs, the model, recruitment and retention, and marketing to guide implementation 

of the program. HIPPY USA also provides an accreditation worksheet that specifies everything a 

site needs to understand and implement the model as designed. 

As the developer of its own program, CAPC and its partners in the Collaborative had similar 

ease of access to materials but also had greater flexibility in adapting them. As of the time of the 

site visit, FRCs had been working with an external developer of the Nurturing Parent Program 

(NPP) curriculum that is used for a portion of their services to adapt that curriculum to new 

languages and to use with the children in the 6–17 age range. But, because B&B is more 

comprehensive than the NPP curriculum, the Collaborative had more flexibility to incorporate or 

adapt other materials to suit their purposes. 

c. Physical space

Organizations easily identified physical spaces to provide their services, but they also needed 

space for support activities. A principal factor that influenced this finding could be that the core 

components of two of the three interventions (B&B and HIPPY) were home visitations. Thus, 

the participants’ homes were a natural and appropriate 

space for the delivery of this service. However, the 

grantees and their partners providing these services 

needed space for other intervention-related activities. 

For example, each HIPPY site needs space to store the 

curriculum and family files and enable coordinators and 

home visitors to meet. One administrator reported that 

it is nice, but not all together necessary, for home 

visitors to have a space to review and complete 

paperwork. Overall, HIPPY personnel thought that the 

lack of physical space would not limit operating the 

program at a site, since the requirements for space are 

minimal. In addition, sites that are running likely could 

accommodate extra space requirements if scaling 

involves a small increase in the number of home visitors. 

Spotlight on Birth and Beyond

Physical space needs might change 

with adaptation of the intervention to 

a new target population. B&B’s 

adaptation to include families with 

older children required physical 

space that was appropriate for such 

children (such as a playground rather 

than a playroom). According to one 

site visit respondent, this need for a 

different type of physical space was 

a factor that led to the relocation of 

one of the program sites.

The United Ways of Iowa relies on schools to provide space for delivery of Reading Corps oneon-one tutoring services because the intervention involves removing students from their school 

classes. In essence, as with the home visiting services of B&B and HIPPY, the frontline 

personnel meet participants where they normally are found. However, one site visit respondent 

reported that, occasionally, finding a computer and an office space in a school for the tutors to 

store materials and enter data could be challenging; for example, they might need to use a spot in 

the school library or a large closet. However, the tutors are able easily to move around the school 

buildings to meet the students in places such as activity areas. 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

14

d. Funding considerations for scaling 

Administrators highlighted the precarious nature of their intervention activities due to the shortterm nature of grants and the need to continually search for new funding. Grant administrators 

discussed the need to continually find funding to sustain the programs that they scaled. A HIPPY 

administrator described needing to patch together different funding streams over time to keep the 

program going, given that federal funding the grantee has relied on is for three years. If a certain 

type of funding is no longer available, the organizational leadership must look for other funding 

sources or reduce the number of participants served. Leadership also discussed another potential 

fund-seeking strategy: pursuing funding from sources that might value the ancillary benefits of 

the program, such as benefits to parents, rather than the primary benefits to children that the 

grantee has historically promoted and received funding for. In addition, an administrator from 

the United Ways of Iowa described how an inability to obtain state funding or fundraise was one 

factor that hindered the grantee’s ability to scale at a level that had been planned. 

Having different funding sources for scaling might lead to inefficiencies and lead to new or 

different requirements to adapt program model activities. Administrators for two grantees 

explained how using a new source of funding to add sites or 

expand services might mean that new data (or additional 

forms) need to be collected. This can lead to duplication of 

data collection efforts. Therefore, scaling based on funding 

from a new source could have different implications than 

scaling using funding from a preexisting source, even if the 

grantee and partners desire to have the same intervention and 

target population under each scenario. One administrator 

reported partial, but not full, success in blending 

performance evaluation requirements to reduce 

inefficiencies. Furthermore, administrators from two grantees perceived potential (or actual) 

funders as wanting to see changes in the intervention model. These administrators discussed the 

ensuing tension between accepting the funds and complying with funders’ goals and maintaining 

fidelity to an evidence-based program model. In addition, different funders might require 

different strategies for monitoring implementation or have different requirements for personnel’s 

education or experience.

Key Finding

Organizations often used 

multiple funding sources to 

support scaling. This can lead 

to the duplication of data 

collection efforts in order to 

satisfy the funding sources’ 

different requirements.

The timing of grant cycles may compress the planning period 

to the detriment of sound implementation. Both HIPPY and 

Reading Corps are implemented with cohorts of participants 

on an annual schedule: HIPPY’s home visits are held over a 

30-week period from September to May, and Reading Corps 

services are provided during the regular school year. However, 

the timing of funding availability might lead to a condensed 

planning period. For example, an administrator from one 

grantee reported knowing from experience that it takes about 

half a year of planning to get a new site up and running. If 

Key Finding

The timing of funding 

availability might induce a 

grantee to condense its 

preferred planning period 

to use a schedule that is 

challenging given the 

intervention features.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

15

funding becomes available with less than this lead-time or before the required start date for a 

new cohort of participants, there can be a strong temptation to compress the planning schedule. 

However, doing so runs the risk of implementation problems such as an inadequate workforce or 

time to train personnel, market the program, and recruit participants. 

2. Activities and considerations to ensure fidelity to the intervention when scaling

A key component of the scaling framework is having implementation supports in place to ensure 

fidelity to the intervention model. In this section we describe insights about several key 

components of implementation support including: training, communications, data systems, 

supervision, and continuous quality improvement (CQI). 

An increase in personnel because of scaling may lead to additional logistical considerations 

for training. Standardized training of AmeriCorps members and other grantee frontline 

personnel delivering the intervention is important to ensure personnel are consistently delivering 

the intervention as intended. All grantees or their partners that were part of the site visits 

provided such trainings, as well as supplementary training on special topics such as the data 

system or professional development. They also provided refresher trainings on a regular basis. 

For example, United Ways of Iowa had returning AmeriCorps members attend the annual 

training it provides for new members, and HIPPY held weekly role-play and informational 

sessions to support their trainings initiatives.

Scaling often led to logistical and programmatic challenges conducting training. Grantees 

identified three challenges:

• Lack of a centralized training. Replication to new sites across the state for two of the 

interventions (HIPPY and Reading Corps) meant additional challenges for providing 

centralized training of staff and AmeriCorps 

members working on the programs. One 

centralized training was not always possible due 

to long travel times for some site personnel and 

funding. Parent Possible (for HIPPY) and the 

United Ways of Iowa (for Reading Corps), as 

well as their partners, addressed this by 

conducting more than one training session for 

personnel for whom centralized training was not 

feasible. With support from the intervention 

developers, the grantees and partners also 

implemented a train-the-trainer model, through 

which a select group of personnel went to a 

centralized training and then conducted local 

trainings in their area. However, grantee and 

partner leadership for these programs still saw 

Key Finding

Grantees reported three logistical 

challenges with training grantee staff and 

AmeriCorps members that arose directly or 

indirectly due to scaling:

• Replicating to new sites meant training 

sometimes needed to be decentralized 

and conducted in more than one 

location, which could lead to a lack of 

uniformity of the training.

• Individuals often needed additional 

information or training specific to their 

site, which also needed to align with the 

materials to implement the intervention. 

• Sites often had to juggle trainings for 

new and existing personnel while 

actively implementing their programs 

and serving participants.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

16

the value of centralizing training for developing consistency across personnel and for 

building rapport. 

• Need for additional site-specific information. Parent Possible and United Ways of Iowa 

held trainings that were developed by the intervention developer. HIPPY and Reading Corps 

staff and AmeriCorps members stated that the standardized training was helpful, and that the 

standardized manuals, guides, and forms distributed by the intervention developer following 

a training were valuable for working with participants and implementing the intervention 

effectively. However, some frontline personnel stated that the training lacked information on 

how to implement the intervention at their particular site and, as a result, felt that this left 

them a great deal to figure out on their own. To address this concern, Parent Possible 

personnel added state-specific training to supplement the training led by HIPPY USA, such 

as training on the specific state data systems that personnel would use. Additionally, some 

frontline personnel reported that when site-specific information was created by the grantee, 

the site-specific forms, policies, and procedures sometimes lacked a connection to the 

training or were not covered during the training. 

• Issues with time management. Personnel from Parent Possible and the CAPC expressed a 

difficulty dedicating time to planning and holding ongoing trainings while actively delivering 

intervention services to participants. Organizational leadership and other personnel also 

expressed a desire for more trainings, but lack of time was still an issue. For example, some 

personnel that are part of the Collaborative wanted more clinical trainings on trauma and 

cultural responsiveness. Personnel implementing Reading Corps wanted more leadership or 

professional development training. However, leadership often did not have the time or budget 

to plan these trainings, nor did many personnel have the time to attend. Site visit discussions 

with grantees and their partners did not identify any solutions to this issue, which could be a 

challenge also in implementation contexts other than scaling. 

Communication strategies that work with a smaller team implementing an intervention might 

need to be revamped when scaling. As the number of sites implementing the intervention and 

the size of the personnel increase, communication between all parties involved might become 

more difficult. Many personnel interviewed during the 

three site visits discussed how easily they communicated 

with individuals in their organizations prior to scaling. 

This often involved regular in-person meetings and faceto-face communications (for example, an ad hoc meeting 

in a hallway). This made it easy for administrators to 

communicate changes, address issues common among 

personnel, and foster collaboration. Due to a reduction in 

these in-person communications resulting from increases 

in the number of staff and AmeriCorps members 

involved in the intervention, the personnel for one 

intervention fostered face-to-face or similar 

communications by using technology, such as video and 

Key Finding

While informal and in-person 

communication was possible 

when the grantee team 

implementing the intervention 

was small, scaling led to the 

need to formalize communication 

procedures and reliance on 

either multiple small-group 

meetings at sites or technological 

solutions like video chatting.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

17

group chat technology. They also relied on email communications. Two of the three grantees’ 

interventions (Reading Corps and HIPPY) also required regular communications between 

personnel in a supervisory role and frontline personnel to facilitate the distilling of information. 

However, one administrator expressed concern that even when regular communication occurs, 

having a larger team made it difficult to ensure that information was conveyed accurately to 

AmeriCorps members and program staff. 

Data systems were critical, but site visit respondents noted challenges when integrating them and 

encouraging personnel to use the system. Regardless of whether or not the grantees and their 

partners were in the process of scaling, they collected data on the implementation of their 

programs and used that data to monitor program performance and fidelity to their intervention 

model. Personnel at each program used a pretest or assessment to determine program eligibility 

and the appropriate services to provide. For example, the Reading Corps model requires use of a 

pretest to determine program eligibility, as well as ongoing assessments to check participants’ 

progress and make decisions about which specific literacy interventions to provide. Similarly, the 

B&B model used an assessment to determine which services participants should receive, and 

AmeriCorps members for the HIPPY program administered pretests to help determine which 

topics they would focus on when working with participants. The data system and the use of the 

data for monitoring led to three types of challenges:

• Additional personnel burden. Personnel involved in all three programs collected paperbased forms and entered the information on them into a data system. In addition, two of the 

three programs currently or recently used more than one system due to funding requirements. 

For example, B&B used two data systems and, as a result, some personnel thought that the 

data collection process was burdensome. Leadership at the Collaborative discussed 

attempting to address this by training personnel on the importance of research, data, and data 

collection. 

• Difficulties with data system use. All three grantees and their partners used a centralized 

database or data systems to capture, store, and analyze the intervention-specific data they 

collected. However, administrators at two of the three grantees reported that some personnel 

felt the data systems were not user-friendly. As of the time of the site visit, B&B program 

personnel were transitioning from one data system to another; they were using an interim 

system provided by a funder and were having difficulty transferring data accurately from the 

old system. Parent Possible personnel also reported needing to provide a good deal of 

technical assistance on how to use the system to partners implementing the program at 

different sites. 

• Finding data management and evaluation personnel to facilitate more usage of the data.

Administrators at the organizations we visited expressed some desire to do more with the 

data they collected, in terms of checking the quality of the data, monitoring implementation 

of the program, and evaluating program outcomes. This was particularly the case in the 

context of scaling, as the addition of new personnel and locations raised the need for more 

complex or nuanced monitoring. The United Ways of Iowa had been able to achieve this to 

some extent, because the developer of the Reading Corps intervention set up reports so the 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

18

program personnel could develop and run them on their own. More generally, other grantee 

administrative personnel wanted a data management system to perform checks on the data to 

ensure its quality and to generate reports related to program monitoring. 

Administrators from two grantees also discussed the need for more formal evaluation of the 

data contained within the system. Parent Possible personnel wanted to link different sources 

of administrative data to identify program nonparticipants and compare outcomes of 

participants and nonparticipants. United Ways of Iowa administrative personnel reported 

wanting to conduct evaluations to demonstrate program effectiveness; an evaluation is 

planned due to a grant requirement. Across the three grantees, however, time constraints, 

budget considerations, and the availability of potential job applicants with the desired skills 

limited the ability to utilize the data to the fullest capacity.

The interventions scaled by the grantees had built-in communication and supervision 

procedures that facilitated scaling. The three interventions implemented by grantees in the 

process study had prespecified monitoring schedules and procedures. For example, HIPPY 

personnel noted that the weekly meetings for personnel performing similar roles across sites, 

which are part of the program model, helped ensure consistent implementation. Additionally, 

with Reading Corps, the internal coach at each school meets weekly with the tutor working in the 

school. The Reading Corps model also has monthly meetings between the tutor, internal coach, 

and master coach to review the progress of all students on the tutor’s caseload and to make 

decisions about changing the specific interventions being used with each student based on 

student assessments. Reading and Math, Inc., also provides supervision either through site visits 

or phone calls. In addition, the sites where HIPPY services were provided needed to maintain a 

certain level of supervision and monitoring to retain their accreditation from the national office. 

Personnel spent the majority of their time implementing the intervention, which left little room 

for continuous quality improvement (CQI). None of the programs that the study team visited had 

formal CQI processes. A grantee administrator for one intervention expressed a desire to do 

more CQI but was concerned that doing so would impose additional burdens on personnel. 

Administrators at other organizations reported that their time was 

focused more on efforts related to human resources (for example, 

hiring) and performance monitoring. They were reluctant to shift the 

burden of the human resources and performance monitoring activities 

to other personnel who were focused on other aspects of the 

intervention. 

Key Finding

None of the three 

grantees had a 

formal CQI 

process.

Although no grantee had a formal CQI process for creating and 

testing solutions to difficulties, they did focus on identifying areas for improvement. For 

example, personnel in the Collaborative implementing B&B noticed that participants had 

improved outcomes after eight hours of programming, so they set that number of hours as a 

minimum goal for dosage. Across the grantees, however, personnel reported needing a process 

for prioritizing the areas they identified as needing program improvement, given the time crunch 

grantee and partner personnel faced with their many preexisting responsibilities.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

19

3. Organizational factors that facilitate or hinder scaling 

Our scaling framework specifies that organizational factors, such as supportive leadership, can 

create an enabling context to facilitate scaling. We learned about these factors, including the 

relationships between grantee personnel and their leaders, their partners and external 

stakeholders. In this section we discuss three findings from the study site visits related to the 

enabling context of the organizations. 

Grantees and partners typically had support from internal and community leaders, as well as 

other stakeholders, but some organizational characteristics, such as personnel turnover or 

policies related to fundraising, presented barriers. Across the grantees and partners, most 

personnel felt well supported by organizational leadership. 

In addition, many frontline and supervisory personnel

described their leaders as trustworthy. This level of trust 

extended broadly to other organizational leaders (for 

example, the board of directors) and other stakeholders. 

Several personnel described their leaders as the glue that 

kept the intervention together. Most site visit interviewees 

reported that they had a good working relationship with 

the leaders. And, in many cases, personnel also reported 

that organizational leaders fostered a collegial and 

collaborative environment. This included using humor, 

supporting the mental health needs of personnel, and teambuilding events. 

Key Finding

Generally, personnel across 

the grantees felt supported by 

their organizational leaders. 

The leaders were trusted and 

often worked to extend trust 

and collaboration between 

staff and AmeriCorps 

members across the 

programs they operated.

Grantee personnel for two of the three interventions reported that state-level agencies, local 

community organizations, and other stakeholders sparked or provided some support for scaling. 

For example, United Ways of Iowa personnel noted that an Area Education Agency (AEA) 

provided most of the master coaches as in-kind support for the Reading Corps program. Grantee 

personnel also reported that state legislation about reading proficiency was one of the catalysts 

for starting the Reading Corps program in Iowa, although they also stated that this type of 

support faded over time. The Collaborative providing B&B services had county-level support 

through funding to expand the ages served through the program. Personnel for both Reading 

Corps and B&B reported thinking that the support from the organizations legitimized and 

increased the intervention’s prestige in the community. (Site visitors did not learn about the 

extent to which HIPPY program personnel felt supported at the state or local levels.)

Despite this high level of support, program implementation was sometimes impeded because of 

organizational turnover, policies, or other challenges. Some personnel at one grantee discussed 

how a transition of organizational leadership led to negative changes, such as a decrease in the 

responsiveness of leadership to frontline personnel. And, as discussed earlier, challenges 

reported to site visitors across the three grantees include ones related to a small organizational 

capacity at the start of scaling efforts and an inability to conduct broad fundraising efforts. 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

20

Grantees valued partners’ contributions, but differences in partners’ goals could influence 

scaling and program implementation. Administrators at all three grantees described the value of 

their partnerships and the contributions of the partners towards scaling. However, personnel also 

noted that differences between the goals of the grantees and their partners, as well as their 

constraints (for example, financial resources), limited scaling in some instances. In the case of 

B&B services, for example, the organizations that are part of the Collaborative have been 

partners for about two decades, each contributing to delivery of program services. The 

Collaborative formed subcommittees for partner personnel serving in the same roles to facilitate 

good cross-organization collaboration and to help the Collaborative provide consistent and 

coordinated services. However, the Collaborative was constrained in its ability to expand their 

services to serve families with open Child Protective Services (CPS) cases because of rules 

imposed by the CPS partner. 

Intervention developers have a stake in the programs. As noted earlier, both Parent Possible 

and the United Ways of Iowa scaled interventions developed by other organizations. Each 

grantee is a state-level organization that worked with the 

intervention developer (HIPPY USA and Reading and Math, 

Inc., respectively), which had a vested interest in both 

ensuring that their model is implemented with fidelity and 

fostering innovations to the model. In addition to providing 

program materials and conducting or facilitating training, the 

developers provided technical assistance to the grantees to aid 

with troubleshooting problems and facilitating smooth 

implementation. Reading and Math, Inc., also provided the 

data system used to monitor the Reading Corps program, and 

the HIPPY USA accreditation process supports HIPPY implementation with fidelity. According 

to administrators, all of this support and guidance helped the programs in implementing scaling.

Key Finding

Intervention developers 

helped to ensure fidelity to 

the intervention model and 

were involved in innovative 

adaptations.

In both cases, personnel noted ways in which the intervention developers also were innovating to 

broaden the target population or to strengthen intervention services. As of the site visit, HIPPY 

USA was testing the value of providing HIPPY services to families with 2-year olds. Reading 

and Math, Inc., tests innovations, such as group tutoring and doubling the daily tutoring time for 

kindergarteners, in locations where it implements Reading Corps and then considers offering 

new components to United Ways of Iowa and other implementing organizations as either a 

required or optional change to the intervention. 

4. The role that evaluation and data (process, outcome, or impact data) play in scaling 

efforts

In general, site visitors noted that the significant time pressures on grantee and partner personnel 

meant they did not often have the opportunity—at least as of when the visits were conducted—to 

take full advantage of all of the possible ways in which they could use data. When grantees used 

data, they often did so to monitor performance and report to funders rather than to assess fidelity 

to the program model and to conduct evaluations of the effectiveness of the scaled interventions. 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

21

(See Section D.2 for more information on how grantees used data.) However, we identified one 

lesson from site visit interviewees about the interplay between evaluation and data and scaling. 

Grantees understood how scaling might facilitate the development of additional evidence 

about the effectiveness of an intervention, which could foster further scaling. An administrator 

with the United Ways of Iowa reported that, with a small scale, it can be hard to develop and 

show analytically rigorous evidence (such as through a random assignment or quasiexperimental design) that the program is effective at improving students’ reading scores. The 

administrator is hopeful that a planned external evaluation (required as part of a current CNCSfunded competitive grant) will provide evidence that the intervention is effective and thus will 

foster opportunities to further scale the program. Parent Possible personnel also were excited that 

HIPPY USA was testing the intervention with 2-year olds and hoped to adopt the intervention 

with this new age range when it was ready and showed evidence of effectiveness. Furthermore, 

HIPPY USA personnel reported that they want to look at which specific pieces of the HIPPY 

program work. Although not stated explicitly by HIPPY USA personnel, it is likely that this type 

of evaluation would be more likely to be feasible with a broader scale of the program, because of 

the complexity and statistical requirements involved in isolating the effects of the different 

intervention components. 

Spotlight on United Ways of Iowa

United Ways of Iowa personnel described the extensive support provided by their two main partners: 

an Area Education Agency (AEA) and the participating schools. Grantee personnel appreciated how 

the AEA-provided master coaches enhanced the grantee’s relationships with schools because those 

master coaches already work with those schools for AEA programs. Several grantee personnel felt 

that they could not have scaled the intervention to the same degree without the in-kind services of the 

AEA-provided master coaches. In addition, the grantee personnel acknowledged how schools 

provide both the physical space for Reading Corps tutoring and the internal coaches, who provide 

support to tutors and serve as liaisons between the program and other school personnel. School 

personnel also give permission for the participating students to be excused from their regular 

classrooms to participate in tutoring. 

As much as they depended on these partners, grantee personnel noted the potential for two tensions. 

Although they did not consider the issue serious or widespread, grantee personnel noted the risk that 

AEA-provided master coaches might not emphasize adherence to the intervention model as much as 

the grantee would like because the master coaches do not want to risk harming their existing 

relationships with school personnel from AEA programs. Furthermore, because school personnel 

focus on their broader educational mission, of which literacy is only one component, some school 

personnel (such as classroom teachers) might be reluctant to excuse students to participate in the 

program.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

22

References

Anderson, Mary Anne, Katie Eddins, Karen Needels, and Scott Richman. “Scaling the Home 

Instruction for Parents of Preschool Youngsters (HIPPY) intervention: Insights from the 

experiences of Parent Possible.” Report submitted to the Corporation for National and 

Community Service. Washington, DC: Mathematica, June 2020. 

Corporation for National and Community Service (CNCS). “AmeriCorps State and National 

Policy Frequently Asked Questions (FAQs).” n.d.a. Available at

https://www.nationalservice.gov/sites/default/files/documents/AmeriCorps_State_National_

Policy_FAQs.pdf. Accessed September 5, 2019. 

Corporation for National and Community Service (CNCS). “Metadata Glossary.” Washington, 

DC: CNCS, 2016. Available at https://www.nationalservice.gov/sites/default/files/

documents/GLOSSARY-Metadata%20Categories%20Definitions%20for% 

20Publication.pdf. Accessed July 31, 2019. 

Corporation for National and Community Service (CNCS). Strategic Plan 2010-2015. 

Washington, DC: CNCS. n.d.b. Available at https://www.nationalservice.gov/sites/

default/files/documents/11_0203_cncs_strategic_plan.pdf . Accessed August 7, 2018. 

Eddins, Katie, Mary Anne Anderson, Karen Needels, and Scott Richman. “Scaling the Birth and 

Beyond (B&B) intervention: Insights from the experiences of the Child Abuse Prevention 

Council (CAPC).” Report submitted to the Corporation for National and Community 

Service. Washington, DC: Mathematica, June 2020. 

Fixsen, Dean, Sandra Naoom, Karen Blase, Robert Friedman, and Frances Wallace. 

“Implementation Research: A Synthesis of the Literature.” Tampa, FL: University of South 

Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation 

Research Network (FMHI Publication #231), 2005. 

Jones, Christopher, Daniel Friend, Mary Anne Anderson, Karen Needels, and Scott Richman. 

“Scaling the Reading Corps intervention: Insights from the experiences of United Ways of 

Iowa.” Report submitted to the Corporation for National and Community Service. 

Washington, DC: Mathematica, June 2020. 

Maxwell, N.L., and S. Richman. “Scaling Programs with Research Evidence and Effectiveness 

(SPREE).” Foundation Review, vol. 11, no. 3, November 2019, pp. 55–67. 
This page has been left blank for double-sided copying.
Appendix A

Details of the process study grantees, their partners, 

and their interventions
This page has been left blank for double-sided copying. 
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

A.3 

In this appendix, we present details on the three grantees included in the process study, as well as 

their partners and the interventions they were scaling as of the process study site visits conducted 

in October 2018. 

Parent Possible—formerly the Colorado Parent and Child Foundation—which is a state-level 

office in Colorado, received a CNCS grant in 2015 to implement HIPPY, a standardized program 

and home visiting curriculum licensed from HIPPY USA. The intervention is intended to 

increase the early language and literacy skills of young children (ages 3 to 5) to improve their 

school readiness, school attendance, classroom behavior, and academic performance. It is also 

designed to boost parents’ involvement in their children’s learning and their own self-esteem as 

educators. Eligible children must be ages 3–5 at the start of a school year and can participate for 

up to three school years.7 Parents of 3- or 4-year old children or 5-year-old children not in 

kindergarten receive weekly home visits, while parents of 5-year-old children enrolled in 

kindergarten receive biweekly visits. Visits typically are 45 to 60 minutes long over a 30-week 

period from September to May. Home visits include instructional exercises, discussion, and 

feedback about the previous and current week’s activities, the learning needs of the child, a roleplay session to simulate the upcoming activities, and a check-for-understanding period. Materials 

for each year, provided by HIPPY USA, include activity packets (either 30 or 15, depending on 

the child’s age), 9 storybooks, and a set of 20 manipulative shapes. Sites implementing the 

HIPPY program can include additional activities, such as wellness activities or legal services, 

with the home visits; the availability of these activities depends on funding. 

Parents are also encouraged to conduct HIPPY activities on their own with their children and to 

participate in monthly group meetings held at FRCs, which are administered by partner 

organizations. The parent-child interactions are based on a standardized curriculum and activities 

and are to last for 15 minutes a day, 5 days per week. Families are also asked to read to their 

child frequently. The 1- to 2-hour group meetings include presentations by guest speakers, 

enrichment activities, and themed discussions to reinforce the home visits. The meetings also 

enable participating parents to learn from and interact with each other. According to HIPPY 

USA’s guidelines, parent monthly meeting should occur at least six times a year, although Parent 

Possible holds more meetings on an annual basis. 

HIPPY is implemented in a mix of urban and rural locations in nine sites in Colorado. Seven of 

the nine sites use AmeriCorps members as home visitors, while others rely on paid staff as home 

visitors. Prior to participation in HIPPY, each site conducted a community needs assessment to 

ensure that there was local support for implementation. While all families in these counties are 

eligible for services, families in which the parents have limited education and that are 

experiencing poverty and social isolation are especially targeted. (Evidence about HIPPY’s 

effectiveness at improving child outcomes pertains to families experiencing poverty, limited 

education, and social isolation.) 

7 To participate at age 5, the child needs to have completed at least one prior year, although not all sites use the program for 5-

year-olds unless they turn that age after starting a year at age 4.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

A.4 

The grantee has been scaling the program in two different ways. First, in the two years prior to 

the site visit, the grantee started four new sites in the state. However, one of the four involved the 

reopening of a site by a new organization after the site had previously ended operations with a 

different organization. Second, the grantee added activities to home visits, which is done as 

funding permits and on a pilot basis. Examples of topics for add-on activities include health, 

nutrition, and housing, with the topics chosen during visits depending on a parent’s interest. 

In addition, the grantee might be able to benefit from HIPPY USA’s efforts to foster scaling of 

the intervention. Examples of scaling efforts by HIPPY USA are explorations of how to adapt 

the program to better meet the needs of families who have children with autism and families 

whose children are 2 years old. 

The Child Abuse Prevention Council (CAPC) received a 2016 CNCS grant to implement, in 

conjunction with partners, the Birth and Beyond (B&B) Home Visitation program. Based in 

Sacramento County, California, CAPC coordinates a countywide collaborative network of 

organizations (called the Collaborative) that provide B&B services designed to reduce child 

maltreatment. The Collaborative, which has been in existence for 20 years, includes partners that 

contribute in different ways or in different geographic areas to the complex intervention. 

B&B consists of (1) home visits to parents of children between 0 to 5 years old; (2) group parent 

education classes; (3) crisis intervention services; and (4) referrals to and the provision of 

enhanced services and supports that help to address families’ other needs, such as assistance 

related to legal, education, food, domestic violence, child safety, and other health concerns. 

AmeriCorps members conduct home visits with participating families, typically on a weekly 

basis that can take place over as many as 55 weeks based on a family’s need. Family Resource 

Centers (FRCs), which are administered by different partner organizations in the Collaborative, 

provide group parent education classes and other services at locations across the county. The 

parent education classes, also led by AmeriCorps members, are typically held weekly for up to 

16 weeks. The curriculum used for both the visits and classes, called Nurturing Parent Program 

(NPP), was created by an external developer. Although anyone can participate in services 

provided at the FRCs, B&B participants receive home visits from an FRC determined by the 

participants’ zip code of residence.

Evidence of the intervention’s effectiveness in reducing child maltreatment was based on both 

rigorous research that examined the provision of home visits to families classified as having a 

high risk based on a risk assessment and parenting workshops for families classified as having a 

medium risk; however, during the site visit, the Collaborative was providing home visiting 

services or group parent education classes to families at any risk level except for those with open 

cases with Child Protective Services (CPS). B&B service providers strive for families to receive 

at least 8 hours of home visits, which is about 6 weekly visits, based on evidence from a recent 

evaluation that this is the minimum point at which improvements in outcomes are detected.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

A.5 

As of the October 2018 site visit, the Collaborative was scaling the B&B intervention in four 

ways8

: (1) replicating the intervention at a new site; (2) expanding services at an existing site to 

serve more families; (3) adapting services from families with children ages 0–5 to families with 

children ages 0–17; and (4) adapting services and NPP materials to be more attuned to the 

cultures of certain demographic groups, particularly African Americans and refugees. 

United Ways of Iowa, a state-level association of Iowa’s local United Way organizations, has 

been implementing a program called Reading Corps since the 2013–2014 school year. The 

Reading Corps program—a standardized literacy program that involves the provision of one-onone, in-school literacy tutoring to K-3 students who struggle to read—was initially implemented 

in Minnesota; using CNCS-provided funds, the United Ways of Iowa had replicated it to Iowa, 

with the goal of boosting the reading skills of students in the state. Reading Corps is overseen by 

Reading and Math Inc. (formerly ServeMinnesota Action Network), which implements it 

directly in some locations (primarily Minnesota) and licenses it for implementation by other 

organizations in other locations, such as Iowa. To select participants for Reading Corps, school 

personnel use an assessment to identify students who (1) are slightly below a cutoff point but 

high enough that they would typically not be eligible for more intensive reading interventions 

and (2) do not have attendance or behavior problems. 

Students in Iowa who participate in the program meet with tutors, who are AmeriCorps 

members, for 20 minutes each school day until they consistently achieve a target level of reading 

performance. Tutors use a set of 10 scripted interventions that target critical phonemic 

awareness, phonics, reading fluency, and comprehension skills. The specific interventions used 

for each student are selected based on that students’ needs and progress. Students are pulled from 

their regular school classes (but not during teacher-led reading instruction) to participate in the 

tutoring sessions. As part of the license to use Reading Corps materials and get support for the 

delivery of the program, the grantee must ensure close adherence to the program model, 

including the specific interventions used, the dosage of one-on-one tutoring per student, the pertutor student caseloads, and the use of data about each student’s skills to monitor his or her 

progress. Students participating in Reading Corps are assessed weekly to ensure that they are 

making progress towards literacy skill targets.

Reading Corps tutors are overseen and supported by two types of personnel—internal and master 

coaches—with expertise in literacy instruction. Internal coaches are school personnel, usually 

teachers or reading specialists who provide oversight and support to tutors. The internal coaches 

also handle much of the communication with parents and observe the tutors on a weekly basis. 

Master coaches oversee and provide support to the internal coaches and tutors at several schools 

and meet with both monthly to review student progress and to provide input on how best to tutor 

students. Most of the master coaches in Iowa are from one of the state’s Area Education 

Agencies (AEAs), although a few are paid for through United Ways of Iowa. 

8 These scaling efforts were not funded exclusively through the 2016 CNCS grant.
Scaling Insights from CNCS-Funded Grantees’ Experiences Mathematica

A.6 

After the initial replication of the Reading Corps program in Iowa, the United Ways of Iowa 

received a large competitive CNCS grant, which enabled it to dramatically scale literacy-focused 

activities starting with the 2015–2016 school year. It expanded implementation of Reading Corps 

in some school districts and replicated it in other districts that did not previously have the 

intervention. In total in Iowa, Reading Corps went from being implemented in fewer than 10 

schools to about 65 schools and from involving about 10–15 AmeriCorps members to about 65–

70 members. 

In addition, the CNCS grant enabled the United Ways of Iowa to expand a summer component to 

supplement the Reading Corps activities that take place during the school year. By 

supplementing the school year-activities with a summer component, AmeriCorps members are 

able to meet a requirement for the number of service hours they must earn per year. The design 

and structure of the summer component are very flexible: examples of activities include 

traditional Reading Corps drills and tutoring, other literacy activities, summer meals offerings in 

underserved areas, and wraparound services to support summer enrichment activities, such as art 

classes and field trips. However, the summer component of the grant activities is not formally 

considered to be Reading Corps because the summer activities do not affect any part of the 

intervention developer’s program model; furthermore, participants in them are not necessarily 

the same children as those who participate in the Reading Corps during the school year. 
This page has been left blank for double-sided copying. 
Mathematica

Princeton, NJ • Ann Arbor, MI • Cambridge, MA 

Chicago, IL • Oakland, CA • Seattle, WA

Tucson, AZ • Woodlawn, MD • Washington, DC 

EDI Global, a Mathematica Company

Bukoba, Tanzania • High Wycombe, United Kingdom

mathematica-mpr.com