Scaling an Intervention:

Recommendations and Resources

A Guide for Funders and Grantmakers

September 25, 2020

Karen Needels, Scott Richman, Nan Maxwell, and Mary Anne Anderson 

Submitted to: Submitted by:

AmeriCorps

250 E Street SW

Washington, DC 20525 

Project Officer: Lily Zandniapour

Contract Number: 

GS10F0050L/CNSHQ16F0049

Mathematica

111 East Wacker Drive, Suite 3000

Chicago, IL 60601-4303

Phone: (312) 994-1002

Fax: (312) 994-1003

Project Director: Scott Richman

Reference Number: 50332
This page has been left blank for double-sided copying. 
Mathematica iii

Contents

Abstract......................................................................................................................................... v

I. Background and Overview of the Guide ................................................................................1

A. Scaling Programs with Research Evidence and Effectiveness (SPREE) Process .........3

B. Purpose of this guide ......................................................................................................4

II. Recommendation 1: Identify or Build Evidence of Effectiveness for the Intervention............7

A. What needs to happen....................................................................................................7

B. Resources from the SEBM project..................................................................................9

III. Recommendation 2: Ensure the Intervention is Ready to Scale..........................................11

A. What needs to happen..................................................................................................11

B. Resources from the SEBM project................................................................................13

IV. Recommendation 3: Ensure the Organization is Ready to Scale the Intervention ..............15

A. What needs to happen..................................................................................................15

B. Resources from the SEBM project................................................................................16

V. Recommendation 4: Anticipate and Plan for Challenges.....................................................19

A. What needs to happen..................................................................................................19

B. Resources from the SEBM project................................................................................20

VI. Recommendation 5: Support the Intervention and Organization During Scaling.................23

A. What needs to happen..................................................................................................23

B. Resources from the SEBM project................................................................................25

References..................................................................................................................................27

Figures

Figure I.1 Phases, activities, and products for the Scaling Evidence-Based Models project........2

Figure I.2. Scaling Programs with Research Evidence and Effectiveness (SPREE) process ......4

Figure I.3. Improving outcomes for more people ..........................................................................4
Mathematica iv

This page has been left blank for double-sided copying. 
Abstract

Mathematica v 

Abstract

Successfully scaling effective interventions can help funders and their grantee organizations improve lives 

for large numbers of individuals and communities. AmeriCorps recognizes this potential and, in 2016, 

funded Mathematica to undertake the Scaling Evidence-Based Models (SEBM) project to gain insights 

about which of the interventions it funded showed evidence of effectiveness and might be ready to scale. 

By undertaking this project, AmeriCorps sought uniformity, strategic learning, and a focused vision to 

evaluate the range of existing evidence frameworks and metrics for determining what constitutes an 

effective intervention as applicable to its programs. The project was also designed to generate practical 

knowledge about how AmeriCorps could foster the successful scaling of effective interventions by 

helping to identify which interventions work and how they can work for more people. Analysis 

undertaken as part of the project identified interventions with the strongest evidence of effectiveness and 

provided AmeriCorps with an understanding of why other interventions did not meet criteria for rigorous 

research. The project also identified the extent to which organizations described intervention and 

organization conditions needed for scaling readiness and which organizations did not provide information 

specific enough to assess readiness. AmeriCorps commissioned this guide to disseminate the knowledge 

gained from the SEBM project and provide practical recommendations for how funders, including 

governmental agencies and philanthropic organizations, should think about and plan for scaling. 

Furthermore, this guide highlights for funders how they might better understand evidence of intervention 

effectiveness and readiness for organization and intervention scaling by undertaking a systematic process 

to assess scaling readiness of the interventions they fund. Resources that funders and their grantee 

organizations can use to help prepare for scaling are also listed in this guide. 
This page has been left blank for double-sided copying. 
Chapter I Background and Overview of the Guide

Mathematica 1 

I. Background and Overview of the Guide

AmeriCorps was established as a federal agency in 1993 with a mission to improve lives, strengthen 

communities, and foster civic engagement through service and volunteering.1 The agency achieves this 

mission through the AmeriCorps State and National program and three AmeriCorps Seniors programs 

(RSVP, Senior Companions, and Foster Grandparent); the Social Innovation Fund (SIF) also provided 

funding to innovative community-based programs from 2010 to 2016.2

 

Since their inception, AmeriCorps programs have used performance measures to demonstrate progress 

toward meeting program and agency goals and objectives. As of 2005, federal regulations governing 

AmeriCorps programs required AmeriCorps grantees to conduct evaluations of their programs.3

Following the signing of the Edward M. Kennedy Serve America Act in 2009, AmeriCorps began to 

more intentionally document and assess the evidence behind the interventions that it funded. The 

legislation called for the implementation of SIF, which was then one of a few tiered evidence

grantmaking initiatives at the federal level. In 2010, AmeriCorps adopted a tiered evidence framework 

and more intentionally integrated the use of evidence into its grantmaking process (U.S. Government 

Accountability Office 2016). Using the framework, AmeriCorps sponsored third-party systematic 

evidence reviews and targeted its funding to identifying evidence-based solutions and bringing those 

solutions to more people and communities in need across the country with the help of national service. 

AmeriCorps continued seeking more uniformity, strategic learning, and a focused vision to evaluate the 

range of existing evidence frameworks and metrics for determining what constitutes an effective 

intervention as applicable to its programs. In 2016, it commissioned Mathematica to conduct the Scaling 

Evidence-Based Models (SEBM) project to identify AmeriCorps-supported interventions with the 

strongest evidence and to ground and deepen the agency’s understanding of scaling practices. Through 

this project, the agency sought to leverage its investments by ensuring that its most effective interventions 

could be scaled to engage more people and communities across the country. The project also generated 

practical knowledge about how AmeriCorps can support and foster scaling of evidence-based 

interventions as well as support its grantees to more thoughtfully engage in scaling practices to ensure 

success. Figure I.1 depicts the phases of the SEBM project, along with the activities that took place and 

the products that resulted from each phase. 

1 Prior to September 29, 2020, AmeriCorps operated under the name Corporation for National and Community 

Service.

2 Because the three AmeriCorps Seniors programs (RSVP, Senior Companions, and Foster Grandparent) were not 

part of the project that generated this guide, we do not discuss these programs. AmeriCorps (2016) provides a more 

detailed description of AmeriCorps’ programs. 

3 AmeriCorps State and National grantees that receive an average annual grant of $500,000 or more must conduct an 

independent evaluation; grantees that receive less than $500,000 must conduct an internal evaluation. All 

evaluations must cover at least one year of AmeriCorps-funded service activity. For more information, see 

https://www.nationalservice.gov/pdf/06_1128_ac_sn_evaluation_faqs.pdf.
Chapter I Background and Overview of the Guide

Mathematica 2 

Figure I.1 Phases, activities, and products for the Scaling Evidence-Based Models project 

As part of the SEBM project, Mathematica designed and implemented the Scaling Programs with 

Research Evidence and Effectiveness (SPREE) process, which provides a comprehensive methodology 

for AmeriCorps and other funders to use when they want to help organizations scale effective 

interventions (Maxwell and Richman 2019). Mathematica applied the SPREE process to the State and 

National grantees 2015–2019 and SIF 2010–2014 interventions funded by AmeriCorps, conducting a 

two-part annual review of AmeriCorps-funded interventions. The first part assessed evidence about 

whether the interventions demonstrated impacts by improving desired participant outcomes; the second 

part assessed whether grantees were ready to scale interventions that were ready to be scaled. The review 

identified interventions with the strongest evidence of effectiveness and provided AmeriCorps with an 

understanding of why other interventions did not meet criteria for rigorous research. The project also 

identified the extent to which organizations described intervention and organization conditions needed for 

scaling readiness and which organizations did not provide information specific enough to assess readiness

(Richman and Streke 2020; Selekman et al. 2020). 
Chapter I Background and Overview of the Guide

Mathematica 3 

The SEBM project also included an in-depth process study that collected rich information from three 

AmeriCorps-funded grantees and their partners about their experiences in scaling evidence-based 

interventions. The process study generated insights about the approaches that grantees and their partners 

took to scaling, as well as the challenges and facilitators they faced with different facets of scaling 

(Anderson et al. 2020; Eddins et al. 2020; Jones et al. 2020; Needels et al. 2020).

To improve scaling readiness and build upon the lessons learned from the process study, AmeriCorps 

commissioned this guide to disseminate the knowledge gained from the SEBM project. This guide 

provides recommendations that funders can use to identify which of the interventions they fund are 

effective, enhance their knowledge base on scaling those interventions, and help them pursue scaling. A 

wide range of grantmakers, including governmental agencies and philanthropic funders, can use this 

guide to learn how to systematically review evidence of effectiveness of the interventions they fund, as 

well as assess how ready their grantees or funded organizations might be for scaling activities. 

A. Scaling Programs with Research Evidence and Effectiveness (SPREE) Process

To deepen AmeriCorps’ knowledge base on scaling evidence-based interventions, Mathematica designed 

and implemented the SPREE process under the SEBM project. The SPREE process provides a 

comprehensive methodology for funders to use when they want to help organizations scale effective 

interventions (Maxwell and Richman 2019). SPREE was grounded in rigorous research standards for 

evidence of effectiveness and implementation science research that identified how organizations might 

scale interventions so they improve desired outcomes for a larger number of participants (Miller et al. 

2006; National Implementation Research Network [NIRN] 2018). SPREE was further informed and 

refined by the lessons learned when applying the SPREE process to a group of organizations that scaled 

interventions using funding received from AmeriCorps. 

The SPREE process has two distinct parts (Figure I.2): 

1. Effective intervention. The first part ensures that the intervention to be scaled is likely to produce 

desired outcomes and is therefore worthy of being scaled. It is the base for preparing for scaling. For 

the SEBM project, the first part—the blue portion of Figure I.2—was designed to identify 

AmeriCorps-funded interventions with evidence of effectiveness, which we defined as those that met 

the project’s effectiveness standards and produced favorable findings on the majority of targeted 

outcomes among participants receiving the intervention. We often refer to interventions that have 

provided evidence of effectiveness as interventions that work. For more information about the 

standards used to determine whether interventions had evidence of effectiveness, see Richman and 

Streke (2020). 

2. Readiness to scale. The second part of the process—the green portion of the Figure I.2—is intended 

to ensure that the effective intervention and the organization implementing it are ready to scale. The 

SPREE process specifies five conditions that should be present for an intervention and organization to 

be ready to scale. Having these five conditions in place before scaling helps foster fidelity during and 

after scaling, which helps ensure that the intervention achieves at least the same positive outcomes for 

participants as it did before scaling. The SPREE process is important for guiding scaling because both 

fidelity and effectiveness often deteriorate as participants, personnel, and resources increase (Larson 

et al. 2017). The first three conditions of the SPREE process indicate that an intervention is ready to 

scale, and the last two indicate the organization is ready to scale it. 
Chapter I Background and Overview of the Guide

Mathematica 4 

Figure I.2. Scaling Programs with Research Evidence and Effectiveness (SPREE) process

Succinctly, the SPREE process provides a framework that funders can use to understand whether the 

interventions and organizations they fund demonstrate the conditions for successful scaling—meaning 

positive outcomes for a large number of people—are in place. The conditions for successful scaling 

include both offering an effective intervention and having the procedures and processes in place to ensure 

that it can be implemented as intended after scaling (Figure I.3). If either part is missing from the formula, 

funders’ goals of improving outcomes for a larger number of people may not be achieved.

B. Purpose of this guide

Drawing on the knowledge generated from applying the 

SPREE process to assessing AmeriCorps’ funded 

organizations and interventions during the SEBM project, 

this guide provides five key recommendations that will help 

funders identify which of their funded interventions are 

effective, enhance their knowledge base on scaling them, 

and pursue scaling. Recommendations are as follows:

1. Identify or build evidence of effectiveness for the 

intervention 

2. Prepare to scale: Ensure the intervention is ready for 

scaling

3. Prepare to scale: Ensure the organization is ready to 

scale the intervention

4. Prepare to scale: Anticipate and plan for challenges

5. Support the intervention and organization during scaling

Figure I.3. Improving outcomes for 

more people

Source: Adapted from Bertram et al. (2001).
Chapter I Background and Overview of the Guide

Mathematica 5 

For each recommendation, the guide frames the 

issue and describes what might need to be done to 

implement the recommendation. Examples from the 

experiences of AmeriCorps grantees show how the 

issues have played out in the field. We also provide 

specific examples of what funders can do to help 

support these recommendations. For each 

recommendation, we also list resources developed 

by this project to help funders and their grantee 

organizations build evidence that their interventions 

are effective, get ready to scale, and address 

challenges and support scaled interventions.

Although recommendations are general and can 

apply to different types of scaling, some resources 

might need to be tailored based on the type of 

scaling that is relevant for a funder and its grantee

organization (Fixsen et al. 2005). The SEBM 

project has focused on three types of scaling: (1) 

expansion, (2) replication, and (3) adaptation.

Expansion extends the intervention to more people 

in the same target population in the same location. 

Successful expansion requires the intervention and the organization to serve a larger number of 

participants than before with the same service quality and in a manner demonstrating fidelity to the 

intervention model’s design. Replication extends the intervention to the same target population, but in a 

new location. Successful replication requires the intervention and the organization to maintain service 

quality and fidelity to the intervention in the new location. Adaptation extends the intervention to a new 

target population. Successful adaption requires the organization to change the intervention in a way that 

maintains beneficial outcomes for the population being served. 

Considerations as you read this 

guide

• The recommendations and resources 

contained in this guide were 

developed with a focus on AmeriCorps 

grantees’ use when scaling their 

interventions; however, they may also 

be useful to a broader audience of 

funders.

• The recommendations and resources 

are based on successful scaling 

principles that have emerged from 

research.

• When tailoring resources, grantees 

should ensure that the general 

principles in which the resources are 

grounded are still applicable and 

relevant.
This page has been left blank for double-sided copying. 
Chapter II Recommendation 1: Identify or Build Evidence of Effectiveness for the Intervention

Mathematica 7 

II. Recommendation 1: Identify or Build Evidence of Effectiveness for 

the Intervention

Before an intervention is scaled to reach a larger 

number of people, funders should first determine if 

the intervention has demonstrated effectiveness for 

the population it initially targets. Funding the 

scaling of an intervention is of value only if the 

intervention has been shown to improve 

participants’ outcomes. Although funders and their 

grantee organizations might desire to scale an 

intervention based on observed improvements in 

participants’ outcomes, funders and grantees should 

confirm that the improvements are due to the 

intervention and not other factors. By either 

reviewing evidence produced by third-party 

evaluators and unbiased sources, or helping grantee 

organizations conduct their own rigorous 

evaluations, funders should seek to establish which 

interventions produce beneficial outcomes for their 

target populations before scaling those 

interventions. 

A. What needs to happen

Whether seeking to identify or build evidence for 

their intervention’s effectiveness, funders can use 

research and data to obtain a thorough 

understanding of how the intervention operates, 

identify what improvements are needed to make it 

effective, and show how it affects the participants 

being served. Five steps can guide how funders 

should approach this issue:

• Make sure grantee organizations have 

defined their interventions. Research and 

evaluation can help funders and their grantee 

organizations continuously improve 

interventions in addition to providing evidence 

of its effectiveness. A well-articulated theory of 

change is central to this effort because it 

defines an intervention and builds an 

understanding of how intervention activities 

change participant behaviors in ways that 

improve outcomes. It provides a road map that 

identifies the human, financial, and physical 

resources (inputs) required to implement the 

Building evidence: The experience 

of one AmeriCorps grantee

Parent Possible, an AmeriCorps grantee, 

has participated in the design and 

evaluation of a possible adaptation to 

Home Instruction for Parents of Preschool 

Youngsters (HIPPY), a home visiting 

intervention for parents of young children. 

Parent Possible and the intervention 

developer, HIPPY USA, recently began 

pilot testing an adaptation of the 

intervention called HIPPY for Friends, 

Families, and Neighbors in two sites in 

Colorado. 

Although HIPPY is typically delivered to 

parents of young children, under this 

adaptation, home visitors delivered 

HIPPY activities to nonrelative homebased childcare providers, as well as 

friends and family members who provide 

care for children while their parents are 

working. Typically, home visitors met with 

caregivers who watch two or three 

children in their homes and shared 

activities that the caregivers could 

perform with the children in their care. 

Home visitors delivered 15 activities in 30 

weeks, offering one activity every other 

week (as opposed to 30 activities weekly, 

as the evidence-based HIPPY curriculum 

requires). 

Parent Possible personnel reported some 

early implementation successes and 

challenges in piloting this adaptation. For 

example, one site made additional 

modifications to this adaptation of HIPPY, 

by meeting with caregivers as a group 

rather than one on one. This facilitated 

connections among the caregivers, which 

caregivers found to be beneficial, 

according to grantee personnel. If HIPPY 

USA chooses to disseminate this 

adaptation more broadly, a grantee 

personnel member said that the grantee

might make recommendations to HIPPY 

USA to incorporate a networking aspect 

into the adaptation for the caregivers.
Chapter II Recommendation 1: Identify or Build Evidence of Effectiveness for the Intervention

Mathematica 8 

intervention’s critical components (activities)

that result in services provided (outputs) that 

ultimately increase the desired outcomes. 

Funders might want to require including a 

theory of change framework in its solicitations 

for funding and review the framework to ensure 

the components are well-defined and the 

overall theory of change appears to be sound.

• Build the groundwork for the evaluation. A 

funder can help build research evidence and use 

it to make decisions if the funder has a culture 

of measurement, learning, and evaluation in 

place. Such a culture requires leadership, 

management, and staff to develop a common 

understanding about the value of measurement 

and evaluation in decision making and to agree 

on what constitutes evidence of an effective 

intervention (Austin and Claassen 2008a). 

Having such a culture also strengthens 

grantees. Although some grantees might have 

an established culture of learning that includes 

measurement and evaluation, others might 

require a cultural change to accommodate 

evidence-based decision making. Funders can 

provide grantee organizations with funding or 

other support to help them understand how 

evaluations can be used for improvement as 

well as decide whether to continue an 

intervention (Austin and Claassen 2008b).

• Devote resources for grantee organizations 

to conduct evaluations. Evaluations require 

financial and personnel resources. Although there may be trade-offs between dedicating the funds 

necessary for an evaluation and providing resources for implementation, using information from an 

evaluation—ideally, conducted by an unbiased, third-party evaluator—for data-driven decision

making increases the likelihood that the funder’s goals are achieved for producing beneficial 

outcomes for program participants. Recognizing that evaluations take financial, personnel, and other 

resources (such as subcontracts to third-party evaluators), funders should seek to devote sufficient 

resources to support organizations in conducting evaluations.

• Promote or encourage use of rigorous evaluation methods. Most interventions have testimonies 

from participants stating how the intervention changed their lives, but such anecdotal evidence does 

not provide convincing evidence that the intervention caused those changes. To demonstrate that the 

intervention caused improvements in participants’ outcomes, evaluation research must meet industry 

standards for rigor and objectivity. Rigorous research studies that establish causal impact provide the 

strongest evidence about whether a program improved the outcomes that it intended to improve. 

How can funders help build 

evidence of effectiveness?

Funders have a vested interest in 

ensuring that interventions are likely to 

help participants improve their outcomes. 

When funders make decisions about 

which applications for funding they will 

support, they could ensure that adequate 

weight is placed upon whether or not the 

funding applicant demonstrates that the 

intervention to be implemented or scaled 

has evidence of effectiveness. To foster 

knowledge about interventions with 

evidence of effectiveness, funding 

announcements could highlight 

clearinghouses that provide information 

about effective interventions, such as the 

U.S. Department of Education’s What 

Works Clearinghouse (WWC), the U.S. 

Department of Labor’s Clearinghouse for 

Labor Evaluation and Research (CLEAR),

the U.S. Department of Health and 

Human Services’ Home Visiting Evidence 

of Effectiveness (HomVEE) project and

Pathways to Work Evidence 

Clearinghouse, and the Pew Charitable 

Trusts’ Results First Clearinghouse 

Database. Funders also could consider 

dedicating portions of their funding to 

support the building of evidence about 

effectiveness, if they do not already do 

so. 
Chapter II Recommendation 1: Identify or Build Evidence of Effectiveness for the Intervention

Mathematica 9 

Funders can develop and provide information, resources, and guidelines to help organizations 

understand the standards for producing such rigorous research.

• Facilitate continuation of the cycle, as needed. If an evaluation does not demonstrate that the 

intervention is effective, the theory of change can help the funder and grantee organization identify 

why that might be the case and how the intervention can be improved. After making changes to 

improve the intervention, funders should again engage in supporting evaluations to establish 

intervention effectiveness. Research and evaluation can be viewed as part of an ongoing, cyclical 

process of service delivery, assessment, and adjustment to ensure that resources are used effectively 

in efforts to improve participants’ lives. 

B. Resources from the SEBM project 

The SEBM project developed five resources to help funders and their grantee organizations engage in 

research and evaluation to continuously improve interventions and demonstrate their effectiveness: 

“Evidence of Effectiveness in AmeriCorps-Funded Interventions” (Richman et al. 2020). This report 

summarizes three years of Mathematica assessments of AmeriCorps’ State and National and SIF grants to 

identify which AmeriCorps-funded interventions had demonstrated evidence of effectiveness, defined by 

industry standards for evaluation rigor and objectivity. 

“How to Fully Describe an Intervention” (Maxwell 2019). This practitioner’s guide was designed to 

help organizations thoroughly describe their intervention and communicate the following to potential 

funders or stakeholders: (1) the intervention model as it was designed, (2) the intervention as it is 

implemented, and (3) the parts of the intervention and the population that were evaluated and its 

effectiveness.

“What Makes for a Well-Designed, Well-Implemented Impact Study” (Anderson and Maxwell 

2019). This practitioner’s guide was designed to help organizations ensure that their evaluators produce

high quality impact studies that provide evidence about whether an intervention is effective and whether it 

is worth scaling to reach more people.

“Baseline Equivalence: What It Is and Why It Is Needed” (Anderson and Maxwell 2018). This 

practitioner’s guide was designed to help organizations work with researchers to design an impact study 

with baseline equivalence—that is, a study with two groups of individuals that are (from a statistical 

perspective) identical to each other except that one group, a treatment group, can receive the intervention, 

whereas the other group, a comparison group, cannot.

“Making the Most of Data” (Friend and Needels 2019). This practitioner’s guide highlights issues for 

organizations to consider when using, collecting, and managing their data.
This page has been left blank for double-sided copying. 
Chapter III Recommendation 2: Ensure the Intervention is Ready to Scale

Mathematica 11

III. Recommendation 2: Ensure the Intervention is Ready to Scale 

Implementation science literature points to three 

conditions that an intervention must have to enable 

it to be scaled with fidelity—that is, with strong 

correspondence between the intervention as 

intended and as implemented. First, the intervention 

needs to have its core activities clearly specified so 

they can be implemented in a manner that will 

achieve beneficial impacts on participants’ 

outcomes that are at least as large as those 

identified through prior rigorous research about the 

intervention’s effectiveness. This first condition—

specificity of the intervention activities—should be 

accompanied by a clear description of the target 

population to be served by the intervention (the 

second condition) and by the supports needed to 

ensure that the intervention is implemented with 

fidelity (the third condition). 

A. What needs to happen

Funders should critically assess whether an 

intervention is ready for scaling before scaling it. 

Doing so can help with strategic planning, such as 

making funding decisions or understanding what 

additional supports and resources grantee 

organizations may need to prepare for scaling. 

Specifically, they should ascertain whether the 

grantee organization’s intervention is wellspecified, has a well-defined target population, and 

has adequate implementation supports in place. 

The intervention must be well-specified. This 

includes having a clear understanding of the 

following: 

• The content of the intervention—that is, the 

services or activities that make up the 

intervention (Blase and Fixsen 2013). Clear 

specification of the core elements of an intervention can boost the likelihood of success with scaling 

efforts (Bauman et al. 1991; Dale et al. 2002; Winter and Szulanski 2001). The specification of the 

intervention’s content could follow naturally from the development of the theory of change (see 

Recommendation 1), which guides how activities are structured to change participants’ behaviors 

and, ultimately, their outcomes. 

• How different intervention services will be delivered, such as through face-to-face or electronic 

communications. There could be differences in the effects of a service based on how it is delivered 

Specifying and monitoring an 

intervention: The experience of an 

AmeriCorps grantee

The Child Abuse Prevention Council 

(CAPC), an AmeriCorps grantee, is part 

of a countywide collaborative network of 

partners (called the Collaborative) that 

implemented the Birth and Beyond (B&B) 

intervention at nine local Family Resource 

Centers (FRCs) in Sacramento, 

California. B&B has the overarching goal 

of educating and supporting parents in 

order to reduce their risk of child 

maltreatment and consists of home visits 

or group classes in parenting education, 

crisis intervention services, and enhanced 

services to support families. The 

Collaborative formed a Policy and 

Procedure Subcommittee that had the 

responsibility of creating a standard 

program manual that defined the 

intervention components and set the 

standards and procedures for 

implementing B&B with fidelity. The 

Subcommittee revisits the manual every 

year and makes updates accordingly.

B&B’s implementation fidelity is 

monitored with a comprehensive system 

where FRC supervisors conduct joint 

visits with home visitors, observe group 

classes, and perform quality assurance 

calls with a sample of families. 

Supervisors then share evaluative 

feedback with frontline personnel. 

Individual FRCs also perform additional

monitoring, such as reviewing individual 

cases or reviewing paperwork.
Chapter III Recommendation 2: Ensure the Intervention is Ready to Scale

Mathematica 12

(that is, its mode of delivery). For example, 

participants might better absorb information 

presented when an intervention service is 

conducted in a one-on-one setting, rather than 

with a group. This, in turn, could influence the 

extent to which the participants incorporate the 

information into their future decision making. 

• The intensity with which services are to be 

delivered, including for how many minutes or 

hours participants are to receive each service, 

the frequency with which they do so, and over 

what total time period they are to receive it. For 

example, an intervention might specify that 

participants are to receive a one-hour service 

twice a week over a period of six months, 

starting right after the participants’ enrollment 

in the intervention. 

• Workforce needs, including which personnel 

will deliver each service, qualifications needed 

for serving in that role, and other details such as 

participant caseloads for each of the personnel 

providing the service. The specification of the 

needs should include anyone involved in the 

implementation of the intervention’s services, 

regardless of whether they are paid personnel 

or volunteers, and regardless of their 

organizational affiliation.

• Setting for service delivery, including the 

location and venue. The location describes the 

geographic region and indicates whether the 

setting is rural or urban. The venue describes 

the place where activities occur, such as the 

service provider’s office facilities, a community center, the participants’ homes, or a school. The 

services for an intervention may take place in different venues; use of each venue could involve 

different logistical, administrative, financial, or other considerations—such as the cost of renting 

space, the time personnel would need to spend traveling to and from the space, and the effect of the 

space on participants’ ability to access services. 

• Intervention completion criteria, including the requirements for being considered a participant in 

the intervention. The criteria also should specify under what conditions a participant is considered to 

have completed the intervention, such as having achieved one or more milestones or having been in 

the program for a certain length of time.

The intervention must have a well-defined target population. This is critical to intervention readiness 

for scaling because needs vary across populations, and the intervention should be designed to serve the 

needs of a specific population (U.S. Centers for Disease Control and Prevention n.d.; Community Tool 

How can funders foster an 

intervention’s readiness for 

scaling?

Funders can take concrete steps to 

determine whether the interventions they 

fund are ready to be scaled to help 

support their grantees as they prepare to 

scale their interventions. To identify the 

extent to which an intervention is ready to 

scale, funders could use the Scaling 

Checklist: Assessing your Level of 

Evidence and Readiness (SCALER), 

which was developed for the SEBM 

project. The SCALER is a scaling 

planning tool that can help to identify the 

strengths, weaknesses, and gaps of the 

interventions that practitioners hope to 

scale, as well as identify potential ways to 

strengthen the intervention as needed. 

Funders of intervention services could 

support efforts to foster an intervention’s 

readiness for scaling in a variety of ways, 

including encouraging (or requiring) use 

of the SCALER by funded practitioners. 

Based on the SCALER results, both 

funders and practitioners could determine 

whether it is feasible to try to strengthen 

an intervention (such as through the 

provision of technical assistance). 

However, the results might suggest that 

resources and efforts could be better 

spent pursuing the use of a completely 

different intervention.
Chapter III Recommendation 2: Ensure the Intervention is Ready to Scale

Mathematica 13

Box n.d.). Defining the target population involves being clear about (1) attributes that are essential for 

people to be selected for participation in the intervention and (2) attributes that disqualify people from 

participating (Garg 2016; McElroy and Ladner 2014). 

The intervention must have supports in place to scale with fidelity. These six types of supports help 

organizations implement and scale an intervention with fidelity: (1) implementation monitoring, (2) 

performance monitoring, (3) continuous quality improvement (CQI) processes, (4) personnel training, (5) 

communication systems, and (6) data systems. As a funder, requiring or recommending that grantee 

organizations have these supports in place can help ensure that the intervention retains fidelity to its 

model over time and delivers the beneficial outcomes for participants when it is scaled (Breitenstein et al. 

2010; Fixsen et al. 2005). Having a complete and operational description for each implementation support, including a clear, comprehensive specification of its goals and purposes and the details of each key 

activity or task within it, can ensure that grantee personnel are conducting activities in a consistent way. 

Effective documentation of implementation support policies and procedures coupled with effective 

communication of them to personnel can help make sure supports are carried out completely and 

consistently. For grantee organizations, this documentation and communication can be especially 

important when there is turnover in personnel. Periodically reviewing implementation support policies 

and procedures and revising them as needed can be useful as well, to ensure that there is faithful 

adherence to them and to identify necessary changes. 

B. Resources from the SEBM project

The SEBM project developed four resources to help funders and their grantee organizations ensure that an 

intervention is ready to scale: 

“Planned Scaling Activities of AmeriCorps-Funded Organizations” (Selekman et al. 2020). This 

document presents a framework that identifies five conditions necessary to successfully scale an 

intervention, including the three that are related to intervention readiness and are the focus of this 

recommendation. 

“How to Structure Implementation Supports” (Eddins and Needels 2019). This practitioner’s guide 

was designed to help organizations set up implementation supports in a way that facilitates implementing 

an intervention with fidelity. 

“Making the Most of Data” (Friend and Needels 2019). This practitioner’s guide was designed to help 

organizations enhance their data collection capabilities and make the most of the valuable data they 

collect. It highlights factors that organizations should consider when assessing whether an existing data 

system is meeting their needs, and it provides guidance for obtaining a system or modifying an existing 

one to meet those needs.

“Scaling Checklists: Assessing your Level of Evidence and Readiness (SCALER)” (Richman et al. 

2020). In addition to providing background information that can help practitioners understand what is 

necessary for successful scaling, this document consists of a series of checklists that practitioners can use 

to identify the strengths, weaknesses, and gaps of the interventions they hope to scale. It also includes 

references to practical resources that will help practitioners identify potential ways to shore up the 

weaknesses and fill the gaps, so that their interventions are ready for scaling. 
This page has been left blank for double-sided copying 
Chapter IV Recommendation 3: Ensure the Organization is Ready to Scale the Intervention

Mathematica 15

IV. Recommendation 3: Ensure the Organization is Ready to Scale the 

Intervention

Successful scaling of an intervention requires both 

a suitable intervention that is ready to be scaled (as 

described within Recommendation 2) and an 

organization with the capacity to successfully 

implement it. Taking the intervention from plans 

and descriptions to on-the-ground reality requires 

personnel and other resources. Implementation 

science literature points to two conditions that an 

organization must have to ensure that it will be able 

to scale an intervention. The organization must 

have both an implementation infrastructure and a 

context that enable successful scaling. 

A. What needs to happen

Just as funders should ensure that interventions they 

are supporting are ready to scale, they need to 

critically assess whether grantee organizations or 

those applying for funding are likely to be 

successful when implementing or scaling an 

intervention. Using information solicited from 

grantee (and potential grantee) organizations 

through responses to grant or funding 

announcements, funders should look for 

organizations that have the following in place:

• An implementation infrastructure that can 

provide the resources necessary to deliver, 

support, and manage the intervention (Bernfeld 

2006; Fixsen et al. 2009; Klingner et al. 2003). 

Key components include sufficient funding, 

materials, physical space, and workforce, as 

well as an appropriate human resources system. 

The infrastructure should be able to support all 

aspects of intervention delivery—even the 

activities that do not pertain directly to 

engagement with participants. Identifying 

resources to conduct activities that are related 

directly to delivering services, which are likely 

to be clearly spelled out in the intervention description, could be straightforward. Other activities, 

such as indirect support and management tasks, might not be fully specified by the intervention 

description; therefore, a funder might have to require a clear description of whether resources are in 

place that are necessary behind the scenes for effective service delivery. Similarly, materials and 

physical space should include those necessary for direct service delivery to participants and those to 

Using partnerships to build an 

implementation infrastructure: The 

experience of an AmeriCorps 

grantee

To help sites implement Home Instruction 

for Parents of Preschool Youngsters 

(HIPPY)—a home visiting intervention for 

parents of young children—in Colorado, 

AmeriCorps grantee Parent Possible and 

HIPPY USA, the intervention developer, 

provide materials both physically and 

virtually. HIPPY USA provides the 

curriculum and training documents for 

home visitors and coordinators to 

facilitate intervention scaling. Since 

HIPPY USA requires sites to use its 

materials for service delivery, grantee and 

local agency personnel acquire the 

materials by purchasing copies or printing 

electronic versions from HIPPY USA’s 

online resource library. Personnel can 

also follow guidance provided in the 

online resource library to prepare 

curriculum packets for each home visitor 

to use with their families. Parent Possible 

coordinates the orders of materials—for 

each family, typically 15 to 30 packets of 

activities; a set of 9 children’s books; and 

a set of 20 manipulative shapes—from 

HIPPY USA for local agencies to use. 

Parent Possible has also adapted some 

of the intervention materials based on 

local circumstances. For example, 

because some home visitors are primarily 

Spanish speaking, materials on training 

and data collection have also been 

translated into Spanish by the grantee. As 

a result, some local agencies do not use 

the original, English-language materials.
Chapter IV Recommendation 3: Ensure the Organization is Ready to Scale the Intervention 

Mathematica 16

be used to monitor and manage service 

delivery. Space might be necessary for holding 

meetings to train personnel, communicate about 

policies and procedures, and monitor service 

delivery. Space also might be needed to store 

materials when they are not in use. 

Furthermore, the human resource system will 

need to be able to handle the selection, training, 

and supervision of all personnel, from frontline 

to managers.

• An enabling context, which means there is a

culture that supports innovation, learning, and 

improvement for an intervention. The relevant 

culture includes the organization that is leading 

the scaling efforts, its partner organizations 

(including funders), and other stakeholders in 

the community, such as political leaders, 

advocacy groups, and community members. An 

enabling context could arise as a result of an 

organization’s past efforts to address challenges 

that arose with intervention implementation (National Implementation Research Network n.d.). 

Regardless of the way in which this culture comes about, however, it is important for an

organization’s structures, roles, and functions to facilitate service delivery and beneficial impacts on 

participant outcomes. It is likely that, when operating in an enabling context, personnel implementing 

an intervention would feel supported by their organizational leaders, and they would describe their 

environment as collegial and collaborative. 

Furthermore, other organizations and stakeholders can influence whether scaling is enabled or hindered. 

Funders might want to carefully examine the partnerships that grantee organizations maintain or pursue 

while scaling an intervention. Two or more organizations can constructively partner to implement an 

intervention by leveraging each other’s organizational strengths and resources. However, problems might 

arise if the organizations’ missions and priorities are not well-aligned. Furthermore, organizations that are 

implementing an intervention can benefit immensely from the support of external stakeholders, who 

might provide funding, publicity, volunteers, or other resources. That support should not be taken for 

granted, however, because the priorities of those stakeholders might change over time. Concerted effort to 

retain, and even boost, stakeholder enthusiasm for the organization and its intervention could be valuable 

for the long-run success of an intervention.

B. Resources from the SEBM project

The SEBM project developed three resources to help funders and their grantee organizations ensure that 

they are ready to scale interventions:

“Planned Scaling Activities of AmeriCorps-Funded Organizations” (Selekman et al. 2020). This 

document presents a framework that identifies five conditions necessary to successfully scale an 

intervention, including the two that are related to organizational readiness and are the focus of this 

recommendation.

How can funders foster 

organizational readiness for 

scaling?

The actions funders could take to ensure 

that an organization is ready to scale an 

intervention overlap to some extent with 

those that will foster an intervention’s 

readiness to be scaled. For example, 

funders could encourage or require 

practitioners to use the SCALER, which 

was described as part of 

Recommendation 2. Resources and 

efforts could be directed at the aspects of 

organizational infrastructure that are 

identified as needing improvement. 

Additional actions could be taken to foster 

collaboration by organizations and other 

stakeholders within a community to 

identify and work toward a shared vision.
Chapter IV Recommendation 3: Ensure the Organization is Ready to Scale the Intervention 

Mathematica 17

“Scaling Evidence-Based Interventions: Insights from the Experiences of Three Grantees” (Needels 

et al. 2019). Based on a cross-site analysis of information primarily from site visits to three AmeriCorpsfunded grantees, this report presents insights into the approaches that grantees and their partners took to 

scaling their interventions, and the organizational and implementation aspects of how they scaled. 

“Build Organizational Capacity to Implement an Intervention” (Jones and Needels 2019). This 

practitioner’s guide was designed to help organizations build their capacity to be ready to implement their 

desired intervention effectively. The guide describes the specific components of the implementation 

infrastructure and the enabling context that an organization needs to implement a planned or current 

intervention. It also explains how organizations can develop those components based on reviewing the 

details of an intervention, and it provides a hypothetical example of the process.
This page has been left blank for double-sided copying. 
Chapter V Recommendation 4: Anticipate and Plan for Challenges

Mathematica 19

V. Recommendation 4: Anticipate and Plan for Challenges

Despite the best efforts of personnel at an 

organization that is scaling an intervention, 

challenges that risk the success of scaling are likely 

to arise because of factors outside of their control. 

The potential challenges are numerous, but 

common challenges with scaling could include 

turnover among personnel, changes in partnership 

arrangements, barriers to participant recruitment 

and enrollment, or fluctuations in funding. 

Although the details of the challenges that arise will 

be difficult to predict in advance, it may be 

beneficial for funders to help grantee organizations 

anticipate and plan for challenges to avoid or 

mitigate their effects. 

A. What needs to happen

Because of the distinctiveness of both the 

challenges that could arise during scaling and the 

resources that could be offered to tackle them, there 

is no one-size-fits-all approach for identifying and 

planning for challenges. However, it stands to 

reason that personnel at organizations that strive to 

prepare for challenges will be better positioned 

when challenges arise if they are willing to learn 

from the experiences of others and use resources

that are available to them. Funders can encourage 

grantee organizations to conduct these types of 

learning activities. Organizations might use these 

activities to supplement their own experiences 

successfully tackling past challenges—an important 

feature of the enabling context that fosters an 

organization’s preparedness to scale an intervention 

(see Recommendation 3). To anticipate and plan for 

challenges, funders might facilitate the following 

activities for their grantee organizations:

• Learn from the experiences of others. Organizations that have successfully scaled interventions 

with evidence of effectiveness have already encountered and tackled challenges, and they have 

insights about how to weigh trade-offs when making decisions about issues that are likely to arise 

during other scaling situations. Funders can facilitate such learning through offering peer learning 

exchanges (such as site visits, or connecting grantees that funders know are having similar 

challenges) or hosting cross-grantee forums where organizations have opportunities to talk with one 

another (see the box on next page for more details). The SEBM project’s process study explored the 

experiences of three AmeriCorps-funded organizations when they scaled interventions, including the 

Having a data system in place to 

help identify implementation 

issues: The experience of one 

AmeriCorps grantee

United Ways of Iowa (UWI), an 

AmeriCorps grantee, implemented the 

Reading Corps intervention in Iowa. 

Reading Corps is a standardized literacy 

intervention that provides 20 minutes of 

one-on-one, in-school reading tutoring 

each school day for students enrolled in 

pre-K through 3rd grade to help them 

achieve reading proficiency. Reading and 

Math, Inc. licenses Reading Corps to 

other organizations, such as UWI, to 

replicate it in other locations. UWI draws 

upon the existing resources that Reading 

and Math, Inc. provides, such as its 

training and technical assistance, data 

system, and intervention materials.

For example, UWI used the data system 

built and managed by Reading and Math, 

Inc. UWI personnel found the system 

easy to operate and used it to track 

students’ weekly progress assessments 

and other implementation processes, 

such as tutor caseloads and studentspecific amounts of tutoring received. 

Using this existing resource helped UWI 

identify and address implementationrelated issues. For example, UWI 

reported using the intervention data 

system to identify that some internal 

coaches (who oversee tutors at their 

schools) were not regularly observing 

tutors but were instead observing tutors 

several times in a short period of time.
Chapter V Recommendation 4: Anticipate and Plan for Challenges

Mathematica 20

challenges they faced and how they addressed 

them. For example, the organizations had to 

determine how many and what types of 

personnel should be in place as program 

implementation ramped up at new sites, 

balancing between (1) having enough personnel 

at the new sites to implement the intervention 

effectively and (2) avoiding having to support 

idle personnel as the kinks in procedures at the 

new site are being worked out. In addition, 

when one of these organizations adapted an 

intervention to serve a new target population 

with distinctive linguistic and cultural needs, its 

managers identified behind-the-scenes 

challenges, such as the need for training 

materials for personnel who were not fluent in 

English. These examples can provide rich 

illustrations of the trade-offs to consider when 

scaling, including adapting implementation 

supports, such as training, communication, and 

supervision. 

• Draw on existing resources. Funders can

share a number of resources with their grantee 

organizations to help them learn about other 

organizations’ experiences implementing and scaling interventions. If an organization is scaling an 

intervention that has been produced by an external developer, the developer may already have 

resources that could be used for guidance about common challenges and potential solutions. In fact, 

the external developer likely would want to ensure that the implementation and scaling of its 

intervention go as smoothly as possible. Regardless of whether the intervention was externally 

developed or home grown, resources such as the clearinghouses for research (see Recommendation 1) 

could be valuable for helping practitioners identify and plan for challenges that will arise with 

scaling. For example, the WWC makes practice guides available to help educators address challenges 

in their classrooms and schools (see https://ies.ed.gov/ncee/wwc/PracticeGuides). The guides are 

based on practitioners’ experiences, reviews of research, and the expert opinions of a panel of 

nationally recognized experts. 

B. Resources from the SEBM project

The SEBM project developed four documents that provide insights about the implementation experiences 

of three AmeriCorps-funded grantees. The first document presents findings from a cross-site analysis of 

information drawn primarily from site visits to these grantees, focusing on how the grantees and their 

partners approached scaling their interventions. Each of the other three documents focuses in depth on 

one of the three grantees, drawing on both the information used for the cross-site analysis and information 

gathered through telephone calls with grantee and partner personnel about a year after the site visits were 

conducted. The grantee-specific documents focus on progress toward scaling since the site visits, changes 

How can funders help address 

scaling-related challenges?

Funders can help their grantees develop 

plans to avoid or mitigate the effects of 

challenges. For example, they could 

support cross-grantee forums where 

grantees can learn from each other, also 

known as communities of practice. Such 

forums could enable organizations to 

share their experiences implementing and 

scaling interventions, including the 

challenges they faced, their strategies to 

overcome them, and the lessons they 

learned to reduce future risks. An 

intervention developer also might have an 

interest in facilitating this type of 

communication among organizations 

scaling its intervention. In addition, these 

types of forums can be used to share 

innovative ideas, which can ultimately 

lead to improvements in interventions 

and/or the implementation of them, so 

that participants’ outcomes are improved. 
Chapter V Recommendation 4: Anticipate and Plan for Challenges

Mathematica 21

to the interventions that occurred during the additional scaling efforts, challenges that arose, and strategies 

to address the challenges.

“Scaling Evidence-Based Interventions: Insights from the Experiences of Three Grantees” (Needels 

et al. 2020). This report focuses on the approaches that the three grantees included in the SEBM project’s 

process study took while scaling, and their experiences during scaling. Although the report focused on 

these three grantees, its insights might be more broadly applicable to a range of interventions and 

organizations, as well as implementation through methods other than scaling. 

“Scaling the Home Instruction for Parents of Preschool Youngsters (HIPPY) Intervention: Insights 

from the Experiences of Parent Possible” (Anderson et al. 2020). This report presents a case study of 

one of three grantees in the SEBM project process study: Parent Possible. This grantee implemented in 

Colorado the HIPPY intervention, a home visiting intervention that seeks to engage the parents of young 

children with activities designed to improve children’s development in reading, math, science, motor, and 

language skills.

“Scaling the Birth and Beyond (B&B) Intervention: Insights from the Experiences of the Child 

Abuse Prevention Council (CAPC)” (Eddins et al. 2020). This report presents a case study of one of 

three grantees in the SEBM project process study: CAPC. This grantee implemented in Sacramento, 

California, the B&B intervention, an evidence-based parenting education and support intervention 

designed to reduce child maltreatment.

“Scaling the Reading Corps Intervention: Insights from the Experiences of United Ways of Iowa” 

(Jones et al. 2020). This report presents a case study of one of three grantees in the SEBM project 

process study: United Ways of Iowa. This grantee implemented in Iowa the Reading Corps intervention, a 

standardized literacy intervention that provides one-on-one tutoring to students enrolled in pre-K through 

3rd grade to help them achieve reading proficiency.
This page has been left blank for double-sided copying. 
Chapter VI Recommendation 5: Support the Intervention and Organization During Scaling

Mathematica 23

VI. Recommendation 5: Support the Intervention and Organization 

During Scaling

Preparing to scale, which involves establishing the 

evidence of effectiveness for an intervention and 

ensuring that both the intervention and the 

organization scaling it are ready to scale, involve a 

considerable amount of time, resources, and 

devotion. Even after scaling readiness is 

established, the necessary work for scaling the 

intervention is only beginning. 

A. What needs to happen

Even when funders and their grantee organizations 

take all the necessary steps to anticipate challenges 

and develop plans accordingly, many internal and 

external challenges can arise during scaling. How 

funders and their grantee organizations navigate 

these challenges can impact whether they scale 

their interventions successfully. Both the 

intervention and the organization scaling it must 

have supports in place to help maintain their scaling 

efforts and achieve the ultimate goal of generating 

beneficial impacts on participants. Although these 

supports can take on different forms and can be 

specific to the individual needs of an organization, 

scaling support should include buy-in, 

communication, and processes to monitor how well 

scaling is going. To help support the intervention 

and organization during the scaling process, funders 

might seek to do the following:

• Help grantee organizations maintain support 

from their leadership, partners, and workforce members. Scaling an intervention does not occur 

in a vacuum. It relies on the people involved in the scaling process—from the frontline personnel who 

are delivering the intervention services to the implementation teams who oversee the training and 

monitoring of the workforce, along with organization leaders and partners who help provide the 

guidance and support for the scaling to realize its objectives. Although specific individuals within an 

organization or partners can change over time, a strong commitment to scaling the intervention must 

remain. Maintaining strong buy-in from each of these key groups of stakeholders makes the 

difference in whether scaling efforts are sustained, and funders might want to periodically check in on 

or discuss with grantee personnel whether buy-in is being maintained. For example, low engagement 

from frontline personnel can impact whether the scaled intervention is being implemented with 

fidelity, and low commitment from organization leaders could result in the organization shifting its 

priorities.

Leadership providing support and 

vision for scaling: The experience 

of one AmeriCorps grantee

The way in which the Birth and Beyond 

(B&B) intervention was scaled in 

Sacramento, California, to support local 

parents and reduce their risk of child 

maltreatment was shaped by the 

leadership at the Child Abuse Prevention 

Council (CAPC) and other network 

partner organizations (called the 

Collaborative). Grantee and frontline 

personnel reported how the Collaborative 

leadership not only strongly supported the 

need to adhere to the B&B intervention 

model and implement it with fidelity, but 

also recognized the need to add new 

components to it to secure the funding 

and political support needed for scaling. 

For example, Collaborative leaders were 

described as being a major catalyst for 

incorporating an additional evidencebased approach into B&B’s home 

visitation services. Had this change not 

been made, the Collaborative may have 

needed to close one of its nine operating 

sites due to funding changes. Instead, 

this decision made by the Collaborative 

leadership helped to maintain their 

scaling efforts with the B&B intervention.
Chapter VI Recommendation 5: Support the Intervention and Organization During Scaling

Mathematica 24

• Participate in continuous monitoring to 

understand how scaling is unfolding. Scaling 

an intervention—whether it is an organization 

serving more individuals in a preexisting 

setting or operating in new settings—requires 

an adherence to the intervention model for 

which research has demonstrated evidence of 

effectiveness. An extensive monitoring system 

with defined schedules and personnel roles 

provide the temperature gauges that can help 

organizations assess the extent to which they 

are maintaining fidelity to the intervention 

model during the scaling process. Funders can 

offer grantees technical assistance on how to 

monitor implementation fidelity as well as 

review performance monitoring metrics with 

grantees on a regular basis. If organizations are 

making adaptations to their interventions as 

they scale, such as adding new services or 

modifying existing ones in an attempt to better 

serve their target population or deliver services 

to new populations, consistent monitoring can 

help determine if these changes are being 

implemented as intended. A monitoring system 

can also foster continuous quality improvement 

with an intervention’s implementation, which 

can help sustain successful scaling efforts. 

• Ensure organizations use a data system that 

continues to meet an organization’s needs 

when scaling. Data systems are a vital 

component for tracking and measuring the 

intervention’s implementation as an 

organization scales it. During the scaling process, these data systems act as the central repository of 

information that can help monitor progress toward performance milestones, identify areas needing

continuous quality improvement, collect program evaluation data, and help report summary 

information to funders. However, the functionality that an organization may need from its data system 

might change over time as an intervention is scaled, and opportunities and challenges can arise while 

the organization addresses its changing needs. Making enhancements to an existing data system or 

selecting a new data system while scaling can help organizations capture new data, streamline internal 

processes, or improve upon access to the data to support its use at a larger scale, which can facilitate 

the intervention’s scaling. Funders can provide guidance on data systems to use, as well as resources 

and supports for how to get the most out of a data system’s capabilities. When considering data 

system changes, organizations will need to take into account the additional training and support that 

workforce members will need for using the data system, the in-house expertise and resources needed 

to make the changes, and the potential for any disruptions in how the organization monitors the 

intervention’s implementation during the data system transition. 

How can funders support 

organizations during scaling?

Funders are in a unique position to 

support the successful scaling of 

interventions by providing focused 

guidance to their funded organizations. 

Many funders require organizations to 

submit annual or semiannual reports on 

their progress and challenges; they also 

use regular meetings (for example, on a 

monthly or semimonthly basis) for status 

updates. These points of contact with 

organizations that are scaling 

interventions can be leveraged by funders 

to systematically identify areas where the 

organizations need greater supports as 

they scale (for example, technical 

assistance on measuring and monitoring 

intervention implementation fidelity, tip 

sheets on communication system best 

practices). Providing trainings to program 

officers overseeing funded grants on the 

essential elements for successfully 

scaling an intervention will enable those 

officers to provide in-depth support to 

grantees. For example, if organizations 

are using the SCALER to prepare for 

scaling their interventions, program 

officers can be trained on this tool to help 

them monitor their portfolio of grantees on 

the various intervention and organization 

conditions needed to scale an 

intervention. 
Chapter VI Recommendation 5: Support the Intervention and Organization During Scaling

Mathematica 25

• Commit to evaluations to understand current scaling and facilitate future scaling efforts. The 

ultimate aim for scaling an intervention is to maintain the level of effectiveness the intervention has 

on participant outcomes. Embracing the importance of and conducting an evaluation of an 

organization’s scaling efforts can help the funder and the organization know if they are achieving this 

objective. Funders and their grantee organizations can also use evaluations to test intervention 

adaptations, which they might pursue because of an interest in serving a new target population or 

otherwise modifying the intervention because they believe doing so could help bolster participant 

outcomes. When pursuing adaptations, funders and their grantee organizations should evaluate the 

adapted intervention to understand if it is still maintaining the same beneficial effects for its 

participants. Evaluations can also help funders and their grantees identify new ways for serving 

existing or new target populations and help build a larger body of evidence that demonstrates the 

intervention’s effectiveness. Funders can provide resources and time for organizations to conduct

evaluations, and organizations may find that commitment to evaluation can facilitate future scaling

because the generation of new, rigorous evidence can lead to additional scaling of the intervention. In 

addition to demonstrating that the intervention works, positive results from scaling an intervention

can enhance the organization’s visibility, increase community buy-in to participate in the 

intervention, and generate new funding for the intervention. Negative results can help funders and 

grantee personnel diagnose problems and adapt their efforts so that, ultimately, they are more 

effective at helping participants to have better outcomes.

B. Resources from the SEBM project

The SEBM project developed four resources that can help organizations understand what supports are 

needed for the intervention and within the organization to help maintain the organization’s scaling efforts. 

These resources have already been mentioned in previous sections and are listed again here: 

“Scaling the Home Instruction for Parents of Preschool Youngsters (HIPPY) Intervention: Insights 

from the Experiences of Parent Possible” (Anderson et al. 2020). This report presents a case study of 

one of three grantees in the SEBM project process study: Parent Possible. This grantee implemented in 

Colorado the HIPPY intervention, a home visiting intervention that seeks to engage the parents of young 

children with activities designed to improve children’s development in reading, math, science, motor, and 

language skills. The report describes the intervention and organization supports that facilitated Parent 

Possible’s scaling of the HIPPY intervention, and barriers that were experienced.

“Scaling the Birth and Beyond (B&B) Intervention: Insights from the Experiences of the Child 

Abuse Prevention Council (CAPC)” (Eddins et al. 2020). This report presents a case study of one of 

three grantees in the SEBM project process study: CAPC. This grantee implemented in Sacramento, 

California, the B&B intervention, an evidence-based parenting education and support intervention 

designed to reduce child maltreatment. The report describes the intervention and organization supports 

that facilitated CAPC’s scaling of the B&B intervention, and barriers that were experienced.

“Scaling the Reading Corps Intervention: Insights from the Experiences of United Ways of Iowa” 

(Jones et al. 2020). This report presents a case study of one of three grantees in the SEBM project 

process study: United Ways of Iowa. This grantee implemented in Iowa the Reading Corps intervention, a 

standardized literacy intervention that provides one-on-one tutoring to students enrolled in pre-K through 

3rd grade to help them achieve reading proficiency. The report describes the intervention and 

organization supports that facilitated United Ways of Iowa’s scaling of the Reading Corps intervention 

and barriers that were experienced.
Chapter VI Recommendation 5: Support the Intervention and Organization During Scaling

Mathematica 26

“Making the Most of Data” (Friend and Needels 2019). This guide was designed to help organizations 

enhance their data collection capabilities and make the most of the valuable data they collect. It highlights 

factors that organizations should consider when assessing whether an existing data system is meeting 

needs, and it provides guidance for obtaining a system or modifying an existing one to meet those needs.
References

Mathematica 27

References

AmeriCorps. “Metadata Glossary.” Washington, DC: AmeriCorps, 2016. Available at 

https://www.nationalservice.gov/sites/default/files/documents/GLOSSARYMetadata%20Categories%20Definitions%20for%20Publication.pdf. Accessed June 4, 2019.

Anderson, Mary Anne, and Nan Maxwell. “Baseline Equivalence: What It Is and Why It Is Needed.” 

Submitted to AmeriCorps by Mathematica. Chicago, IL, September 2018. Available at

https://nationalservice.gov/impact-our-nation/evidence-exchange/baseline-equivalence-what-it-andwhat-it-needed. Accessed September 14, 2020. 

Anderson, Mary Anne, and Nan Maxwell. “What Makes for a Well-Designed, Well-Implemented Impact 

Study.” Submitted AmeriCorps by Mathematica. Chicago, IL, May 2019. Available at

https://nationalservice.gov/impact-our-nation/evidence-exchange/what-makes-well-designed-wellimplemented-impact-study. Accessed September 14, 2020.

Anderson, Mary Anne, Katie Eddins, Karen Needels, and Scott Richman. “Scaling the Home Instruction 

for Parents of Preschool Youngsters (HIPPY) Intervention: Insights from the Experiences of Parent 

Possible.” Submitted to AmeriCorps by Mathematica. Washington, DC, February 2020. Available at 

https://nationalservice.gov/impact-our-nation/evidence-exchange/scaling-home-instruction-parentspreschool-youngsters-hippy. Accessed September 14, 2020.

Austin, M. J., and J. Claassen. “Impact of Organizational Change on Organizational Culture.” Journal of 

Evidence-Based Social Work, vol. 5, no. 1-2, 2008a, pp. 321–359. doi:10.1300/J394v05n01_12. 

Austin, M. J., and J. Claassen, J. “Implementing Evidence-Based Practice in Human Service 

Organizations.” Journal of Evidence-Based Social Work, vol. 5, no. 1-2, 2008b, pp. 271–293. 

doi:10.1300/J394v05n01_10. 

Bauman, L., R. Stein, and H. Ireys. “Reinventing Fidelity: The Transfer of Social Technology Among 

Settings.” American Journal of Community Psychology, vol. 19, no. 4, August 1991, pp. 619–639.

Bernfeld, G. A. “The Struggle for Treatment Integrity in a ‘Dis-integrated’ Service Delivery System.” The 

Behavior Analyst Today, vol. 7, no. 2, April 2006, pp. 188–205. http://dx.doi.org/10.1037/h0100086. 

Bertram, Rosalyn, Karen Blase, David Shern, Pat Shea, and Dean Fixsen. “Policy Research Brief: 

Implementation Opportunities and Challenges for Prevention and Promotion Initiatives.” Alexandria, 

VA: National Association of State Mental Health Program Directors (NASMHPD), 2011. Available 

at 

https://www.researchgate.net/profile/Rosalyn_Bertram/publication/233885310_Policy_Research_Bri

ef_Implementation_Opportunities_and_Challenges_for_Prevention_and_Promotion_Initiatives/links/

0912f50c9016f213b2000000.pdf. Accessed November 5, 2019. 

Blase, Karen, and Dean Fixsen. “Core Intervention Components: Identifying and Operationalizing What 

Makes Programs Work.” Washington, DC: Office of the Assistant Secretary for Planning and 

Evaluation, Office of Human Services Policy, U.S. Department of Health and Human Services, 

February 2013. 

Breitenstein, S. M., D. Gross, C. Garvey, C. Hill, L. Fogg, and B. Resnick. “Implementation Fidelity in 

Community-Based Interventions.” Research in Nursing & Health, vol. 33, no. 2, April 2010, pp. 164–

173. http://doi.org/10.1002/nur.20373. 
References

Mathematica 28

Community Tool Box. “Assessing Community Needs and Resources: Understanding and Describing the

Community.” n.d. Available at https://ctb.ku.edu/en/table-of-contents/ assessment/assessingcommunity-needs-and-resources/describe-the-community/main. Accessed March 15, 2018. 

Dale, Nan, Amy Baker, and David Racine. “Lessons Learned: What the WAY Program Can Teach Us 

About Program Replication.” Washington, DC: American Youth Policy Forum, 2002.

Eddins, Katie, and Karen Needels. “How to Structure Implementation Supports.” Submitted to 

AmeriCorps by Mathematica. Chicago, IL, September 2019. Available at 

https://nationalservice.gov/impact-our-nation/evidence-exchange/how-structure-implementationsupports. Accessed September 14, 2020.

Eddins, Katie, Mary Anne Anderson, Karen Needels, and Scott Richman. “Scaling the Birth and Beyond 

(B&B) Intervention: Insights from the experiences of the Child Abuse Prevention Council (CAPC).” 

Submitted to AmeriCorps by Mathematica. Washington, DC, February 2020. Available at 

https://nationalservice.gov/impact-our-nation/evidence-exchange/scaling-birth-and-beyond-bbintervention-insights-experiences. Accessed September 14, 2020.

Fixsen, D. L., K. A. Blase, S. F. Naoom, and F. Wallace. “Core Implementation Components.” Research 

on Social Work Practice, vol. 19, no. 5, May 2009, pp. 531–540. 

http://dx.doi.org/10.1177/1049731509335549.

Fixsen, D. L., S. F. Naoom, K. A. Blase, R. M. Friedman, and F. Wallace, F. Implementation Research: A 

Synthesis of the Literature (FMHI Publication #231). Tampa, FL: University of South Florida, Louis 

de la Parte Florida Mental Health Institute, The National Implementation Research Network, 2005. 

Available at https://nirn.fpg.unc.edu/sites/nirn.fpg.unc.edu/files/resources/NIRN-MonographFull-01-

2005.pdf. Accessed December 26, 2019.

Friend, Daniel, and Karen Needels. “Making the Most of Data.” Submitted to AmeriCorps by 

Mathematica. Chicago, IL, September 2019. Available at https://nationalservice.gov/impact-ournation/evidence-exchange/making-most-data. Accessed September 14, 2020.

Garg, R. “Methodology for Research I.” Indian Journal of Anaesthesia, vol. 60, no. 9, 2016, pp. 640–645. 

http://doi.org/10.4103/0019-5049.190619. 

JBS International. “National Service Systematic Review and Synthesis of National Service Literature.” 

Burlingame, CA: JBS International, 2015. Available at 

http://www.nationalservice.gov/sites/default/files/evidenceexchange/FR_NationalServiceSynthesisRe

port.pdf. Accessed June 4, 2019.

Jones, Christopher, and Karen Needels. “Build Organizational Capacity to Implement an Intervention.” 

Submitted to AmeriCorps by Mathematica. Chicago, IL, September 2019. Available at 

https://nationalservice.gov/impact-our-nation/evidence-exchange/build-organizational-capacityimplement-intervention. Accessed September 14, 2020.

Jones, Christopher, Daniel Friend, Mary Anne Anderson, Karen Needels, and Scott Richman. “Scaling 

the Reading Corps Intervention: Insights from the Experiences of United Ways of Iowa.” Submitted 

to AmeriCorps by Mathematica. Washington, DC, February 2020. Available at 

https://nationalservice.gov/impact-our-nation/evidence-exchange/scaling-reading-corps-interventioninsights-experiences-united. Accessed September 14, 2020.

Klingner, J., S. Ahwee, P. Pilonieta, and R. Menendez. “Barriers and Facilitators in Scaling Up ResearchBased Practices.” Exceptional Children, vol. 69, no. 4, July 2003, pp. 411–429. 
References

Mathematica 29

Larson, S. R., J. W. Dearing, and T. E. Backer. Strategies to Scale Up Social Programs: Pathways, 

Partnerships, and Fidelity. New York: The Wallace Foundation, 2017. Available at 

https://www.wallacefoundation.org/knowledge-center/pages/how-to-scale-up-social-programs-thatwork.aspx. Accessed November 28, 2018.

Maxwell, Nan. “How to Fully Describe an Intervention.” Submitted to AmeriCorps by Mathematica. 

Chicago, IL, May 2019. Available at https://nationalservice.gov/impact-our-nation/evidenceexchange/how-fully-describe-intervention. Accessed September 14, 2020.

Maxwell, Nan L., and Scott Richman. “Scaling Programs with Research Evidence and Effectiveness 

(SPREE).” Foundation Review, vol. 11, no. 3, November 2019, pp. 55–67. Available at 

https://nationalservice.gov/impact-our-nation/evidence-exchange/scaling-programs-researchevidence-and-effectiveness. Accessed September 14, 2020.

Miller, W. R., J. L. Sorensen, J. A. Selzer, and G. S. Brigham. “Disseminating Evidence-Based Practices 

in Substance Abuse Treatment: A Review with Suggestions.” Journal of Substance Abuse Treatment, 

vol. 31, no. 1, 2006, pp. 25–39.

McElroy L. M., and D. P. Ladner. “Defining the Study Cohort: Inclusion and Exclusion Criteria.” In 

Success in Academic Surgery: Clinical Trials, edited by T. Pawlik and J. Sosa. London: Springer, 

2014. 

National Implementation Research Network. “The Hexagon: An Exploration Tool.” August 2018. 

Available at 

https://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/resources/NIRN_Hexagon

Tool_11.2.18.pdf. Accessed January 17, 2019.

National Implementation Research Network. “Active Implementation Hub.” n.d. Available at 

https://nirn.fpg.unc.edu/module-2/implementation-drivers. Accessed April 30, 2020. 

Needels, Karen, Daniel Friend, Scott Richman, Katie Eddins, and Christopher Jones. “Scaling EvidenceBased Interventions: Insights from the Experiences of Three Grantees.” Submitted to AmeriCorps by 

Mathematica. Chicago, IL, September 2019. Available at https://nationalservice.gov/impact-ournation/evidence-exchange/scaling-evidence-based-interventions-insights-experiences-three. Accessed

September 14, 2020. 

Richman, Scott, and Andrei Streke. “Evidence of Effectiveness in AmeriCorps-Funded Interventions.” 

Submitted to AmeriCorps by Mathematica. Chicago, IL, September 2020. Available at 

https://americorps.gov/evidence-exchange/evidence-effectiveness-americorps-funded-interventions. 

Accessed April 6, 2021.

Richman, Scott, Nan Maxwell, Karen Needels, and Mary Anne Anderson. “Scaling Checklists: Assessing 

your Level of Evidence and Readiness (SCALER).” Submitted to AmeriCorps by Mathematica. 

Chicago, IL, September 2020. Available at https://americorps.gov/evidence-exchange/scalingchecklists-assessing-your-level-evidence-readiness-scaler. Accessed April 6, 2021.

Selekman, Rebekah, Katie Eddins, Natalie Reid, and Scott Richman. “Planned Scaling Activities of 

AmeriCorps-Funded Organizations.” Submitted to AmeriCorps by Mathematica. Chicago, IL, 

September 2020. Available at https://americorps.gov/evidence-exchange/planned-scaling-activitiesamericorps-funded-organizations. Accessed April 6, 2021.

U.S. Centers for Disease Control and Prevention. “Chapter One: Planning, Implementing, and Evaluating 

an Intervention—An Overview.” n.d. Available at 

https://stacks.cdc.gov/view/cdc/27357/cdc_27357_DS1.pdf. Accessed March 15, 2018.
References

Mathematica 30

U.S. Government Accountability Office. “Tiered Evidence Grants: Opportunities Exist to Share Lessons 

from Early Implementation and Inform Future Federal Efforts.” GAO-16-818. Washington, DC: U.S. 

Government Accountability Office, September 2016.

Winter, S., and G. Szulanski. “Replication as Strategy.” Organization Science, vol. 12, no. 6, November 

2001, pp. 730–743.

Zhang, X., and J. Sun. “Meta-Analysis of Evaluations Across the Social Innovation Fund Program: Final 

Report.” Prepared for AmeriCorps, Office of Research and Evaluation. Fairfax, VA: ICF 

International, 2016. Available at 

https://www.nationalservice.gov/sites/default/files/documents/FR_SIF_MetaAnalysis_Final_Report.pdf. Accessed June 4, 2019.
This page has been left blank for double-sided copying. 
Mathematica

Princeton, NJ • Ann Arbor, MI • Cambridge, MA 

Chicago, IL • Oakland, CA • Seattle, WA

Tucson, AZ • Woodlawn, MD • Washington, DC 

EDI Global, a Mathematica Company

Bukoba, Tanzania • High Wycombe, United Kingdom

mathematica.org