Planned Scaling Activities of AmeriCorpsFunded Organizations

September 25, 2020

Rebekah Selekman, Katie Eddins, Natalie Reid, and Scott Richman

Submitted to:

AmeriCorps

Office of Research and Evaluation 

250 E Street, SW

Washington, DC 20525

Project Officer: Lily Zandniapour

Contract Number: GS10F0050L/CNSHQ16F0049

Submitted by:

Mathematica

111 East Wacker Drive

Suite 920

Chicago, IL 60601

Telephone: (312) 994-1002

Facsimile: (312) 994-1003

Project Director: Scott Richman

Reference Number: 50332
This page has been left blank for double-sided copying.
Planned Scaling Activities Report Mathematica

iii

ACKNOWLEDGMENTS 

The authors acknowledge several people who contributed to and supported this report funded by 

AmeriCorps. At AmeriCorps, Lily Zandniapour, Anthony Nerino, Roshni Menon, Sarah Yue, 

Mary Hyde, and Katy Hussey-Sloniker provided advice and direction that greatly enhanced this 

report. We appreciate their thoughtful insights into AmeriCorps programs and their comments on 

our products. We also thank the following study team members at Mathematica and its partner, 

Twin Peaks Partners, LLC, for their assistance in reviewing and extracting information from 

documents that were used as the source of the data analyzed in this report: Annie Buonaspina, 

Kylie Cannon, Karina Edouard, Alix Gould-Werth, Chris Jones, Ellen Kisker (Twin Peaks 

Partners LLC), Rebecca Kleinman, Hande Inanc Lenowitz, Nan Maxwell, Ryan Ruggiero, Noa 

Sager, Jill Spielfogel, and Mathew Stange. In addition, Karen Needels, Patricia Del Grosso, and 

Samina Sattar provided feedback that greatly enhanced the report. Colleen Fitts helped prepare 

it, and Larisa Wiseman provided editorial assistance. 
This page has been left blank for double-sided copying.
Planned Scaling Activities Report Mathematica

v 

Contents 

ACKNOWLEDGMENTS................................................................................................................................iii

EXECUTIVE SUMMARY ..............................................................................................................................ix

I. INTRODUCTION................................................................................................................................... 1

A. How do we know whether interventions are ready to be scaled? ................................................. 2

B. How we determined whether organizations and their interventions were ready to scale ............. 6

1. Research questions ............................................................................................................... 6

2. Overview of organizations being examined........................................................................... 7

3. Analysis approach ................................................................................................................. 8

C. Road map for the report .............................................................................................................. 10

II. FINDINGS ON INTERVENTION READINESS................................................................................... 11

A. Well-specified intervention .......................................................................................................... 11

B. Well-defined target population..................................................................................................... 14

C. Modification to the intervention core elements and target population......................................... 14

D. Implementation supports ............................................................................................................. 18

III. KEY FINDINGS ON ORGANIZATIONAL READINESS...................................................................... 21

A. Enabling context .......................................................................................................................... 21

B. Implementation infrastructure...................................................................................................... 24

IV. OVERALL READINESS FOR SCALING ............................................................................................ 27

A. Scaling readiness based on scaling readiness framework ......................................................... 27

1. Intervention readiness ......................................................................................................... 27

2. Organizational readiness..................................................................................................... 28

3. Overall scaling readiness .................................................................................................... 29

B. Scaling readiness based on AmeriCorps reporting requirements............................................... 29

C. Conclusion................................................................................................................................... 32

REFERENCES............................................................................................................................................ 33

APPENDIX A: SCALING PLAN REVIEW PROCESS............................................................................A.1

A. Documents for review.................................................................................................................A.3

B. Study sample and unit of analysis..............................................................................................A.6

C. Review methodology and analysis strategy ...............................................................................A.8

D. Scaling readiness over time .....................................................................................................A.12
Planned Scaling Activities Report Mathematica

vi

APPENDIX B: RUBRIC FOR INTERVENTION AND SCALING ACTIVITIES ........................................B.1

I. Intervention Review Section...............................................................................................................B.3

A. Context for the intervention ........................................................................................................B.3

B. Well-specified intervention .........................................................................................................B.5

C. Well-defined target population..................................................................................................B.11

D. Implementation supports ..........................................................................................................B.13

E. Enabling context .......................................................................................................................B.20

F. Implementation infrastructure...................................................................................................B.22

II. SCALING READINESS SECTION...................................................................................................B.23

A. Scaling approach and its context .............................................................................................B.24

B. Well-specified intervention .......................................................................................................B.25

C. Well-defined target population..................................................................................................B.30

D. Implementation supports ..........................................................................................................B.30

E. Enabling context .......................................................................................................................B.33

F. Implementation infrastructure...................................................................................................B.35
Planned Scaling Activities Report Mathematica

vii

Tables 

I.1 Research questions for the readiness for scaling analysis.............................................................. 6 

I.2 Alignment of scaling readiness framework and AmeriCorps State and National NOFO 

and SIF Growth Plan requirements................................................................................................10

IV.1 Description of intervention readiness conditions ...........................................................................28

IV.2 Description of organizational readiness conditions........................................................................28

IV.3 Description of all scaling readiness conditions ..............................................................................29

IV.4 Intervention readiness using AmeriCorps reporting requirements ................................................30

IV.5 Organizational readiness using AmeriCorps reporting requirements ............................................31

IV.6 Overall scaling readiness using AmeriCorps reporting requirements............................................31

Figures 

I.1 The Scaling Programs with Research Evidence and Effectiveness (SPREE) 

process............................................................................................................................................. 2

I.2 Necessary conditions for successful scaling of interventions .......................................................... 4

II.1 Percentage of organizations that provided a full, a limited, or no description of 

each core element dimension ........................................................................................................ 13

II.2 Example of differences in planned and evaluated interventions and rationale for 

change............................................................................................................................................ 15

II.3 Percentage of organizations that planned core element dimensions and target 

population that differed from those in the evaluated interventions ................................................ 16

II.4 Percentage of organizations that provided rationale for differences between the 

planned and evaluated core element dimensions and target population ...................................... 17

II.5 Percentage of organizations that provided a full, a limited, or no description of 

each implementation support......................................................................................................... 20

III.1 Percentage of organizations that provided a full, a limited, or no description of 

each component of an enabling context ........................................................................................ 23

III.2 Percentage of organizations that provided a full, a limited, or no description of 

each implementation infrastructure component............................................................................. 26
This page has been left blank for double-sided copying.
Planned Scaling Activities Report Mathematica

ix

EXECUTIVE SUMMARY

Agencies throughout the federal government and other grant-making institutions are 

increasingly conducting systematic evidence reviews and synthesizing the results of wellexecuted research to identify interventions that have evidence of effectiveness. These efforts 

can help funders make strategic decisions on how they can most effectively and efficiently 

address individual and community needs. These reviews and syntheses can also enhance 

funders’ decisions about scaling up specific interventions that show the most promise in order to 

expand the scope of their impact.

AmeriCorps invests significant resources in interventions designed to improve lives and 

strengthen communities.

1 AmeriCorps and its funded organizations also invest resources in 

evaluating the effectiveness of these interventions. To gain insights about which of its funded 

interventions with evidence to improve participant outcomes might be ready to scale, the agency 

funded the project known as Scaling Evidence-Based Models in 2016. Conducted by 

Mathematica, this project is designed to deepen the agency’s understanding of the most 

effective interventions it has funded and its knowledge base on scaling them. The project also is 

designed to generate both practical knowledge about how AmeriCorps and other funders can 

successfully scale effective interventions. The project can be thought of as having two parts. 

The first, which is discussed in the companion report “Evidence of Effectiveness in AmeriCorpsFunded Interventions” (Richman and Streke 2020), is to identify AmeriCorps-funded 

interventions with evidence of effectiveness—in other words, those supported by rigorous 

research to improve outcomes for participants. The second, the focus of this report, is to

examine the extent to which organizations describe the intervention and organizational factors 

that are important for being ready to scale an intervention with evidence of effectiveness. 

This report presents scaling readiness findings for a cohort of 25 organizations that provided 

plans to scale interventions with evidence of effectiveness. This report also examines variations 

in demonstrated scaling readiness across AmeriCorps funding years. 

A. Research approach and questions

Mathematica developed a framework, based on implementation science literature, that identifies 

five conditions necessary to successfully scale an intervention (Figure ES.1). We define 

successful scaling as expanding, replicating, or adapting the intervention while maintaining or 

1 As of September 29th, 2020, the Corporation for National and Community Service is operating under the name 

AmeriCorps.
Planned Scaling Activities Report Mathematica

x 

exceeding the beneficial impacts documented in evidence about its effectiveness.2 The first 

three conditions indicate whether an intervention might be ready to be successfully scaled:

Figure ES.1. Necessary conditions for successful scaling of interventions

• A well-specified intervention, consisting of a description of the content, mode of service 

delivery, intensity, personnel needs,

3 and setting for each core element. A well-specified 

intervention also includes a definition of participation in and completion of the intervention.

• A well-defined target population, consisting of a description of the population for which the 

intervention was found to be effective. 

• Implementation supports, consisting of a description of supports that can ensure fidelity, 

such as have an implementation monitoring team and performance benchmarks. 

Implementation supports also include a description of the procedures for putting the 

supports into action, such as describing the processes the monitoring team follows and a 

process for measuring performance benchmarks.

2 Some AmeriCorps-funded organizations planned to sustain or deepen intervention services. Because the goal of 

the analysis is to provide insights about the readiness for successful scaling, we included organizations that had 

funding for sustaining and/or deepening services and no explicit plans for scaling. Chapter I defines the five 

scaling approaches proposed by AmeriCorps-funded organizations. 

3 We use the term personnel to refer to organization and partner staff as well as AmeriCorps members who deliver 

intervention services.
Planned Scaling Activities Report Mathematica

xi

The final two conditions indicate that an organization might be ready to scale an intervention:

• Enabling context, consisting of a description of the presence of organizational and partner 

agency leadership and culture that supports the scaling effort. Enabling context is 

demonstrated with examples of ways that the organization is innovative and has improved 

upon past interventions, particularly in the face of implementation challenges. 

• Implementation infrastructure, consisting of a description of the organizational infrastructure, 

such as the personnel, materials, and physical space that support implementation.

In light of the framework, Mathematica developed three 

research questions about the scaling readiness of the 

interventions with evidence of effectiveness and the 

organizations that implemented them (see sidebar). The first 

question pertains to the three conditions about intervention 

readiness in Figure ES.1. The second question pertains to the 

two conditions about organizational readiness in Figure ES.1. 

The third research question pertains to all five conditions in 

Figure ES.1, and this question reflects a synthesis of the first 

and second questions. 

Research questions

• How do organizations

demonstrate that their 

interventions are ready 

for scaling?

• How do organizations 

demonstrate they are 

ready for scaling 

interventions?

• How do organizations 

demonstrate overall 

readiness for scaling?

To answer the questions, we developed a two-part systematic 

review process to collect and record information about the 

intervention and the extent to which organizations’ scaling 

plans describe the intervention and organization conditions 

that are necessary for successful scaling. The sources of 

information for this review process were the funding 

applications submitted by AmeriCorps State and National

organizations and the plans for growing suborganization impact submitted by SIF organizations 

(referred to collectively as “scaling plans”), as well as the documents that provided evidence of 

the intervention’s effectiveness.4 It is important to note that the information provided in these 

funding applications and plans for growing suborganization impact were in response to existing 

reporting requirements set forth by the AmeriCorps State and National and SIF program offices 

that organizations needed to address. Although these existing AmeriCorps requirements asked 

organizations to provide information that aligned to many aspects of the scaling framework, 

there were also some important differences. For example, the framework developed for this 

study requires specification of more details related to organizational readiness for scaling than 

required by AmeriCorps. Readers of this document should consider these differences, as we 

applied our systematic review process to these documents based on the scaling framework 

after the fact.

4 Appendix A provides more details about the process of identifying documents for review and the process of 

extracting information from these documents.
Planned Scaling Activities Report Mathematica

xii

The analysis sample for this report includes 25 organizations that submitted scaling plans for 

interventions that received a strong or moderate evidence rating from another independent, 

third-party evaluator contracted by AmeriCorps. For each scaling plan, we rated the level of 

specificity provided for each necessary condition for successful scaling on a three-point scale: 

(1) a full description, (2) a limited description, or (3) no description. We conducted a tabular 

analysis of the information from the organizations’ plans for each condition for successful 

scaling.

B. Research findings and their implications

Our analysis shows that organizations provided greater specification of the intervention

readiness conditions for scaling (28 percent) than of the organization readiness conditions for 

scaling (4 percent) in their scaling plans. However, no organization described all the conditions 

of intervention and organizational readiness. Our findings of the extent to which organizations 

specify various conditions of scaling readiness generally align with what AmeriCorps requires 

organizations to specify. For instance, the most commonly described conditions were the target 

population (described by 100 percent of organizations) and the intervention core elements 

(described by 56 percent of organizations), and the least commonly described conditions were 

enabling context of the organization (described by 12 percent of organizations) and 

implementation infrastructure (specified by 36 percent of organizations). However, we do find 

some instances in which organizations are required to specify specific elements of a condition, 

but those elements are not actually well specified by organizations in their documents, such as 

specifying the personnel needs of the intervention. When we analyze scaling readiness 

according to the information AmeriCorps requested organizations to specify, we find that 

specification of scaling readiness conditions increases. Thus, the extent to which organizations 

demonstrate scaling readiness, based on the level of specification in the scaling plans, may 

reflect AmeriCorps requirements rather than the organizations’ readiness for scaling.

Organizations may have a higher degree of scaling readiness than what was detectable through 

a review of scaling plans. 

C. Conclusion

Through this project, we have assessed the degree to which AmeriCorps organizations have 

described the conditions for scaling readiness. The analysis of scaling readiness identified those 

areas in which organizations specified critical scaling readiness components as well as those 

areas that would have benefited from more detail. The framework developed to guide this 

assessment is based on the necessary elements for scaling readiness as identified in the 

implementation science literature. The use of such a framework is applicable to understanding 

scaling readiness for other organizations beyond AmeriCorps. Our study found that, generally, 

organizations report what they are asked to report. Thus, funders and grant makers can use the 

scaling framework to inform their reporting requirements, such that their requirements will align 

to the different conditions that organizations should speak to demonstrate their readiness to 

scale. Additionally, our study finds that even among those conditions related to scaling that 
Planned Scaling Activities Report Mathematica

xiii

AmeriCorps organizations were already asked to report, some areas would have benefited from

more detail. Thus, our framework can help funders provide more guidance on the details they 

are looking for. In the end, aligning reporting requirements with the conditions specified in our 

framework may foster more successful scaling efforts and more efficient use of funds.
This page has been left blank for double-sided copying.
Planned Scaling Activities Report Mathematica

1 

I. INTRODUCTION 

Agencies throughout the federal government and other grant-making institutions are increasingly 

conducting systematic evidence reviews and synthesizing the results of well-executed research to 

identify interventions that have evidence of effectiveness. These efforts can help funders make 

strategic decisions on how they can most effectively and efficiently address individual and 

community needs. These evidence reviews can also enhance funders’ decisions about scaling up 

specific interventions that show the most promise in order to expand the scope of their impact. 

AmeriCorps was established as a federal agency in 1993 with a mission to improve lives, 

strengthen communities, and foster civic engagement through service and volunteering.5 The 

AmeriCorps State and National program and three AmeriCorps Seniors programs (RSVP, Senior 

Companions, and Foster Grandparent) are the main avenues through which AmeriCorps achieves 

this mission.6 In 2020, an estimated 75,000 AmeriCorps members were located at more than 

21,000 sites across the country. In addition, from 2010 to 2016 the Social Innovation Fund (SIF) 

supported the agency’s mission through funding innovative community-based programs to 

address the most challenging social problems communities face. These programs fund the 

implementation of interventions, which are combinations of activities or services designed to 

change knowledge, attitudes, or behaviors or improve conditions for specific individuals or 

groups. The types of organizations that can receive AmeriCorps State and National and SIF 

funding are diverse, but they include public or private nonprofit organizations, including faithbased and other community organizations; institutions of higher education; government entities 

within states or territories, such as cities or counties; partnerships, consortia, and intermediary 

organizations; and Indian Tribes. 

Because AmeriCorps strives to be a good steward of taxpayer dollars and to operate programs 

cost effectively (AmeriCorps 2017), it funded the Scaling Evidence-Based Models project in 

2016. This project seeks to shed light on how AmeriCorps organizations approach program 

implementation and scaling and assesses the level of scaling readiness of AmeriCorps’ State and 

National funded grantees 2015–2019 and SIF 2010–2015 interventions with demonstrated 

evidence of effectiveness. Conducted by Mathematica, this project is designed to deepen the 

agency’s understanding of the most effective interventions it has funded and its knowledge base 

on scaling them. The project also is designed to generate both practical knowledge about how 

AmeriCorps could successfully scale effective interventions and recommendations for going 

forward. The research in this project will help AmeriCorps find which of its funded interventions 

improve participant outcomes and make them work for more people.7

5 As of September 29th, 2020, the Corporation for National and Community Service is operating under the name 

AmeriCorps.

6 Because the AmeriCorps Seniors programs were not part of the analysis in this report, we do not discuss them. 

AmeriCorps (2016) provides a more detailed description of AmeriCorps’ programs. AmeriCorps’ focus areas 

include disaster relief services, economic opportunity, education, environmental stewardship, healthy futures, 

nonprofit organizational capacity, and veterans and military families.

7 See https://www.nationalservice.gov/programs/social-innovation-fund for additional information.
Planned Scaling Activities Report Mathematica

2 

Since 2016, AmeriCorps has applied a process called Scaling Programs with Research Evidence 

and Effectiveness (SPREE) to help AmeriCorps identify which of its funded interventions 

demonstrate evidence of effectiveness and a readiness to scale (Maxwell and Richman 2019). 

SPREE is a two-part process that (1) identifies interventions that are most likely to achieve 

desired outcomes and (2) identifies which effective interventions demonstrate a readiness for 

scaling and which organizations might be ready to scale them. Figure I.1 presents the SPREE 

process. The first part of the process, identifying effective interventions, is discussed in the 

companion report, “Evidence of Effectiveness in AmeriCorps-Funded Interventions” (Richman 

and Streke 2020). The second part of the process is the focus of this report. 

Figure I.1. The Scaling Programs with Research Evidence and Effectiveness (SPREE)

process

A. How do we know whether interventions are ready to be scaled?

AmeriCorps-funded organizations used three main approaches to scaling their interventions so 

more people could receive the beneficial impacts of an intervention:

• Expansion: Extending the intervention to more people in the same target population and 

location as the intervention had previously been offered. 

• Replication: Extending the intervention to the same target population but in a new location. 

Replication is about creating a new implementation infrastructure, whereas expansion is 

about increasing the capacity of an existing infrastructure. 

• Adaptation: Modifying an existing intervention to serve a new target population or to 

implement it in a new setting (type of location). 
Planned Scaling Activities Report Mathematica

3 

Successful scaling uses at least one, and possibly 

more, of these approaches while maintaining or 

surpassing the beneficial impacts of the intervention 

that have been documented to have evidence of 

effectiveness.

AmeriCorps also has funded other organizations 

that planned to implement interventions with 

evidence of effectiveness through methods other 

than scaling. Some planned to sustain services that 

have previously been provided; that is, they planned 

to serve the same population in the same location 

without making purposeful changes to the 

intervention. Others intended to deepen services to 

serve the same target population in the same 

location with enhanced services of the same 

intervention (for example, more hours of a specific 

service component). 

Examples of types

of scaling

Expansion

• Increasing the number of 

unemployed adults served at a 

work center by hiring five more job 

search specialists who will each 

serve 20 more adults

Replication

• Implementing a reading program 

designed for 5th graders in a new 

school district, city, and state, but 

serving the same target population

Adaptation

• Adapting a parent training 

curriculum designed for mothers 

to include language that is more 

inclusive of fathers

Regardless of the scaling approach used, our 

framework for successfully scaling an intervention 

with evidence of effectiveness identifies five 

conditions that indicate that an organization is ready 

to scale an intervention (Figure I.2). Though 

AmeriCorps provides funding to many organizations, we prioritize inclusion of interventions 

with evidence of effectiveness because it is these interventions for which there is an 

implementation model known to achieve positive impacts and to which organizations should 

sustain fidelity to during scale up. The conditions in the framework were drawn from 

implementation science literature. The first three conditions indicate that the intervention is 

ready for scaling because it is clearly organized and well designed—to generate beneficial 

participant outcomes, interventions are designed to be implemented in a standardized way (using 

an “intervention model”): 

• It is a well-specified intervention if it makes clear what set of activities are critical for 

achieving good outcomes for the intervention’s participants, although the intervention can be 

implemented with some flexibility if it adheres to the principles underlying each core element 

(Fixsen et al. 2005).8 Implementation science finds that for each core element of an 

intervention, there should be a description of the features necessary to produce the intended 

outcomes (Blase and Fixsen 2013). A well-specified intervention will, for each core element, 

describe the content of the intervention, the mode of service delivery, the intensity, personnel 

8 Core elements of an intervention are the pieces of an intervention that are central to achieving program outcomes 

and are often seen as the elements that define a program. For example, a reading intervention may deliver 

intensive one-on-one tutoring using a specific reading curriculum in the classroom whereas a non-core, or 

supplemental, activity could be disseminating monthly newsletters to parents or caregivers with book 

recommendations that could be read to children during non-school hours. 
Planned Scaling Activities Report Mathematica

4 

needs, and setting. This condition also includes a definition of participation in and 

completion of the intervention, as defined by the organization and is specific to each 

intervention. 

Figure I.2. Necessary conditions for successful scaling of interventions

Having a clear understanding of the content of the intervention, the mode of service delivery, 

the intensity, personnel needs, setting, and criteria for completion can contribute to 

implementing an intervention with fidelity—that is, with strong correspondence between the 

intervention as intended and as implemented—and to increasing the likelihood of successful 

scaling. Interventions with evidence of effectiveness implemented without fidelity might not 

achieve the improvements in participant outcomes that could be expected based on the 

intervention’s prior success. For instance, a description of exactly how the core elements of 

the intervention will be delivered (the mode of service delivery) is central to ensuring that 

services are delivered as intended. A description of the qualifications for those who will 

deliver the intervention (personnel needs) ensures that there are qualified individuals who are 

able to deliver services with fidelity. A clear definition of criteria needed for intervention 

participation and completion is also critical to ensuring that participants receive the necessary 

amount of intervention services to exhibit impacts. For example, a program participant can 

be defined as someone who completes the program orientation or receives a minimum 

number of services, whereas program completion may be defined as those who receive all 

services or achieve a specific benchmark. The absence of such specifications may indicate 

that an organization has not considered how to implement a particular dimension so that it is 

done consistently for the duration of the intervention and across locations. Additionally, the 

intervention might not improve participants’ outcomes to the extent expected, given the 

intervention’s prior success. 
Planned Scaling Activities Report Mathematica

5 

• It has a well-defined target population, which is critical to intervention readiness for scaling 

because an intervention should be designed to serve the needs of a specific population (U.S. 

Centers for Disease Control and Prevention n.d.; Community Tool Box n.d.). The 

organization needs to be clear about inclusion and exclusion criteria: inclusion criteria are 

attributes that are essential for potential participants to be selected for participation in an 

intervention, whereas exclusion criteria are attributes that disqualify people from 

participating in an intervention (Garg 2016; McElroy and Ladner 2014). 

In some instances, organizations may first identify a population in need and then develop or 

identify an intervention that addresses those needs. In other instances, organizations may first 

develop or identify an intervention to serve a particular need and then identify the target 

population that is appropriate for the intervention. In either scenario, thoroughly describing 

the characteristics of the population receiving the intervention helps ensure that (1) the 

proper services are in place for successful service delivery and (2) the target population is 

appropriate for the intervention. For example, an employment preparation program designed 

for youth ages 14 to 16 with learning disabilities might require different services than an 

employment preparation program designed for youth ages 14 to 16 without learning 

disabilities. This condition was unchanged in the refined scaling readiness measure. 

• Implementation supports are in place so the intervention can be implemented as intended 

(that is, it retains fidelity to its model). Implementation supports include a team that monitors 

implementation, continuous quality improvement processes, and personnel training 

requirements. Implementation supports also include procedures for putting the supports into 

action, such as describing a process for monitoring implementation fidelity that an 

implementation monitoring team will follow. These supports are derived from best practices 

and commonalities observed among successfully implemented interventions, and they can 

help ensure that the intervention retains fidelity to its model and delivers the beneficial 

outcomes for participants when it is scaled (Breitenstein et al. 2010; National Implementation 

Research Network n.d.). In a refined scaling readiness measure, we broke down these 

supports into finer details so that we could isolate nuances of the supports. Additional 

nuances that we can isolate with the revised measure include procedures for putting 

implementation supports into action and analyzing data for implementation supports. The 

specific additions are discussed in more detail in Chapter II. 

The next two conditions indicate what an organization must have in place to be ready to scale an 

intervention: 

• An enabling context in which the organization’s leadership and culture support innovation, 

learning, and improvement for an intervention. Although this context develops in different 

ways, a common one is through having successfully tackled challenges in the past. The 

organization’s structures, roles, and functions facilitate (rather than hinder) service delivery 

as well as the achievement of beneficial participant outcomes. We expanded our measure of 

this condition to include a description of the support that organizational and partner agency 

leadership provide for scaling. We also broke out innovation and improvement so that we can 

isolate each separately.
Planned Scaling Activities Report Mathematica

6 

• An implementation infrastructure to support scaling the intervention. The organization’s 

infrastructure must permit the organization to work with both internal and external systems to 

ensure that it has sufficient resources to support the intervention (Bernfeld 2006; Fixsen et al. 

2009; Klingner et al. 2003). Infrastructure requirements include sufficient resources, such as 

financial resources; personnel who will deliver the intervention; materials required to deliver 

services; and a human resource system for hiring and managing personnel. A refined scaling 

measure includes the presence of physical space for implementation.

B. How we determined whether organizations and their interventions were 

ready to scale

We used the framework presented in Figure I.2 to determine whether AmeriCorps-funded 

organizations have provided a level of specification that indicates that they and the interventions 

they have planned to implement are ready for scaling. In this section, we discuss the research 

questions that guide our analysis, the organizations included in the analysis, and the 

methodology we used. We also provide guidance for how to interpret the findings. 

1. Research questions

Our analysis is structured to answer three research questions and related subquestions (Table 

I.1). The first question pertains to the three conditions about intervention readiness, shown in 

Figure I.2. The second question pertains to the two conditions about organizational readiness, 

shown in Figure I.2. The third research question pertains to all five conditions, shown in Figure 

I.2; this question reflects a synthesis of the first and second research questions. 

Table I.1. Research questions for the readiness for scaling analysis

Research questions

1. How do organizations demonstrate that their interventions are ready for scaling?

1a. To what extent do organizations describe their proposed intervention? 

1b. To what extent do organizations describe their proposed target population?

1c. When organizations propose changes to the proposed intervention and/or target population as part of their plans 

for scaling, to what extent do they provide a rationale for doing so?

1d. To what extent do organizations describe supports they put in place to foster implementation?

2. How do organizations demonstrate they are ready for scaling interventions?

2a. To what extent do organizations describe their past experience successfully implementing interventions? 

2b. To what extent do organizations describe their infrastructure for supporting implementation?

3. How do organizations demonstrate overall readiness for scaling?

3a. To what extent do organizations demonstrate that their intervention meets all the conditions for scaling? 

3b. To what extent do organizations demonstrate that they meet all the conditions for scaling? 

3c. To what extent do organizations demonstrate that both they and their interventions jointly meet all the conditions 

for scaling?
Planned Scaling Activities Report Mathematica

7 

2. Overview of organizations being examined 

To answer the research questions, this analysis builds upon a companion meta-synthesis that

provides details about the identification of interventions that have evidence of effectiveness of 

improving participants’ outcomes (Richman and Streke 2020). In this report, we present findings 

for 25 organizations that planned to implement interventions identified as having evidence of 

effectiveness.

9 Because the goal of the analysis is to provide insights about the readiness for 

successful scaling, we included organizations that had funding for sustaining or deepening 

services and no explicit plans for scaling. It is important to note that the information provided in 

these funding applications and plans for growing suborganization impact were in response to 

existing reporting requirements set forth by the AmeriCorps State and National and SIF program 

offices that organizations needed to address. Although these existing AmeriCorps requirements 

asked organizations to provide information that aligned to many aspects of the scaling 

framework, there were also some important differences. For example, the framework developed 

for this study requires specification of more details related to organizational readiness for scaling 

than required by AmeriCorps. Readers of this document should consider these differences, as we 

applied our systematic review process to these documents based on the scaling framework after 

the fact. 

Among the organizations included in this analysis, 20 received AmeriCorps State and National 

grants during 2015–2019. Of the 20 State and National grantees that are included in our analysis, 

8 received funding in 2015, 8 received funding in 2016, 5 received funding in 2017, 3 received 

funding in 2018, and 11 received funding in 2019. The number of organizations receiving 

funding exceeds the number of organizations in the analysis because 9 organizations received 

funding in multiple funding years. The remaining 5 organizations included in our analysis 

received SIF grants during 2010–2015. Of the 5 SIF organizations included in our analysis, 3 

received funding in 2010, 1 received funding in 2011, 1 received funding in 2012, and 1 received 

funding in 2014. The number of organizations that received funding exceeds the number of 

organizations included in the analysis because 1 organization received funding in multiple 

funding years. Seventeen organizations (68 percent) planned to implement education-related 

interventions. Five organizations (20 percent) planned to scale interventions with a focus on 

providing services to help participants to have healthy futures, and 5 (20 percent) planned to 

implement an intervention focused on helping participants with economic opportunities.10 The 

organizations sought to engage low-income and at-risk populations for services. Because most 

interventions related to education, the target population was most commonly children and their 

families. Among the organizations, there were plans to implement the interventions in 

communities across the nation and in both urban and rural areas. Some organizations focused on 

particular cities or states, whereas others planned to scale in communities across a region or the 

country. 

9 Appendix A describes the approach used to select the interventions for this study, as well as what we mean by an 

“organization” for the purposes of this report. 

10 The percentages of organizations focused on an education intervention, healthy futures intervention, and 

economic opportunities intervention sum to more than 100 percent because two organizations focused on two of 

these areas.
Planned Scaling Activities Report Mathematica

8 

Eleven of the 25 organizations (44 percent) planned to implement an intervention using one 

scaling method only, such as expanding or replicating services. The other 14 organizations (58 

percent) planned to use two or more implementation methods. Across the organizations, 15 

organizations (60 percent) planned adaptations to the proposed intervention. The plans for 13 

organizations (52 percent) described intent to expand an intervention, and 11 organizations (44 

percent) intended to replicate an intervention. Eight organizations (32 percent) intended to 

sustain their intervention and 4 organizations (16 percent) proposed to deepen an intervention. 

3. Analysis approach 

Our review and assessment of organizations’ readiness to scale interventions with evidence of 

effectiveness aligns with each of the five conditions for scale-up presented in the framework. 

First, we analyzed the level of specification of the three intervention readiness conditions. 

Second, we analyzed the level of specification of the two organizational readiness conditions. 

Finally, we analyzed the level of specification of all five conditions together to assess overall 

scaling readiness. 

To conduct the analysis, we developed a two-part systematic review process to collect and record 

information about the intervention and the extent to which organizations’ scaling plans describe 

the intervention and organization conditions that are necessary for successful scaling. The 

sources of information for this review process were the funding applications AmeriCorps State 

and National organizations submitted and the scaling plans SIF organizations submitted: 

• To receive funding from the AmeriCorps State and National program, organizations submit 

an application that describes their project that will use AmeriCorps members to solve a 

significant community problem. AmeriCorps State and National dictates the information that 

organizations must describe in their funding applications, including a description of the 

program design (including a description of the duration and dosage of core activities, the 

target population, inputs, outputs, and outcomes of the intervention), organizational 

capabilities (including personnel structure and training and supervision plans), cost 

effectiveness and budget adequacy, and an evaluation plan. Applications for AmeriCorps 

State and National funding include a description of how the proposed program model builds 

on, or scales, an intervention for which there is evidence of effectiveness. Our review of 

application materials included reviewing the application narrative as well as attachments that 

include the program logic model and performance measures. Over time, AmeriCorps State 

and National program has modified the information it requires applicants to specify in their 

application narratives. In Appendix A, we describe how these reporting requirements have 

changed most recently and present results of a sub-analysis that examines overall scaling 

readiness over time. The results of the over-time analysis did not affect the findings of our 

main analysis.

11

• As part of the funding that SIF organizations receive, organizations must submit a plan for 

growing, or scaling, suborganization impacts. For each suborganization, SIF organizations 

11 The over-time analysis examined variations in specification but did not find any noticeable patterns as they relate 

to specifying all the conditions for scaling readiness. 
Planned Scaling Activities Report Mathematica

9 

are asked to describe their approach to growing the suborganization impact. They are also 

asked to specify the growth goals for suborganizations, as well as the supports necessary for 

helping facilitate suborganization impact growth, their plan for monitoring progress toward 

their growth goals, and the potential risks and challenges they may encounter. 

Collectively, we refer to the AmeriCorps State and National funding application and the SIF plan 

for growing suborganization impact as “scaling plans.” We also reviewed the documents that 

provided evidence of the intervention’s effectiveness.12 

Readers should interpret the analysis results with the understanding that the AmeriCorps-funded 

organizations might describe only the information they are required to report by AmeriCorps. 

Given the purpose of the application narrative and growth plan, Table I.2 shows the extent to 

which the scaling readiness framework aligns with AmeriCorps State and National Notice of 

Funding Opportunity (NOFO) requirements and the SIF growth plan requirements, separately. 

Though there is overlap in what information should be included in the scaling plans and the 

study framework, our assessment of scaling readiness is restricted to what organizations describe 

in the reviewed documents. Interpretation of study findings should keep in mind that a lack of 

description of demonstrating readiness for scaling may be more reflective of reporting 

requirements than of a lack of readiness to scale (see Tables A.1 and A.2 for more details). 

We compared information in the scaling plans to information in the documents that provided 

evidence of intervention effectiveness to ascertain how the planned intervention differed from 

the evaluated intervention and, where there were differences, whether the organization provided 

a rationale for the difference. This comparison of the planned intervention to the evaluated 

intervention is valuable because with some scaling efforts, one might expect that the changes to 

the intervention would be needed to ensure that the services are appropriate for a new population 

or location, for example. 

Focusing on the organization for our analysis is consistent with the framework for successful 

scaling presented in Figure I.2 because the individuals at organizations are the actors who define 

what the intervention is and how they intend to scale it. To assess scaling readiness, we 

categorized whether each component of a condition had been described fully, in a limited way, 

or not at all (see Appendix A for details). We also present examples of full descriptions of the 

conditions for scaling to illustrate the richness of information that is necessary to demonstrate 

readiness for scaling. A full description included several details regarding the component, 

whereas a limited description mentioned the component but lacked details. 

We did not have a priori expectations of what percentage of organizations will provide full or 

partial details for every condition of scaling readiness. The analysis can help funders by 

identifying the conditions of successful scaling that could be emphasized in requests for 

proposals and other funding announcements. 

12 Appendix A describes the process used to identify documents for review to assess scaling, including the 

development or use of a rubric to document the extent to which each organization described its plans to scale the 

intervention, as well as how we conducted the reviews.
Planned Scaling Activities Report Mathematica

10

Table I.2. Alignment of scaling readiness framework and AmeriCorps State and National 

NOFO and SIF Growth Plan requirements

Scaling readiness framework

Do scaling readiness 

framework component 

align to AmeriCorps

State and National

NOFO requirements?

Do scaling readiness 

framework component 

align to SIF Growth Plan

requirements?

Intervention readiness

Well-specified intervention

Content Yes No

Delivery Yes No

Intensity Yes No

Personnel Yes No

Setting Yes Yes

Completion Yes No

Well-specified target population Yes Yes

Implementation supports

Implementation monitoring Yes Yes

Performance benchmarks Yes Yes

Continuous quality improvement (CQI) No No

Training Yes Yes

Communication plan Yes No

Data systems Yes Yes

Organizational readiness

Enabling context

Support for implementation Yes Yes

Prior implementation experience No Yes

Prior implementation challenges No Yes

Implementation infrastructure

Funding Yes Yes

Dedicated personnel Yes Yes

Program materials Yes Yes

Space for implementation Yes Yes

Human resource system No No

Note: For more information on how the AmeriCorps State and National NOFO requirements and the 

SIF Growth Plan requirements align with the scaling readiness framework components, see 

Tables A.1 and A.2. 

C. Road map for the report

This report presents our review of the scaling plans submitted by AmeriCorps-funded 

organizations that were seeking to implement interventions with evidence of effectiveness. 

Chapter II presents findings of the extent to which organizations described the three conditions 

of intervention readiness for scaling in their scaling plans. Chapter III presents our findings about 

organizations’ descriptions of the two conditions related to the readiness of the organization for 

scaling. Chapter IV presents findings about readiness for scaling across all five conditions 

described in the Figure I.2 framework. Appendix A describes in detail our methodology for 

reviewing the scaling plans and includes a supplemental analysis of changes in scaling readiness 

over time. Appendix B contains the rubric used to extract information from the plans. 
Planned Scaling Activities Report Mathematica

11

II. FINDINGS ON INTERVENTION READINESS

In this chapter, we describe the extent to which organizations described the three intervention 

readiness conditions in their scaling plans: (1) intervention core elements, (2) the target 

population, and (3) implementation supports. As described in Chapter I, we categorized the level 

of description for each condition as full, limited, or no description.13 A full description included 

several details regarding the dimension, whereas a limited one mentioned the dimension but 

lacked details. In this chapter, we also discuss the extent to which organizations provided a 

rationale for planned modifications to the intervention and target population. We also discuss the 

extent to which the descriptions of the intervention readiness conditions in the scaling plan 

indicate overall intervention readiness for successful scaling. 

Key findings

• Well-specified intervention. Fifty-six percent of organizations specified the intervention in their 

scaling plans with full or limited details. When describing the intervention, organizations most 

often lacked a description of the qualifications for those who will deliver the intervention (personnel

needs) and a definition of intervention completion.

• Well-defined target population. All organizations described the target population for the 

intervention with full or limited details.

• Implementation supports. Forty-eight percent of organizations described all implementation 

supports with full or limited details. When organizations did not describe all supports, they most 

often lacked a description of continuous quality improvement (CQI) processes.

A. Well-specified intervention

A well-specified intervention describes the essential elements for generating beneficial 

outcomes, referred to as core elements, with enough detail as to enable an element to be taught, 

learned, and done in practice, and it must promote consistent service delivery. Specificity is 

especially important for consistent implementation. For example, describing an element as being 

offered “regularly” or “as needed” is not specific enough to allow a program implementer to 

understand how often services should be delivered. This information is needed for consistent 

service delivery across locations or settings. 

Evidence suggests that clear specification of core elements can boost the likelihood that scaling 

efforts will be successful (Bauman et al. 1991; Dale et al. 2002; Winter and Szulanski 2001). 

When not well specified, the intervention may not be appropriately implemented, does not 

achieve the intended outcomes, or both (Michie and Lester 2005; Michie et al. 2009; Blase and 

Fixsen 2013). Each core element should include a description of the following: 

13 As described in Chapter I, results of all analyses should be interpreted with the understanding that it is possible 

that organizations have unreported details for scaling that might demonstrate a higher degree of intervention and 

organizational readiness for scaling than what is detectible through the review of their scaling plans.
Planned Scaling Activities Report Mathematica

12

• Identification of content, such as the 

services or activities that make up the 

intervention. 

• Mode of service delivery, such as faceto-face or electronic communications.

• Intensity, including how often and how 

long (including how many hours and over 

what time period) participants receive of 

each element and over what time frame.

• Personnel needs, including who will 

deliver the core element(s), qualifications 

needed, and personnel arrangements (for 

example, service provider-to-participant 

caseloads). Personnel includes anyone 

needed for the element’s implementation 

regardless of whether they are volunteers, 

paid by the main provider organization, or 

are employed by a partner or affiliated 

with another entity (such as AmeriCorps 

members). 

• Setting, including location and venue. 

The location describes the geographic 

region and indicates whether the setting is 

rural or urban. The venue describes the 

place where activities occur, such as a 

community center, home, nonprofit 

organization, park, or school. When the 

intervention is implemented in different 

settings, the description should explain 

which elements occur in each setting (for 

example, job training might happen at a 

training site, and job coaching might take 

place at both the training site and an 

employer’s workplace).

• Intervention completion, including the 

minimum participation requirements for 

being considered a participant in the 

intervention and criteria for being 

considered a completer of the 

intervention. 

Example of a well-specified intervention

HIPPY (Colorado Parent and Child 

Foundation)

Identification of content: The intervention will 

deliver an early childhood curriculum that 

addresses seven domains of learning goals: 

literacy, language development, physical and 

motor development, social/emotional development, 

mathematics, science, and creativity. The domains 

will be taught using role-play. The curriculum 

includes weekly activity packets, nine storybooks 

for each year, as well as a set of 20 manipulative 

shapes.

Mode of service delivery: The curriculum will be 

delivered through in-person, one-on-one home 

visits, activity packets for parents to complete at 

home with their children, and parent group 

meetings. 

Intensity: Home visits will last 45 to 60 minutes 

and be offered weekly or biweekly over 30 weeks. 

Parents of children ages 3 and 4 will receive 

weekly home visits, and parents of 5-year-old 

children will receive biweekly visits. Parents will 

work with their children on weekly packets for 15 

minutes per day, five days per week. Parent group 

meetings will be held monthly. 

Personnel needs: The program will use 

approximately 50 AmeriCorps members, recruited 

from the parent population served, to reach more 

than 700 families, establishing a member-toparticipant ratio of roughly 1:15.

Setting: The intervention will be implemented in 

five urban counties (counties included in 

metropolitan areas), two rural counties (counties 

not included in metropolitan areas), and two 

frontier counties (counties with six or fewer people 

per square mile). The program sites will include 

one school district, one county government, and 

four nonprofits. 

Completion: HIPPY USA defines program 

completion as having completed at least 26 weeks 

of the 30-week curriculum.
Planned Scaling Activities Report Mathematica

13

Our analysis examines the extent to which each of these dimensions is specified. We describe the 

extent to which organizations provided any specification (that is, limited or full details) and full 

specification. We also examine the extent to which organizations describe all the dimensions, in 

order to assess overall intervention specification.

Across dimensions, 96 percent of AmeriCorps organizations provided at least some identification

of the content, intensity, mode of service delivery, and setting for the intervention, most of which 

were fully described (Figure II.1). In contrast, 68 percent of organizations defined completion 

and 64 percent of the organizations specified personnel needs. These findings align with what 

organizations—in particular, AmeriCorps State and National organizations (which make up a 

majority of the study sample)—are expected to describe in the scaling plans, because content, 

intensity, mode of service delivery, and setting are clearly required to be described in the 

program logic models and supported in the narrative. For AmeriCorps State and National 

organizations in particular, personnel descriptions are targeted to a description of AmeriCorps 

participants, and thus a broader description of the personnel needed for program implementation 

would not be expected. Guidance for SIF organizations’ growth plans did not require a thorough 

description of these components, but they make up a very small portion of the sample. 

Figure II.1. Percentage of organizations that provided a full, a limited, or no description of 

each core element dimension

Note: The analysis includes 25 organizations that (1) received either an AmeriCorps State and National 

grant during years 2015–2019 or a Social Innovation Fund grant during 2010–2014 and (2) 

planned to implement an intervention that had evidence of effectiveness. See Appendix A for 

details about the analysis. Results for a given core element dimension may not add to 100 

percent due to rounding. 
Planned Scaling Activities Report Mathematica

14

Fifty-six percent of AmeriCorps organizations provided at least a limited specification of all six 

dimensions (results not shown). Sixteen percent of the organizations described five dimensions, 

20 percent described four dimensions, 4 percent described three dimensions, and 4 percent 

described two dimensions in at least a limited way. In contrast, 12 percent of organizations 

provided a full specification of all six dimensions, and another 16 percent did so for five of the 

six dimensions. 

B. Well-defined target population

A well-defined target population helps ensure the target population is appropriate for the 

intervention, and individuals will be properly included or excluded from access to services. This 

specification should include both the main characteristics of the people served and eligibility 

criteria for participation (McElroy and Ladner 2014; Garg 2016), although these two facets are 

often interrelated. For example, an obesity intervention would specify that it will include adults 

ages 18 to 64 and who have a body mass index of at least 30, but that pregnant women are 

ineligible to participate. A well-defined target population increases the likelihood of successful 

scaling, as program implementers will have a greater potential to align the intervention with the 

needs of the target population and use appropriate 

screening for admission to the program—thus 

achieving higher impacts for the population. 

Overall, we found that 64 percent of the organizations 

fully described the target population, and the 

remaining 36 percent provided a limited description 

(results not shown). Organizations categorized as 

providing a limited description gave broad 

descriptions of the target population without 

additional details, such as only stating their target 

population was “at-risk youth.” If the organization 

provided additional details, such as the risk factors 

that put youth at risk, or other population details, such 

as age of the youth, then we categorized the 

organization as providing full details. The analysis 

results show that all organizations described the target 

population to some extent, which could align the 

scaled intervention with the target population. These 

results also align with information organizations are 

expected to provide in their scaling plans. 

Example of a well-defined 

target population

Minnesota Reading Corps

The intervention will enroll students 

who need supplemental instruction to 

become successful readers in 

grades K–3. Students will be 

screened three times per year for 

program eligibility, using valid and 

reliable assessments that predict 

state proficiency performance at 3rd 

grade. The assessment results will 

be used to identify students who are 

below grade level but not scoring low 

enough to otherwise qualify for 

individualized assistance.

C. Modification to the intervention core elements and target population

To identify the most effective or core elements of an intervention prior to scaling, interventions 

are usually developed and tested with a small number of participants (Arthur and Blitz 2000; 

Gallagher 2001; Harachi et al. 1999; Winter and Szulanski 2001; Wolf et al. 2008). Furthermore, 

modifying an intervention that is supported by evidence can be an important way to help a new 
Planned Scaling Activities Report Mathematica

15

population or identify new effective core elements. However, intervention modifications can 

inhibit the intervention’s effectiveness (Chen et al. 2013; Stanton et al. 2005). Justifying the 

differences between the evaluated intervention and the planned intervention suggests that 

organizations have a specific reason for the change and have anticipated the effect these changes 

might have on the potential impacts of the intervention. An example of differences in the planned 

intervention and the evaluated intervention and the justification for such changes is provided in 

Figure II.2. 

In this section, we compare the descriptions of the planned core element dimensions and the 

target population found in the scaling plans to their descriptions found in the evidence 

documents.14 We describe the extent to which organizations provided any description of 

differences between the scaling plans and evidence document (that is, limited or full details) and 

full description of differences. 

Figure II.2. Example of differences in planned and evaluated interventions and rationale 

for change

The results indicate that organizations described differences in the setting for services, the 

population to be served, the content of the intervention, and the way the intervention would be 

delivered in their scaled intervention. They did not describe as many differences in the intensity

of services offered (Figure II.3). In most cases, we could not assess differences in personnel 

needs and definition of intervention completion because more than half of the organizations did 

not describe changes to these elements. Seventy-two percent of organizations described an 

implementation setting in the scaling plan that differed from the implementation setting 

described in the evidence documents. The next most common differences were in the target 

population and the planned and evaluated content of the core elements, described by 56 percent 

and 52 percent of organizations, respectively. Forty-four percent of the organizations planned a 

14 In some instances, we could not assess if a difference existed between the planned intervention and the evaluated 

intervention because the evidence documents or the scaling plans did not describe all core element dimensions. 

We indicate these scenarios in Figure II.3. When a difference could be expected and both the scaling plans and 

evidence documents describe the core element, we present the percentages of organizations in which the 

description does and does not differ.
Planned Scaling Activities Report Mathematica

16

mode of delivery that differed from the description of the delivery mode in the evidence 

documents. 

Figure II.3. Percentage of organizations that planned core element dimensions and target 

population that differed from those in the evaluated interventions 

Note: The analysis includes 25 organizations. See Figure II.1 note for additional information on the 

organizations. The “Same as or expected to be the same as the evaluated intervention” bar 

segments combine (1) instances in which we would not expect differences between the evaluated 

intervention and the planned intervention due to the plan for expanding and/or replicating the 

intervention with (2) instances in which the planned core elements described in the scaling plan 

were the same as evaluated core elements described in the evidence document. Results for a 

given core element dimension may not add to 100 percent due to rounding.

Figure II.4 presents findings about the rationale provided for the differences described in the 

scaling plans. In this figure, the middle and bottom bar segments show, among those 

organizations for which differences were described, the percentage of organizations that 

provided a rationale for differences (bottom) and did not provide a rationale for differences 

(middle). The top bar segments show those organizations that did not provide a rationale because 

a) the description of the core element dimension was the same in the scaling plan and evidence 

document, b) differences in the planned and evaluated core element dimension could not be 

assessed because either the scaling plan or the evaluation did not describe the element at all, or c) 

the planned core element dimensions did not differ from the evaluated core element dimensions 

because of the proposed scaling approach (that is, if the organization proposed to replicate an 

intervention, the target population would not be expected to change).15 The percentage of 

15 The top bar segments in Figure II.4 equal the total of the top and middle bar segments in Figure II.3 for 

identification of content through workforce needs and completion criteria, and they equal the top bar segments in 

Figure II.3 for setting and target population.
Planned Scaling Activities Report Mathematica

17

organizations that justified differences in the core element dimensions or target population 

ranged from 4 percent for definition of completion, target population, and personnel needs to 24 

percent for identification of content and mode of service delivery. 

Figure II.4. Percentage of organizations that provided rationale for differences between 

the planned and evaluated core element dimensions and target population

Note: The analysis includes 25 organizations. See Figure II.1 note for additional information on 

organizations. 

When we examine the percentage of organizations that described differences, looking only at 

those for which a difference existed (that is, when we examine the breakdown between the 

middle and bottom portions of the bars and ignore the top portion), descriptions of differences in 

core element dimensions and target population varied across dimensions. Of those organizations 

in which differences were described, 33 percent to 50 percent of organizations justified the 

identification of content (46 percent), intensity (36 percent), mode of service delivery (50 

percent), setting (33 percent), and personnel needs (33 percent). This suggests that differences in 

intervention content and mode were the most likely to be purposeful deviations from the 

intervention model. Differences in completion criteria were described by only four organizations, 

only one of which provided a rationale for the differences. Given these small numbers, patterns 

of rationale for this dimension are not very telling. Fourteen organizations described differences 

in the target population, but only one of those (7 percent) provided a rationale for those changes. 

Overall, these results could call into question the purposefulness of some of the changes and the 

implications of the changes on fidelity to the models and anticipated intervention impacts. It is 

important to note that before 2019, AmeriCorps State and National did not require organizations 

to provide a rationale for differences between the evaluated and proposed interventions for 
Planned Scaling Activities Report Mathematica

18

scaling, and thus, a lack of description may reflect the reporting requirements more so than the 

purposefulness of the changes proposed (see Appendix A for more discussion of changes in 

reporting requirements over time). 

D. Implementation supports

Although organizations often rely on informal ways of supporting implementation, such as 

informal mentoring for personnel, having formal procedures to support scaling helps to ensure 

sustained fidelity to the model as time goes on (Fixsen et al. 2005). As mentioned previously, we 

broke down the measure of each of the six supports into two subcomponents. For three supports, 

we captured both the presence of the implementation support and the procedures for 

implementing the support. Categorizing the specificity of both subcomponents provides a richer 

description of each implementation support. For example, a full description of performance 

procedures will include a definition of the performance benchmarks as well as a description of 

the procedures used to assess progress on those benchmarks. For the other three supports, we 

broke out the separate activities related to the support. For example, we broke out training for 

personnel to capture preservice training and ongoing training separately. Formal implementation 

supports include the following: 

• An implementation monitoring team: (1) A team of people who monitor implementation 

and (2) procedures the team follows to ensure implementation fidelity. 

• Performance procedures: (1) Benchmarks that service providers must meet to ensure that 

the intervention is delivered as designed and (2) procedures for monitoring those 

benchmarks. 

• Continuous quality improvement (CQI) processes: (1) Identification, description, and 

communication of challenges during implementation and implementation of strategies to 

address challenges in order to continually improve the program and (2) use of data to support 

the CQI process.

• Personnel training requirements: (1) Specification of training that personnel need to 

receive prior to implementation (preservice) to ensure personnel have the up-front skills 

needed to begin delivering services and (2) specification of ongoing training (inservice) to 

ensure that personnel will continue to acquire new or different knowledge, skills, and 

abilities to improve and enhance service delivery over time. Training includes training for 

supervisors to ensure that supervisors can provide effective training to those providing direct 

services.

• Communication system supports among personnel and partners: (1) Procedures for 

regular, formal communication to ensure that communication continues as implementation of 

the intervention becomes more complex, such as during expansion of personnel or 

diversifying roles, and (2) procedures for obtaining feedback that the communication system 

functions as intended. 
Planned Scaling Activities Report Mathematica

19

• Data systems: (1) Tracking, measurement, and storage of information about intervention 

implementation that is used to make decisions relevant to maintain fidelity to the model and 

(2) procedures for ensuring data quality. 

Although each support on its own is 

important to maintaining fidelity, in some 

instances the combination of formal 

implementation supports is also important. 

For example, to identify implementation 

challenges, an implementation monitoring 

team may use data maintained in the data 

system to track performance. Organizations 

may use this information to improve the 

intervention’s processes, maintaining or 

bolstering fidelity on an ongoing basis (U.S. 

Health and Human Services 2016). 

Moreover, having a robust communication 

system supports the ability to address 

implementation challenges and for the 

intervention to be strengthened and improved 

continually (Nord and Tucker 1987). 

We found that all organizations provided at 

least some detail about performance 

procedures and 96 percent of organizations 

provided at least some detail about training 

requirements. The majority of organizations 

provided some description of the 

communication system (92 percent), 

implementation monitoring team (84 

percent), and data systems (80 percent). 

Sixty-eight percent of organizations 

described CQI processes (Figure II.5). When 

we analyzed each subcomponent of 

implementation support, we found similar 

patterns. More organizations described the 

subcomponents of implementation 

monitoring, performance procedures, 

personnel training, data systems, and 

communication systems; fewer organizations 

described the subcomponents of CQI. The 

most frequently specified subcomponent of 

implementation support was the existence of 

performance benchmarks (96 percent of organizations provided a full or limited description), and 

Examples of two implementation 

supports

United Way of Iowa – Iowa Reading Corps

Implementation monitoring team: The 

program utilizes a multilayered supervisory 

structure that includes internal coaches, master 

coaches, and program managers. Each 

position has a clearly defined role that 

contributes to program fidelity, including 

supervision and support to members and 

provision of ongoing coaching in day-to-day 

service and school settings, review of student 

progress and fidelity, and oversight to support 

program implementation. Implementation 

monitoring procedures include nine hours of 

supervision each month using checklists 

explicitly tied to intervention core elements, 

monthly site visits, and review of program data.

Personnel training requirements: 

AmeriCorps members serving as tutors will 

receive initial and ongoing training. 

Preservice training for AmeriCorps members 

will include information on (1) AmeriCorps 

101, (2) program expectations (including 

prohibited activities), (3) the literacy needs, 

(4) the fundamentals of evidence-based

reading instruction, (5) student 

confidentiality, and (6) the tutoring model. 

Tutors will also receive an on-site orientation 

from their internal coach and participate in 

two additional days of literacy content 

training during the year. Tutors are coached 

each month in effective practices from 

master literacy coaches and meet regularly 

with internal coaches to further develop their 

skill set. Finally, tutors participate in 

AmeriCorps-specific trainings and member 

development trainings throughout the school 

year to develop professional skills and create 

team spirit and camaraderie within the 

program.
Planned Scaling Activities Report Mathematica

20

the least specified subcomponent was the presence of a communication system monitoring plan 

(12 percent of organizations provided a full or limited description) (results not shown). 

These results generally align with what organizations were expected by AmeriCorps to describe 

in their scaling plans. For instance, organizations provided a great deal of specification of 

personnel training and performance procedures, both of which are required for specification by 

AmeriCorps. Also, for components that were not required to be specified, like in the case of CQI 

procedures, we saw very little specification of this component by organizations. However, in 

some cases, organizations lacked specification of components that were expected to be specified, 

such as data systems and an implementation monitoring team. As our framework includes 

implementation supports that implementation science research has documented as critical to 

successful implementation, AmeriCorps may want to request more details of how organizations 

will support implementation in practice. Moreover, for those components that are AmeriCorps 

requirements but organizations did not specify, AmeriCorps may want to provide more detailed 

guidance to organizations on how to specify these components. 

Figure II.5. Percentage of organizations that provided a full, a limited, or no description of 

each implementation support

Note: The analysis includes 25 organizations. See Figure II.1 note for additional information on 

organizations.

CQI = continuous quality improvement.

We found that 48 percent of organizations provided a limited or full specification of all six 

implementation supports (results not shown). Another 28 percent did so for five of the six 

implementation supports. When calculating only organizations that provided a full description of 

implementation supports, none fully specified all six implementation supports and only 12 

percent fully specified five of the six implementation supports. 
Planned Scaling Activities Report Mathematica

21

III. KEY FINDINGS ON ORGANIZATIONAL READINESS

In this chapter, we provide detail on the two organizational readiness conditions—an enabling 

context and an implementation infrastructure—and assess how well the 25 AmeriCorps 

organizations included in the analysis specified the conditions in their scaling plans. As 

discussed in Chapter I, we assessed whether the organizations provided a full, a limited, or no 

specification of each component within the organizational readiness conditions.16 We also 

present overall results for each condition based on how many components of the condition each 

organization described.

Key findings

• Enabling context. Twelve percent of organizations specified all three components of enabling 

context in at least a limited way. Organizations were more likely to describe support from 

organization leadership, stakeholders, and partners and least likely to describe encountering past 

implementation challenges. 

• Implementation infrastructure. Thirty-six percent of organizations provided in their scaling plans 

at least some description of financial resources, intervention materials, sufficient personnel, 

physical space, and human resource management systems. Organizations’ scaling plans most 

often lacked a description of intervention materials. Financial resources and personnel for the 

intervention were most frequently described.

A. Enabling context

Organizations that support innovation, learning, and improvement and are able to overcome 

implementation challenges have an implementation context that will help enable them to 

effectively scale an intervention. Successful scaling requires support from organizational and 

partner agency leadership for the scaling effort during either the planning or implementation 

phase. Organizations that have previous experience with program innovation and learning during 

either phase can apply those experiences to improve upon the intervention and how it is 

delivered to facilitate its successful scaling (National Implementation Research Network n.d.). 

An organization that can support learning and improve its systems is better equipped to support 

innovations and to implement interventions as designed (or with improvements) to produce 

desired outcomes. Additionally, organizations that have experienced and successfully addressed 

implementation challenges in the past might be more likely than other organizations to be able to 

address issues that arise when implementing the scaled intervention. Specifically, describing 

experience in addressing challenges signifies that the organization has (1) mechanisms in place 

for identifying challenges, (2) the ability to develop strategies to overcome challenges, and (3) a 

16 Results of all analyses should be interpreted with the understanding that it is possible that organizations have 

unreported details for scaling that might demonstrate a higher degree of intervention and organizational readiness 

for scaling than what is detectible through the review of their scaling plans. 
Planned Scaling Activities Report Mathematica

22

track record implementing similar types of 

strategies successfully. Taken together, these 

experiences indicate that an organization can 

assess whether implementation is occurring as 

planned and change procedures to improve the 

likelihood of successful intervention scaling. 

Moreover, organizations can use past 

experiences overcoming challenges to develop 

procedures and train personnel to avoid similar 

challenges.

Figure III.1 shows the percentage of 

AmeriCorps organizations that described each 

of the three components of enabling context. 

The first component is represented by the 

percentage of organizations that described 

support for the scaled intervention from 

organizational leadership and/or partner agency 

leadership. The second component is 

represented by the percentage of organizations 

that described experience with innovation and 

learning. The third component is presented in 

two ways: the percentage of organizations that 

described encountering past implementation 

challenges, and the percentage of organizations 

that described their approach to addressing 

those challenges. By definition, the issue of 

whether an organization addressed past 

implementation challenges is not relevant for 

organizations that did not describe any past 

implementation challenges. Thus, the black 

horizontal line in Figure III.1 separates these 

organizations from the organizations that 

described past challenges and, therefore, might 

have described how they addressed such 

challenges.

Overall, every organization described support 

from organization leaders and partner agency 

leaders, with 84 percent providing a full 

description. These results align with the 

AmeriCorps reporting requirements in which 

organizations were asked to describe 

organizational and partner support for 

implementation. Sixty-eight percent of 

Examples of an enabling 

context

Organizational and partner agency 

leadership support for scaling

Birth and Beyond Program. The Child 

Abuse Prevention Council (CAPC) has 

implemented evidence-based programs 

for 40 years. CAPC sees AmeriCorps as 

critical to its mission as demonstrated by 

receipt of AmeriCorps State and National 

funds for 14 years. Community 

engagement is a core Birth and Beyond 

program value. Leadership is shared at 

all levels of staff and community. There 

are monthly meetings of partner agency 

staff, members, and parent leaders to 

exchange best practices and 

standardization of services across the 

partner agencies. 

Past experience with innovation and 

learning

Minnesota Reading Corps Pre-K 

Program. The organization runs an 

Innovation Center that uses a continuous 

improvement process to develop and 

field test interventions, analyze data, and 

modify and scale the interventions. The 

organization previously modified the 

intervention to serve more students, 

increase dosage, and broaden the scope 

of the intervention for the youngest grade 

level. Based on the results of a pilot of 

the changes, the organization planned to 

permanently incorporate them into the 

intervention.

Past experience addressing 

implementation challenges

North State Rural Assets Project. One 

organization described challenges 

retaining AmeriCorps members who 

deliver intervention services. The 

organization described several strategies 

it has used to increase retention, 

including more thoroughly recruiting and 

screening members, offering more 

training for members, providing more 

support to sites and members, and 

recognizing members’ contributions and 

accomplishments more consistently and 
Planned Scaling Activities Report Mathematica

23

organizations provided at least some description of experiences with innovation and learning, 

and 20 percent provided a full description. Thirty-six percent provided at least some description 

of encountering past implementation challenges, and 8 percent provided a full description. Of the 

9 organizations (of 25) that described past challenges, all but one described their approach to 

addressing those challenges. It is possible that the other 16 organizations did not encounter prior 

implementation challenges and, therefore, did not have anything to describe in their scaling plan; 

however, it is also possible that they encountered challenges but did not discuss them in their 

scaling plan. Additionally, AmeriCorps State and National organizations were not required to 

describe implementation challenges and approaches for handling challenges in their scaling 

plans. Regardless, these scaling plans lacked supporting evidence that these organizations had 

experience with addressing challenges when implementing interventions. 

Figure III.1. Percentage of organizations that provided a full, a limited, or no description 

of each component of an enabling context

Note: The analysis includes 25 organizations. See Figure II.1 note for additional information on 

organizations. By definition, the question about addressing past implementation challenges is not 

applicable for organizations that did not describe any past implementation challenges. The black 

horizontal line separates these organizations from the organizations that described past 

challenges and which, therefore, might have described how they addressed such challenges.

Looking across the enabling context condition, 12 percent of organizations provided at least a 

limited description of organizational and partner leadership support, their experiences with 

innovation and learning, and implementation challenges (results not shown).17 A majority (76 

17 To count whether the organization described the third component, we used the result for whether the organization 

described its approach to addressing past challenges, not the result for whether the organization described 

encountering past challenges.
Planned Scaling Activities Report Mathematica

24

percent) of organizations described two of these three components in at least a limited way, and 

the remaining 12 percent of the organizations described one component in at least a limited way. 

B. Implementation infrastructure

From our review of implementation science literature, we identified several components of an 

implementation infrastructure that ensure sufficient resources for service delivery. 

• Resources. Organizations that ensure the availability of resources before implementation 

begins can develop and put into place any needed space, equipment, and other supports 

(Metz and Albers 2014). Implementing the intervention may require certain resources, so 

securing them beforehand is important for implementing with fidelity to the model, instead of 

having to alter the model because necessary resources are missing. Three types of resources 

are important components to an organization’s implementation infrastructure: 

1. Financial resources. Organizations need financial resources to pay the costs of 

implementing the intervention. These include costs of service providers and supervisors 

(such as salaries, wages, or benefits); costs for intervention activities (such as advertising 

to recruit participants); and costs of infrastructure (such as rent for the physical space 

where the intervention will be implemented). In addition to AmeriCorps grants, 

organizations may obtain financial resources from other government grants, donations 

from philanthropic organizations and individuals, or fees charged for services. 

2. Materials. The interventions may require producing materials for service providers (such 

as training manuals or implementation guides) or for participants (such as workbooks or 

supplies for activities). Producing or acquiring these resources is necessary for delivering 

the intervention to participants as it was designed. 

3. Physical space. The interventions may require specific space for delivering the 

intervention (such as classrooms or space for one-on-one tutoring). Obtaining the space 

for delivering the intervention to participants is necessary for fidelity to the program 

model. 

• Sufficient personnel. Having a plan for identifying and selecting individuals who have the 

necessary qualifications to provide services can influence whether personnel can deliver the 

intervention as intended (Fixsen et al. 2005; Metz and Albers 2014). For example, without 

adequate personnel, an organization and its partners might not be able to implement an 

intervention with the planned dosage and duration or even implement it in the first place.

• Human resource system. Human resource systems are critical because they typically 

oversee three key implementation functions—hiring, training, and ongoing supervision of 

personnel (National Implementation Research Network n.d.). An organization’s scaling 

efforts can be hindered without adequate human resource management to coordinate and 

perform these functions. For example, without such a system, the burden to train and hire 
Planned Scaling Activities Report Mathematica

25

personnel may fall to other personnel, reducing their availability to deliver services directly. 

Having in place a human resource system can reduce this burden and other risks. 

All organizations described financial resources, 

and 92 percent of organizations described the 

personnel for the intervention at least to some 

extent (Figure III.2). A majority of organizations 

described these components with full details (56 

percent for financial resources and 52 percent 

for a sufficient personnel). Seventy-two percent 

also mentioned a human resource management 

system; 16 percent described it fully. Materials 

were the least-described component, as 40 

percent of organizations mentioned them and 4 

percent provided a full description. These results 

suggest that all or almost all the organizations 

had at least some plans for how the intervention 

would be funded and the personnel who would 

deliver it. However, many scaling plans did not 

demonstrate that the organizations had plans for 

securing the materials or space needed to 

implement the intervention with fidelity. 

These findings mostly align with AmeriCorps 

reporting requirements, but with some 

divergence. AmeriCorps requires both State and 

National and SIF organizations to specify 

funding and personnel for scaling and for the 

most part, organizations provide the required 

information. However, we found that in other 

cases, organizations did not describe what is 

required for specification, such as program 

materials and space for implementation. 

Conversely, although AmeriCorps does not 

require a description of the human resource 

system, many organizations did describe having 

a human resource system. One potential 

explanation for the trends we observed is that 

materials and physical space may be easier to 

overlook when describing an implementation 

plan, compared to funding or having sufficient 

personnel to deliver services, especially if organizations assume they can develop or purchase 

materials easily using available funds or know they already have access to space needed. 

Additionally, the human resource system may be well specified by organizations because 

specification of a human resource system would be related to supporting the organization’s 

Example of an implementation 

infrastructure

Colorado Parent & Child Foundation: 

HIPPY

Resources:

Financial resources. The planned 

AmeriCorps State and National funding

represents only part of the overall intervention 

budget. The remaining funds come from a 

broad base of supporting organizations, 

including other federal programs, a local 

school district, private foundations, and 

charitable organizations.

Materials. The intervention curricula for each 

targeted age group contains weekly activity 

packets, storybooks for each year, and a set 

of manipulative shapes. These materials and 

the rest of the curriculum are designed to 

support school readiness. 

Physical space. The organization specifies 

that the services are delivered at the 

participant’s home.

Sufficient personnel: Intervention services 

are primarily provided by home visitors, who 

are AmeriCorps members. At each site, an 

intervention coordinator oversees the home 

visitors. Senior organization staff overseeing 

the intervention include an executive director, 

deputy director, program officer, federal home 

visiting program manager, and an 

accountant/finance manager.

Human resource system: The 

organization recruits parents who 

previously participated in the intervention to 

be home visitors. The organization also 

ensures—through initial training as well as 

ongoing training and guidance—that each 

intervention coordinator thoroughly 

understands the priorities, rules, and 

regulations of the intervention and 

AmeriCorps.
Planned Scaling Activities Report Mathematica

26

ability to recruit, monitor, and support AmeriCorps participants, which AmeriCorps State and 

National requires for specification. 

Figure III.2. Percentage of organizations that provided a full, a limited, or no description 

of each implementation infrastructure component

Note: The analysis includes 25 organizations. See Figure II.1 note for additional information on 

organizations.

Thirty-six percent of organizations described all five components in either full or limited detail 

(results not shown). This indicates that almost all the organizations that described intervention 

materials (the least-specified component) also described the other four components. Another 24 

percent of organizations described four of the five implementation infrastructure components, 16 

percent described three components, 20 percent described two components, and the remaining 4 

percent described one component. Although no organization fully described all five components 

of its implementation infrastructure, 4 percent of organizations provided a limited description of 

all five components and another 4 percent provided full descriptions for four components. 
Planned Scaling Activities Report Mathematica

27

IV. OVERALL READINESS FOR SCALING 

Key findings

• Overall scaling readiness. Using the scaling readiness framework developed for this study, 

we found that 32 percent of organizations provided at least some description of all three 

intervention readiness conditions; 4 percent of organizations provided at least some 

description of both organizational readiness conditions; and no organization provided at least 

some description of all five conditions of scaling readiness. 

• Scaling readiness by AmeriCorps requirements. When examining scaling readiness 

based on an assessment of those elements of our framework that align with AmeriCorps

reporting requirements, we found that demonstration of scaling readiness increases. With a 

revised definition of scaling readiness, we found that 40 percent of all organizations specify 

all three conditions of intervention readiness, 36 percent of all organizations specify both 

conditions of organizational readiness, and 24 percent of all organizations provide some 

description of all five conditions of scaling readiness.

Chapters II and III of this report presented findings about whether organizations described in 

their scaling plans the three conditions for intervention readiness and the two conditions of 

organizational readiness, according to the criteria laid out in our framework for scaling readiness. 

This chapter examines the readiness of interventions with evidence of effectiveness for scaling 

by presenting a holistic assessment of both intervention and organizational scaling readiness. We 

also examine scaling readiness by conducting an assessment based only on those conditions 

required by AmeriCorps for specification in the scaling plans. We report findings of overall 

intervention and organizational readiness for scaling separately and then examine overall scaling 

readiness. We conclude by discussing recommendations for consideration by AmeriCorps or 

other potential funders in light of the findings from this analysis. 

A. Scaling readiness based on scaling readiness framework 

1. Intervention readiness

As discussed in Chapter II, 56 percent of organizations provided a full or limited description of 

the intervention core element dimensions, 100 percent of organizations had a full or limited 

description of the target population, and 48 percent of organizations provided a full or limited 

description of all implementation supports.18 Looking across these three intervention readiness 

18 We define a well-specified intervention as having a full or limited description of all five dimensions 

(identification of content, intensity, mode of service delivery, workforce needs, and setting) and of criteria for 

completion. We consider a well-defined target population as having a full or limited description of the target 

population, including inclusion or exclusion criteria. We define having implementation supports as having a full 

or limited description of all six implementation supports (an implementation monitoring team, performance 

procedures, continuous quality improvement processes, workforce training requirements, communication system 

supports among workforce and partners, and data systems).
Planned Scaling Activities Report Mathematica

28

conditions, 32 percent of organizations provided a full or limited description for all conditions of 

intervention readiness (Table IV.1). Ten organizations specified only two of the three 

intervention readiness conditions in at least a limited way: six specified core elements and the 

target population and four specified implementation supports and target population. Of the 

organizations that described only one condition in at least a limited way, the condition described 

was the target population. Restricting the analysis to a narrower standard of readiness, including 

only those organizations that provide full descriptions, we found that no organization provided a 

full description of all three intervention readiness conditions (results not shown). 

Table IV.1. Description of intervention readiness conditions

Described

all three 

intervention 

readiness 

conditions

Described two of the three conditions only Described one condition only Did not 

describe any 

intervention 

readiness 

conditions

Intervention core 

elements and 

target population

Intervention core 

elements and 

implementation 

supports

Implementation supports 

and target 

population

Intervention 

core 

elements

Target 

population

Implementation 

supports

Percentage of organizations 32% 24% 0% 16% 0% 28% 0% 0%

Note: The analysis includes 25 organizations and presents percentages of organizations that provided a full or limited description of each condition. See 

Figure II.1 note for additional information on organizations. 

2. Organizational readiness

As described in Chapter III, 12 percent of organizations provided a full or limited description of 

the enabling context, and 36 percent of organizations provided a full or limited description of 

implementation infrastructure.19 Looking across the organizational readiness conditions, we 

found that 4 percent of organizations provided a full or limited description for enabling context 

and implementation infrastructure (Table IV.2). Although implementation infrastructure was 

more likely to be described on its own compared to enabling context, more than half of the 

organizations did not describe either organizational readiness condition. Restricting the analysis 

to a narrower standard of readiness, including only those organizations that provide full 

descriptions, we found that no organization fully described both organizational readiness 

conditions (results not shown). 

Table IV.2. Description of organizational readiness conditions

Described both 

organizational 

readiness conditions

Described one condition only Did not describe either 

organizational 

readiness condition

Enabling 

context

Implementation 

infrastructure

Percentage of organizations 4% 8% 32% 56% 

Note: The analysis includes 25 organizations and presents percentages of organizations that provided a full or limited 

description of each condition. See Figure II.1 note for additional information on organization. 

19 We consider the enabling context to be well described if the organization provides a full or limited description of 

all three contextual components (leader, stakeholder, and partner support; experiences with innovation and 

learning; and addressing implementation challenges). We consider the implementation infrastructure to be well 

described if the organization provides a full or limited description of all five infrastructure components (financial 

resources, materials, physical space, sufficient workforce, and a human resource system).
Planned Scaling Activities Report Mathematica

29

3. Overall scaling readiness 

Looking across all the scaling readiness conditions, no organizations provided a full or limited 

description of all the intervention and organizational readiness conditions, 28 percent provided a 

full or limited description for only the intervention readiness conditions, and 4 percent provided 

a full or limited description for only the organizational readiness conditions (Table IV.3). Sixtyeight percent of organizations did not describe either type of readiness. 

Table IV.3. Description of all scaling readiness conditions

 

Described all 

scaling readiness 

conditions

Described one set of scaling

readiness conditions Did not describe either 

set of scaling 

readiness conditions

Intervention readiness 

conditions 

Organizational 

readiness conditions

Percentage of organizations 0% 32% 4% 64% 

Note: The analysis includes 25 organizations and presents percentages of organizations that provided a full or limited 

description of each condition. See Figure II.1 note for additional information on organization. 

These results indicate that organizations that received AmeriCorps funding to implement 

interventions with evidence of effectiveness did a better job in their scaling plans of describing 

their interventions than they did of describing the organizational readiness to support scaling. 

There are two possible reasons for this pattern. First, scaling plans may provide more 

information related to the intervention because of the requirements of the plans. The AmeriCorps 

State and National NOFO emphasizes the importance of information that specifies the 

components of their interventions rather than implementation supports, though implementation 

supports are also expected to be described. Second, organizations may have considered what 

they will implement rather than how they will support implementation. However, to increase the 

likelihood of successful implementation, organizations must have the infrastructure to support 

implementation. A strong implementation infrastructure and enabling context are needed to 

increase the likelihood of successful scaling. 

B. Scaling readiness based on AmeriCorps reporting requirements

As previously discussed, the primary sources of information for this review process were the 

funding applications AmeriCorps State and National organizations submitted and the scaling 

plans SIF organizations submitted. For each of the documents, the AmeriCorps funding 

programs established their own requirements for the information that organizations should 

report. These reporting requirements did not align perfectly with the scaling readiness framework 

established for this study; therefore, we examine scaling readiness including only the readiness 

components and elements that AmeriCorps also requires. Additionally, we examined readiness 

for the whole sample of organizations as well as a subsample of only AmeriCorps State and 

National organizations. Eighty percent of the sample are AmeriCorps State and National 

organizations, thus reporting requirements for AmeriCorps State and National organizations 

overwhelmingly dictate the content of the data sources examined for this study. 
Planned Scaling Activities Report Mathematica

30

Both AmeriCorps reporting requirements and our scaling framework expect organizations to 

specify six dimensions of a well-specified intervention. Because of this alignment, our findings 

reported in Chapter II related to specification of the intervention remain the same for this subanalysis. We found that 56 percent of all organizations specified all six dimensions of a wellspecified intervention. When we restricted the sample to only AmeriCorps State and National 

organizations, that percentage increased to 65 percent, thus indicating that AmeriCorps State and 

National organizations provide more specification of these elements compared to SIF 

organizations, which aligns with the information AmeriCorps State and National requires these 

organizations to provide in their scaling documents (results not shown). Similar to having a wellspecified intervention, AmeriCorps requirements and our scaling framework both expect 

specification of the target population. Because our main analysis presented in Chapter II found 

that 100 percent of all organizations specify the target population (Figure II.1), additional 

analysis of AmeriCorps State and National organizations was not needed (results not shown). 

When examining organizations’ specification of only those implementation supports required for 

specification by AmeriCorps (that is, when we removed CQI procedures from our scaling 

readiness framework), the percentage of all organizations’ specification of implementation 

supports increased from 48 percent (reported in Chapter II) to 64 percent. When we included 

only AmeriCorps State and National organizations, the percentage increased to 70 percent 

(results not shown). Thus, we found that when the implementation supports dimensions are 

restricted, we see better specification of the implementation supports component, but there still 

remains several organizations that do not provide specification of all implementation support 

dimensions. When looking across the intervention readiness condition and examining only those 

elements required by AmeriCorps, the percentage of organizations that specified all the 

conditions of intervention readiness for scaling increased from 32 percent to 40 percent of all 

organizations and 45 percent of AmeriCorps State and National organizations (Table IV.4). 

Table IV.4. Intervention readiness using AmeriCorps reporting requirements

Described all 

three 

intervention 

readiness 

conditions

Described two of the three conditions only Described one condition only

Did not describe 

any intervention 

readiness 

conditions

Intervention 

core elements 

and target 

population

Intervention core 

elements and 

implementation 

supports

Implementation 

supports and 

target population

Intervention 

core 

elements

Target 

population

Implementation 

supports

Percentage of all organizations 

(n = 25)

40% 16% 0% 24% 0% 20% 0% 0%

Percentage AmeriCorps State 

and National organizations 

(n = 20) 

45% 20% 0% 25% 0% 10% 0% 0%

AmeriCorps did not require organizations to provide the same level of specification of items that 

our model identifies as central to demonstrating organizational readiness for scaling. Using our 

model and as described in Chapter III, we found that 12 percent of organizations provided 

specification of enabling context components. Including only those components that align with 

the AmeriCorps requirements, we found that specification of enabling context increased to 100 

percent of all organizations. That is, all organizations described their support for implementation 

of the proposed intervention. However, when comparing specification of implementation 

infrastructure as defined by our model to the AmeriCorps requirements, we found that under 
Planned Scaling Activities Report Mathematica

31

both definitions, 36 percent of all organizations specified all components of the implementation 

infrastructure, though this increased to 45 percent when including only AmeriCorps State and 

National organizations (Table IV.5). This is expected, as nearly three-quarters of all 

organizations provided a specification of a human resource system even though it was not 

required by AmeriCorps. Thus, removing this element of implementation infrastructure did not 

change the overall specification of the component. Overall, when examining specification of 

organizational readiness using only those components and elements required by AmeriCorps, we 

found that specification of both organizational readiness conditions increased from 4 percent of 

all organizations to 36 percent of all organization and 45 percent of AmeriCorps State and 

National organizations. 

Table IV.5. Organizational readiness using AmeriCorps reporting requirements

Described both 

organizational 

readiness conditions

Described one condition only Did not describe either 

organizational 

readiness condition

Enabling 

context

Implementation 

infrastructure

Percentage of all organizations (n = 25) 36% 64% 0% 0%

Percentage AmeriCorps State and 

National organizations (n = 20)

45% 55% 0% 0%

Finally, when examining overall scaling readiness by those components and elements required 

for specification by AmeriCorps, we found that 24 percent of organizations specified all scaling 

readiness conditions, 16 percent provided a description of only the intervention readiness 

conditions, and another 12 percent provided a description of only the organizational readiness 

conditions (Table IV.6). These percentages increased slightly when including only AmeriCorps 

State and National organizations; 30 percent of State and National organizations described all 

scaling readiness conditions, while 15 percent described only the intervention readiness 

conditions and another 15 percent provided a description of only the organizational readiness 

conditions (Table IV.6). This is a dramatic increase from the overall scaling readiness 

demonstrated when using our framework for scaling readiness. This analysis supports our 

original hypothesis that findings of scaling readiness may be indicative of the information 

AmeriCorps required organizations to provide. 

Table IV.6. Overall scaling readiness using AmeriCorps reporting requirements 

Described all scaling 

readiness conditions

Described one set of scaling

readiness conditions

Did not describe either 

set of scaling 

readiness conditions

Intervention 

readiness 

conditions 

Organizational 

readiness 

conditions

Percentage of all organizations

(n = 25)

24% 16% 12% 48% 

Percentage AmeriCorps State and 

National organizations (n = 20)

30% 15% 15% 40%
Planned Scaling Activities Report Mathematica

32

C. Conclusion

AmeriCorps aims to fund evidence-based models that are ready for scaling. Through this project, 

we have assessed the degree to which AmeriCorps State and National and SIF organizations 

have demonstrated that readiness. The analysis of scaling readiness identified those areas in 

which organizations specified critical scaling readiness components as well as those areas that 

would have benefited from more detail. Our sub-analysis of scaling readiness components that 

align with AmeriCorps reporting requirements supports the hypothesis that organizations will 

provide the information they are required to. With such information, AmeriCorps can make 

important decisions about future funding requests, such as what scaling components 

organizations should be asked to describe more completely in order to assess scaling readiness. 

The framework developed to guide this assessment is applicable to understanding scaling 

readiness for other organizations beyond AmeriCorps. Reviewing applications for funding based 

on the level of evidence provided for the proposed intervention, the readiness of that 

intervention, and the applicants’ organizational readiness for scaling will foster more successful 

scaling efforts and more efficient use of funds. The findings of this project provided important 

information to AmeriCorps on how to approach its application guidelines to obtain the highest 

level of information relevant for successful scaling. Funders and grant makers can use the scaling 

framework to inform their reporting requirements, such that their requirements will align to the 

different conditions that organizations should speak to demonstrate their readiness to scale. 

Additionally, our study finds that even among those conditions related to scaling that 

AmeriCorps organizations were already asked to report, some areas that would have benefited 

from more detail. Thus, our framework can help funders provide more guidance on the details 

they are looking for. In the end, aligning reporting requirements with the conditions specified in 

our framework may foster more successful scaling efforts and more efficient use of funds.
Planned Scaling Activities Report Mathematica

33

REFERENCES 

AmeriCorps. “Metadata Glossary.” Washington, DC: AmeriCorps, 2016. Available at

https://www.nationalservice.gov/sites/default/files/

documents/GLOSSARY-Metadata%20Categories%20Definitions%20for% 

20Publication.pdf. Accessed August 6, 2018. 

AmeriCorps. Annual Management Report, Fiscal Year 2017. Washington, DC: AmeriCorps, 

2017. Available at https://www.nationalservice.gov/ 

sites/default/files/documents/CNCS-FY17-AMR-508_v2r2.pdf. Accessed August 6, 2018. 

Arthur, M.W., and C. Blitz. “Bridging the Gap Between Science and Practice in Drug Abuse 

Prevention Through Needs Assessment and Strategic Community Planning.” Journal of 

Community Psychology, vol. 28, no. 3, May 2000, pp. 241–255. doi:10.1002/(SICI)1520-

6629(200005)28:3<241:AID-JCOP2>3.0.CO;2-X.

Bauman, L., R. Stein, and H. Ireys. “Reinventing Fidelity: The Transfer of Social Technology 

Among Settings.” American Journal of Community Psychology, vol. 19, no. 4, August 1991, 

pp. 619–639. 

Bernfeld, G.A. “The Struggle for Treatment Integrity in a ‘Dis-Integrated’ Service Delivery 

System.” The Behavior Analyst Today, vol. 7, no. 2, April 2006, pp. 188–205. 

http://dx.doi.org/10.1037/h0100086. 

Blase, Karen, and Dean Fixsen. “Core Intervention Components: Identifying and 

Operationalizing What Makes Programs Work.” Washington, DC: Office of the Assistant 

Secretary for Planning and Evaluation, Office of Human Services Policy, U.S. Department 

of Health and Human Services, February 2013. 

Breitenstein, S. M., D. Gross, C. Garvey, C. Hill, L. Fogg, and B. Resnick. “Implementation 

Fidelity in Community-Based Interventions.” Research in Nursing & Health, vol. 33, no. 2, 

April 2010, pp. 164–173. http://doi.org/10.1002/nur.20373. 

Chen, E.K., M.C. Reid, S.J. Parker, and K. Pillemer. “Tailoring Evidence-Based Interventions 

for New Populations: A Method for Program Adaptation Through Community 

Engagement.” Evaluation & the Health Professions, vol. 36, no. 1, March 2013, pp. 73–92. 

http://doi.org/10.1177/0163278712442536. 

Community Tool Box. “Assessing Community Needs and Resources: Understanding and 

Describing the Community.” n.d. Available at https://ctb.ku.edu/en/table-of-contents/ 

assessment/assessing-community-needs-and-resources/describe-the-community/main. 

Accessed March 15, 2018. 

Dale, Nan, Amy Baker, and David Racine. “Lessons Learned: What the WAY Program Can 

Teach Us About Program Replication.” Washington, DC: American Youth Policy Forum, 

2002. 
Planned Scaling Activities Report Mathematica

34

Fixsen, Dean, Sandra Naoom, Karen Blase, Robert Friedman, and Frances Wallace. 

“Implementation Research: A Synthesis of the Literature.” Tampa, FL: University of South 

Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation 

Research Network (FMHI Publication #231), 2005. 

Fixsen, D.L., K.A. Blase, S.F. Naoom, and F. Wallace. “Core Implementation Components.” 

Research on Social Work Practice, vol. 19, no. 5, May 2009, pp. 531–540. 

http://dx.doi.org/10.1177/1049731509335549. 

Gallagher, Kaia. “Bringing Research to Scale: The Nurse-Family Partnership Program.” Denver, 

CO: The Colorado Trust, June 2001. 

Garg, R. “Methodology for Research I.” Indian Journal of Anaesthesia, vol. 60, no. 9, 2016, pp. 

640–645. http://doi.org/10.4103/0019-5049.190619.

Harachi, T.W., R.D. Abbott, R.F. Catalano, K.P. Haggerty, and C.B. Fleming. “Opening the 

Black Box: Using Process Evaluation Measures to Assess Implementation and Theory 

Building.” American Journal of Community Psychology, vol. 27, no. 5, October 1999, pp. 

711–731. doi:10.1023/A:1022194005511. 

Klingner, J., S. Ahwee, P. Pilonieta, and R. Menendez. “Barriers and Facilitators in Scaling Up 

Research-Based Practices.” Exceptional Children, vol. 69, no. 4, July 2003, pp. 411–429. 

Maxwell, N.L. and S. Richman. “Scaling Programs with Research Evidence and Effectiveness 

(SPREE).” Foundation Review, vol. 11, no. 3, November 2019, pp. 55–67.

McElroy L.M., and D.P. Ladner. “Defining the Study Cohort: Inclusion and Exclusion Criteria.” 

In Success in Academic Surgery: Clinical Trials. Success in Academic Surgery, edited by T. 

Pawlik and J. Sosa. London: Springer, 2014. 

Metz, A., and B. Albers. “What Does It Take? How Federal Initiatives Can Support the 

Implementation of Evidence-Based Programs to Improve Outcomes for Adolescents.” 

Journal of Adolescent Health, vol. 54, issue 3, supplement, March 2014, pp. S92–S96. 

Michie, S., and K. Lester. “Words Matter: Increasing the Implementation of Clinical 

Guidelines.” Quality and Safety in Health Care, vol. 14, no. 5, October 2005, pp. 367–370. 

Michie, S., D. Fixsen, J. Grimshaw, and M. Eccles. “Specifying and Reporting Complex 

Behaviour Change Interventions: The Need for a Scientific Method.” Implementation 

Science, vol. 4, no. 1, July 2009, p. 40. 

National Implementation Research Network. “Active Implementation Hub.” n.d. Available at 

https://nirn.fpg.unc.edu/module-2. Accessed March 19, 2008. 

Nord, W.R., and S. Tucker. Implementing Routine and Radical Innovations. Lexington, MA: D. 

C. Heath and Company, 1987. 

Richman, Scott, and Andrei Streke. “Evidence of Effectiveness in AmeriCorps-Funded 

Interventions.” Submitted to AmeriCorps. Chicago, IL: Mathematica, September 2020. 
Planned Scaling Activities Report Mathematica

35

Stanton, B., J. Guo, L. Cottrell, J. Galbraith, X. Li, C. Gibson, et al. “The Complex Business of 

Adapting Effective Interventions to New Populations: An Urban to Rural Transfer.” Journal 

of Adolescent Health, vol. 37, no. 2, August 2005, p. 163. 

U.S. Centers for Disease Control and Prevention. “Chapter One: Planning, Implementing, and 

Evaluating an Intervention—An Overview.” n.d. Available at

http://www.sbhihelp.org/files/CDCPBestPracticesCh1.pdf. Accessed March 15, 2018.

U.S. Health and Human Services, Office of Adolescent Health. “Continuous Quality 

Improvement. Part 1: Basics for Pregnancy Assistance Fund Programs.” Washington, DC: 

U.S. Health and Human Services, September 2016. 

Winter, S., and G. Szulanski. “Replication as Strategy.” Organization Science, vol. 12, no. 6, 

November 2001, pp. 730–743. 

Wolf, Monstrose, Kathryn A. Kirigin, Dean L. Fixsen, Karen A. Blase, and Curtis J. Braukmann. 

“The Teaching-Family Model.” Journal of Organizational Behavior Management, vol. 15, 

no. 1–2, 2008, pp. 11–68. doi:10.1300/J075v15n01_04. 
This page has been left blank for double-sided copying.
APPENDIX A: 

SCALING PLAN REVIEW PROCESS
This page has been left blank for double-sided copying. 
Planned Scaling Activities Report Mathematica

A.3 

This appendix describes the review rubric and process that Mathematica used for extracting 

information on the characteristics of the 25 organizations and their interventions with evidence 

of effectiveness that were included in the analysis, as described in the organizations’ scaling 

plans. Here we describe the scaling plans that we reviewed, our rationale for the unit of analysis, 

and our method for reviewing and analyzing the plans. We also include the rubric used to extract 

information related to scaling readiness. 

A. Documents for review 

AmeriCorps required applicants to submit different types of documents to plan for 

implementation of an intervention based on whether the request for funding was through the 

AmeriCorps State and National program or the SIF program. In 2015 through 2019, those 

applying for AmeriCorps State and National funding submitted applications that explained the 

rationale and approach for their planned interventions, a plan for supervising AmeriCorps 

members, organizational capacity to provide the intervention, and other details of their planned 

implementation approach. We used this application as the scaling plan for these applicants, 

although not all the organizations in the analysis were planning to scale the interventions. The 

reporting requirements established by AmeriCorps aligned with the scaling readiness framework 

developed for this study in many ways but also differed in some important ways (see Table A.1) 

In 2010 through 2015, SIF organizations were required to submit scaling plans that described 

their approach for scaling, their growth goals and indicators, the actions required to achieve their 

growth goals, and a monitoring plan. Table A.2 shows the extent to which the growth plan 

reporting requirements aligned with our scaling readiness framework. Although many 

AmeriCorps State and National and SIF organizations might not characterize their funding 

applications as scaling plans, we use that terminology for ease in exposition. We use these 

documents as the primary information source for our review. 

AmeriCorps also required applicants to submit documents, typically research reports or journal 

articles, which provided evidence that the interventions they planned to implement were 

effective.20 Although these evidence documents were not the primary source of information for 

our review, we did use them specifically to determine how the planned interventions differed 

from the interventions with evidence of effectiveness. 

20 Both AmeriCorps State and National and SIF have tiered-evidence frameworks to rate the quality of evidence 

submitted to them. Evidence ratings are assigned at the organization level only for AmeriCorps State and National 

organizations and at either the SIF organization or suborganization level, depending on which entity is 

implementing a SIF-funded intervention. Organizations can submit multiple evidence documents for their one 

intervention or multiple interventions that they are seeking to implement with AmeriCorps funding. The evidence 

rating is based upon the evidence document(s) deemed as having the most rigorous study design. The evidence 

rating was assigned by a third-party evaluator. See Richman and Streke (2020) for additional details on the 

evidence of effectiveness and documents submitted by organizations. 
Planned Scaling Activities Report Mathematica

A.4 

Table A.1. Alignment of scaling readiness framework and AmeriCorps State and National 2019 NOFO narrative application 

requirements

Program design Organizational capacity Does scaling 

readiness framework 

component align to 

NOFO 

requirements?

Scaling readiness 

framework

Theory of 

change and 

logic model

Performance measures 

(from application 

narrative attachment)

Evidence 

base

Notice 

priority

Member 

experience

Org. 

background 

and staffing

Compliance 

and 

accountability

Culture that 

values 

learning 

Member 

supervision

Intervention readiness

Well-specified intervention

Content X X X Yes

Delivery X X X Yes

Intensity X X X Yes

Personnel X X X X Yes

Setting X X Yes

Completion X Yes

Well-specified target 

population

X X Yes

Implementation supports

Implementation monitoring X X X Yes

Performance benchmarks X X X Yes

CQI No

Training X Yes

Communication plan X Yes

Data systems X Yes

Organizational readiness

Enabling context

Support for 

implementation

X Yes

Prior implementation 

experience

No

Prior implementation 

challenges

No

Implementation infrastructure

Funding X Yes

Dedicated personnel X X Yes

Program materials X Yes

Space for implementation X Yes

Human resource system No
Planned Scaling Activities Report Mathematica

A.5 

Table A.2. Alignment of scaling readiness framework and SIF Growth Plan reporting requirements

Approach to 

growing impact Growth goals

Monitoring 

plan

Potential risks and 

challenges Does scaling 

readiness 

framework 

component 

align to 

Growth plan 

requirements?

Scaling readiness 

framework

Overall 

approach 

to growing 

impact

How is 

readiness 

for and 

extent of 

expansion 

determined

Growth 

goals

How growth 

goals will 

occur and 

be tracked

Key actions 

and activities 

to enable 

growth

Resources 

invested Timeline

How 

progress 

will be 

monitored 

Potential 

risks or 

challenges 

to meeting 

growth 

goals

Mitigation 

of risks or 

challenges

Intervention readiness

Well-specified intervention

Content No

Delivery No

Intensity No

Personnel No

Setting X X Yes

Completion No

Well-specified target 

population

X Yes

Implementation supports

Implementation monitoring X X Yes

Performance benchmarks X Yes

CQI No

Training X X Yes

Communication plan No

Data systems X X Yes

Organizational readiness

Enabling context

Support for 

implementation

X X Yes

Prior implementation 

experience

X X Yes

Prior implementation 

challenges

X X Yes

Implementation infrastructure

Funding X X Yes

Dedicated personnel X X Yes

Program materials X X Yes

Space for implementation X X Yes

Human resource system No
Planned Scaling Activities Report Mathematica

A.6 

B. Study sample and unit of analysis

The analysis presented in this report focuses on the organization—not the scaling plan or the 

intervention—because our research questions pertain to how organizations define and conduct 

scaling. We define the organization to include organizations receiving AmeriCorps funding to 

implement their interventions and suborganizations selected by AmeriCorps organizations during 

their own competitions (in the case for SIF) to implement their interventions. However, it is 

important to note that organizations submit a scaling plan for their own or their suborganizations’ 

scaling activities. Regardless, we focus on organizations because it is the individuals at 

organizations who act to define what the intervention is that they intend to scale and how they, as 

an organization, intend to scale it. This approach is consistent with the framework for successful 

scaling described in Chapter I, which requires that the organization must be ready to scale the 

intervention, which itself must be ready to be scaled. 

Our analysis included AmeriCorps-funded organizations that had grants that received a strong or 

a moderate evidence rating from another independent, third-party evaluator contracted by 

AmeriCorps based on the evidence they submitted for the intervention they planned to 

implement. Richman and Streke (2020) describe the effectiveness standards that the third-party 

evaluator used to assess the evidence of the interventions’ effectiveness, with the goal of 

identifying interventions that had statistically significant beneficial impacts on participants’ 

outcomes that could be reliably attributed to access to the intervention. Richman and Streke 

(2020) reviewed the evidence of effectiveness for 32 interventions submitted by 30 organizations 

that met the study inclusion criteria. An organization could have described in a scaling plan one 

or possibly more planned approaches for implementing an intervention, and those approaches 

might or might not meet the definition of scaling. For example, an organization could have 

planned both to expand and replicate intervention services.

We developed additional screening criteria to determine the final sample of organizations and 

scaling plans to include in our analysis. First, a scaling plan must be informed by evidence that 

supports the intervention’s effectiveness. Therefore, evidence of effectiveness must be 

established before the submission of the scaling plan. This ensures that organizations first have a 

clear understanding of the intervention’s components and the supports and context that were in 

place during the implementation that helped enable the intervention to have a favorable impact 

on its target populations. This would help organizations understand what worked well and what 

did not, thus informing their development of plans for scaling interventions that will maintain 

effectiveness. However, eight SIF organizations submitted scaling plans before evidence was 

established for their interventions’ effectiveness, and were treated as follows in the final sample: 

• Three SIF organizations each submitted one scaling plan for one or more intervention(s). 

However, the accompanying evidence documents did not precede the scaling plan(s), and the 

organizations and the intervention(s) were removed from the sample. 

• One SIF organization submitted one scaling plan for one intervention; however, the 

accompanying evidence documents did not precede the scaling plan. This particular 

intervention was removed from the sample. The same organization submitted a separate 
Planned Scaling Activities Report Mathematica

A.7 

scaling plan for a separate intervention that did meet the inclusion criteria. The organization 

and the intervention that met the inclusion criteria are included in the sample. 

• One SIF organization submitted one scaling plan for two separate interventions. One of the 

interventions did not meet the inclusion criteria because the evidence document did not 

precede the scaling plan. We included the organization in the overall sample and the one 

intervention that met the inclusion criteria. However, the excluded intervention was proposed 

for scaling by other AmeriCorps organizations that were included in the sample. Thus, 

although this particular scaling plan was not included in the analysis, other plans for scaling 

this particular intervention were included. 

• One SIF organization submitted one scaling plan for three separate interventions. Two of the 

interventions did not meet the inclusion criteria because the evidence document did not 

precede the scaling plan. The third intervention proposed also did not meet the inclusion 

criteria; however, the excluded intervention was proposed for scaling by other AmeriCorps 

organizations that were included in the sample. Therefore, we excluded the organization and 

the first two interventions from the sample but included the third intervention in the analysis, 

based only on information from proposed scaling plans submitted by the other AmeriCorps 

organizations. 

• One SIF organization submitted a scaling plan for a SIF-funded intervention, but the 

evidence document did not precede the scaling plan. However, the same organization also 

submitted an application for AmeriCorps State and National funding for the same 

intervention, so although the SIF scaling plan was not included in the analysis, the 

organization and the intervention are included. 

• One SIF organization submitted one scaling plan for four separate interventions. Two of the 

interventions did not meet the inclusion criteria because the evidence document did not 

precede the scaling plan. However, the other two interventions proposed in the scaling plan 

did meet our inclusion criteria, thus we included the organization and the two interventions in 

the analysis sample. 

Table A.3 provides an overview of these different scenarios and presents the total number of 

organizations, scaling plans, and interventions included in this analysis. Of the analysis sample, 

12 organizations provided 12 scaling plans for 12 unique interventions; that is, the interventions 

were not proposed by any other organization. However, other organizations submitted more than 

one scaling plan for one intervention. Seven organizations submitted a total of 30 scaling plans 

for 7 unique interventions; that is, these 7 interventions were not proposed by any other 

organization. Another 3 organizations submitted a total of 7 scaling plans for the same 

intervention; that is, 3 organizations planned to implement the same intervention. Finally, 3 

organizations each submitted one scaling plan that described more than one intervention, 

yielding a total of 3 scaling plans and 6 interventions. In the next section we describe how we 

accounted for all these scenarios in our analysis. 
Planned Scaling Activities Report Mathematica

A.8 

Table A.3. Number of organizations, scaling plans, and interventions in the analysis

Scenario

Number of 

organizations

Number of scaling 

plans

Number of 

interventions

A single organization submitted one scaling plan for 

one unique intervention 

12 12 12

A single organization submitted more than one scaling 

plan for one unique intervention 

7 30 7 

Multiple organizations submitted at least one scaling 

plan for the same intervention 

3 7 1 

Multiple organizations submitted multiple scaling plans 

for multiple interventions 

3 3 6 

Total 25 53 26

C. Review methodology and analysis strategy

We used a two-part systematic review process to collect and record information about the 

intervention and the extent to which organizations’ scaling plans demonstrated that the 

organizations and their interventions had the conditions necessary for successful scaling. 

Members of the study team developed a two-part rubric for systematically reviewing documents 

submitted by AmeriCorps organizations that (1) provided evidence of an intervention’s 

effectiveness and (2) described plans for scaling the effective intervention.21 

• The intervention review section is first used to review the evaluation reports submitted to 

AmeriCorps by organizations as support for their interventions’ effectiveness. In this section, 

reviewers focused on describing the intervention as implemented and evaluated. When 

completing this section, we did not consult other external sources (for example, we did not 

query authors to collect information not available in the documents or web searches to find 

additional information on the interventions). If more than one evidence document that met 

the project’s effectiveness standards was submitted, a separate assessment was created for 

each evidence document, and the information was then combined to provide an interventionlevel summary of characteristics that informed the evidence supporting each intervention. 

The completed assessments of the evidence documents were used to provide context for the 

review of the scaling plan. 

• The scaling readiness section is used to review the scaling plans and application narratives 

submitted by AmeriCorps organizations designed to understand organizations’ planned 

activities for scaling the interventions with evidence of effectiveness. This is designed to 

capture an intervention’s readiness to be scaled and the organization’s ability to support that 

scaling. This section contains six subsections that align with our framework for successful 

scaling. The first subsection collects information about the approach to scaling and the 

context in which the intervention will be implemented. Each of the next five subsections 

collects information on one of the five conditions needed for intervention and organizational 

21 This document presents the details of the rubric used for reviewing evidence documents and scaling plans 

submitted by AmeriCorps organizations. However, this document is more generally applicable for other audiences 

in need of a rubric to systematically review documents that present evidence of intervention effectiveness or 

assess readiness for scaling.
Planned Scaling Activities Report Mathematica

A.9 

readiness for scaling: (1) a well-specified intervention, (2) a well-defined target population, 

(3) implementation supports, (4) the implementing organization’s enabling context, and (5) 

the organization’s implementation infrastructure.

For each condition of scaling readiness, reviewers determined how well the organization 

described the condition and demonstrated readiness for scaling. Most conditions consisted of 

subcomponents that made up the overall condition. For example, the overall condition of 

implementation infrastructure was measured by the specification of five specific infrastructure 

components: (1) financial resources, (2) personnel to implement the intervention, (3) materials, 

(4) physical space, and (5) a human resource system. When a condition consisted of components, 

we measured the specification of each component and used this to describe readiness for the 

overall condition. Reviewers pulled information from the scaling plans to determine if 

organizations provided a full specification, a limited specification, or no specification for each 

condition. 

Over time, minor modifications were made to the review rubrics to allow for enhanced analysis 

of scaling readiness. All scaling plans were ultimately reviewed with the finalized rubric, so the 

same rubric items were applied to all scaling plans. 

We broke out several components for implementation supports and enabling context so 

reviewers could analyze these components at their subcomponent level. In the first round, for 

example, reviewers described the specification of presence of an implementation monitoring 

team and procedures to monitor fidelity in one response; in the second round of reviews, 

implementation monitoring team and procedures to monitor fidelity were two separate review 

items, creating a pair of subcomponent questions. We created paired subcomponent questions for 

all 6 implementation subcomponents, thus creating 12 implementation subcomponents. 

The breakdown of components into subcomponents required us to modify our tabulation of the 

results of these components. To report a single value for each of the paired subcomponents, we 

created composite categorizations of specification of each subcomponent pair. We did so by 

attributing a numerical value to each specification value (2 = full description, 1 = limited 

description, 0 = no description). For each paired subcomponent, we averaged the values of the 

two individual subcomponents and rounded to the nearest whole number to assign a single 

categorization for each subcomponent. Rounding to the nearest whole number tilts the 

specification of each subcomponent toward a more full or limited specification than no 

description. For example, a component in which the first subcomponent was described fully and 

the second subcomponent was described in a limited way will have a combined specification that 

indicates a full description. Additionally, a component in which the first subcomponent was 

described in a limited way and the second subcomponent was not described at all will have a 

combined specification that indicates a limited description. Table A.4 presents an example of 

how we created the composite score, and Tables A.5 and A.6 present the results of 

implementation support and enabling context subgroup analysis, respectively. 
Planned Scaling Activities Report Mathematica

A.10

Table A.4. Example of creating composite specification score from implementation 

support subcomponents

Subcomponents Composite

Implementation 

Organization monitoring

Implementation 

monitoring team

Implementation 

monitoring process

A 2 2 2 

B 2 1 2 

C 2 0 1 

D 1 1 1 

E 1 0 1 

F 0 0 0 

Note: 2 = full description; 1 = limited description; 0 = no description.

Table A.5. Percentage of organizations that provided a full, a limited, or no description of 

each dimension of implementation supports

Implementation monitoring team Performance procedures

Organization

Monitoring 

team

Monitoring 

process Combined

Performance 

benchmarks

Benchmarks 

process Combined

Full description 60 52 72 56 36 56

Limited description 20 28 12 40 56 44

No description 20 20 16 4 8 0 

CQI processes Personnel training requirements

Organization CQI process Data for CQI Combined

Preservice 

training

Inservice 

training Combined

Full description 12 4 4 60 56 60

Limited description 56 28 64 32 36 36

No description 32 68 32 8 8 4 

Communication system Data systems

Organization

Communication plan

Communication 

monitoring Combined Data system Data quality Combined

Full description 36 8 8 48 12 36

Limited description 56 4 84 32 36 44

No description 8 88 8 20 52 20

Note: For each of the six components in this table, the combined result for an organization is the average of the 

results for the component’s two subcomponents (where full description = 2, limited description = 1, and no 

description = 0), rounded to the nearest whole number.

CQI = continuous quality improvement.
Planned Scaling Activities Report Mathematica

A.11

Table A.6. Percentage of organizations that provided a full, a limited, or no description of 

each dimension of an enabling context

Leader, stakeholder, and 

partner support

Innovation and learning 

organization

Organization Leader support

Stakeholder/

partner support Combined Past innovation

Previous 

improvement 

activities Combined

Full description 76 64 84 20 8 20

Limited description 20 28 16 36 44 48

No description 4 8 0 44 48 32

Note: For each of the two components in this table, the combined result for an organization is the average of the 

results for the component’s two subcomponents (where full description = 2, limited description = 1, and no 

description = 0), rounded to the nearest whole number.

For two conditions of intervention readiness for scaling—having a well-specified intervention 

and a well-defined target population—we compared the organization’s description in the scaling 

plan with the description of these conditions in the evidence documents. Doing so helped us 

identify modifications to the intervention for which there is evidence of effectiveness. 

Specifically, we noted whether the two documents specified the feature in the same way. If they 

specified it differently, we noted whether the scaling plan provided a rationale for the change. 

The rationale could be theoretical or practical, and evidence-based or not. We recorded this 

information to document the extent to which the departures from the evaluated intervention were 

purposeful. 

We conducted a tabular analysis of the information about the 25 organizations’ plans for each 

condition for successful scaling, with a focus on whether the organizations provided a full, a 

limited, or no information about each component. For the 7 organizations that submitted more 

than one scaling plan for a unique intervention and the 3 organizations that submitted at least one 

scaling plan for the same intervention, reviewers finalized an assessment for each submitted 

scaling plan. For these cases, we replicated the process of creating a composite score by 

averaging the values included in the finalized assessment for the scaling plans and rounded to the 

nearest whole number to assign a single categorization for each component of the condition. 

Table A.7 provides an example of this process. We used the same process for the organizations 

that submitted a single scaling plan for two interventions—completing a separate rubric for each 

planned intervention and creating a composite categorization of overall scaling readiness for the 

organization based on the average values across the two rubrics. 
Planned Scaling Activities Report Mathematica

A.12

Table A.7. Example of categorizing intervention readiness for organizations that 

submitted multiple scaling plans for their interventions

Organization Intervention

Scaling 

plan

Are the 

services 

well 

specified?

Is the 

intensity 

well 

specified?

Is the 

service 

delivery 

mode well 

specified?

Is the 

personnel

well 

specified?

Is the 

setting well 

specified?

A B C 2 1 2 0 1 

A B D 1 1 2 0 1 

A B E 0 1 2 1 2 

Overall categorization of intervention 

readiness component used for analysis

1 1 2 0 1 

Note: 2 = full description; 1 = limited description; 0 = no description.

Reviewing the scaling plans required systematic procedures to ensure that the study team reliably 

extracted standardized information from each evaluation report. Study team leaders trained 10 

reviewers, including a third-party consultant, to identify the scaling approach and document 

scaling readiness criteria using the rubric. The consultant was external to Mathematica and 

reviewed all evidence documents in which Mathematica was the evaluator of the intervention to 

avoid any potential conflicts of interest. Reviewers classified the scaling approach of each 

intervention and documented the characteristics related to scaling readiness of each intervention 

being reviewed. Study team leaders reviewed all finalized assessments to ensure that they 

contained complete and accurate information. Finalized assessments were also reviewed by a 

Mathematica researcher external to the rest of the review process as a final quality assurance 

procedure. 

D. Scaling readiness over time

The AmeriCorps State and National program office will periodically make modifications to the 

Notice of Funding Opportunity (NOFO) requirements and guidelines. Because of the periodic 

changes that occur, examining how conditions of scaling readiness are specified over time could 

point to patterns of the extent to which organizations respond to changes in reporting 

requirements. To test this relationship, we compared 2019 organizations—those that received the 

most recent round of funding—with organizations that applied for funding in earlier years. 

In 2019, the AmeriCorps State and National program office modified the NOFO requirements 

and guidelines for the interventions eligible for funding and the supporting evidence for these 

interventions. The changes to the NOFO, which included decreased page limits for grant 

applications and changes in the review criteria that applicants were asked to address, could result 

in differences in the demonstration of scaling readiness among organizations funded in 2019 

compared with those funded in prior cohorts. We compared intervention readiness of the 2019 

applicants with that of applicants from the prior funding years. We found that 2019 applicants 

had similar trends of specification across most intervention and organizational readiness 

components. Despite having similar patterns of specification of scaling readiness components, no 

2019 organizations specified all three intervention readiness conditions or both organizational 

readiness conditions. 
Planned Scaling Activities Report Mathematica

A.13

Between 2018 and 2019, AmeriCorps State and National updated the guidance for interventions 

eligible for funding and the required supporting evidence for these interventions. Additionally, in 

response to feedback from applicants, the funding opportunity guidelines removed duplicative 

information and reduced the page length of the application. Through these changes, a higher 

value of evaluation points was placed on applicants’ description of the evidence base for their 

proposed intervention. Applicants were also instructed to ensure that the proposed intervention 

clearly aligned with the evidence base on characteristics of the beneficiary population, 

characteristics of the population delivering the intervention, dosage and design of the 

intervention, the context in which the intervention is delivered, and outcomes of the intervention. 

In some instances, the changes to the NOFO could result in differences in the demonstration of 

scaling readiness among organizations funded in 2019 compared with those funded in prior 

cohorts. In Table A.8, we list the elements of the 2019 NOFO that are different from previous 

funding years and what the potential implications of those changes might be. 

Table A.8. Changes to 2019 NOFO and implications for scaling activities report

Change 2019 NOFO

Potential implications for scaling 

activities report

Page length The narrative application page length is 10 

pages. 

The 2019 application page length is shorter 

than in prior years, potentially resulting in 

briefer descriptions of all components due to 

less space. However, this may not impact 

the presence of information (as our study 

analysis reports on full description or limited 

description).

Point values More points are allocated to theory of 

change/logic model, evidence base, and 

organizational background/staffing. 

Less points are allocated to notice priority, 

member experience, and compliance and 

accountability.

Potentially more details in the description of 

theory of change/logic model, alignment with 

evidence, and organizational 

background/staffing in 2019. This may result 

in fewer details in the description of member 

experience. However, this may not impact 

the presence of information (as our study 

analysis reports on full description or limited 

description).

Theory of change 

(TOC) and logic 

model

The TOC shows how the intervention will 

lead to outcomes and clearly articulates 

design, dosage, target population, and roles 

of AmeriCorps members. 

The logic model states the community 

problem, describes the inputs (intervention 

locations, number of AmeriCorps members 

who will deliver the intervention, core 

activities that define the intervention or 

program model that members will implement 

or deliver, including: duration of the 

intervention, dosage of the intervention, and 

target population), measurable outputs, and 

outcomes.

A thorough specification of core intervention 

has always been required for AmeriCorps 

State and National applications. Therefore, 

there is little expected difference in the 

presence of a description of core intervention 

components (as our study analysis reports 

on full description or limited description).

Performance 

measures

Applicants must include at least one aligned 

performance measure that corresponds to 

the primary intervention. All performance 

measures must be consistent with the TOC. 

Applicants are not expected to select 

performance measures that correspond to 

each and every activity or impact.

2019 applications required less detail of 

performance measures. This may result in 

less detail with the information on core 

element intensity, delivery, completion, and 

performance benchmarks. 
Planned Scaling Activities Report Mathematica

A.14

Change 2019 NOFO

Potential implications for scaling 

activities report

However, this may not impact the presence

of information (as our study analysis reports 

on full description or limited description).

Evidence base Organizations must describe how the 

proposed evaluation matches the 

intervention described in the evaluation 

documents and aligns with the intervention 

described in the logic model. 

If there are differences between the 

proposed intervention and the intervention 

described in the submitted evaluation 

documents, 2019 applicants may be more 

likely to provide a rationale for these 

differences. This may impact the study 

analysis that reports on whether 

organizations propose adaptations to an 

intervention and, if so, whether they provide 

a rationale for doing so. 

Notice priority Describe how the intervention fits with the 

funding priorities. 

There are no expected implications for the 

analysis.

Member 

experience

Organizations must describe the extent to 

which AmeriCorps members will gain skills 

through training and service experience, how 

AmeriCorps members will be recruited from 

the geographic or demographic communities 

in which the programs operate, and how that 

they will foster an inclusive service culture.

2019 applicants are expected to discuss the 

use of training to build AmeriCorps 

members’ skill, but 2019 applicants are no 

longer required to discuss how the training 

will be delivered. This may result in fewer

details on AmeriCorps member training. 

However, this may not impact the presence

of information (as our study analysis reports 

on full description or limited description).

Organization 

background and 

staffing

Organizations must include a description of 

roles and responsibilities of staff. 

There are no expected implications for the 

study analysis.

Compliance and 

accountability

Describe how to comply with AmeriCorps 

rules and monitor compliance on an ongoing 

basis. 

There are no expected implications for the 

study analysis.

Culture that 

values learning

Organizations must specify that their board, 

management, and staff collect and use 

information, including performance data, for 

learning and decision making. 

May see a greater presence of information of 

performance monitoring processes and use 

of data in 2019. 

To assess the potential impact the change in NOFO requirements may have had on 

organizations’ demonstration of scaling readiness, we examined changes in intervention 

readiness over time in a few ways. First, we compare scaling readiness of the pre-2019 

applicants (n = 21) to 2019 applicants (n = 11) to using the scaling readiness framework 

developed for this study.22 Then we examined the same sample but examined scaling readiness 

using only those components and elements required for specification in the AmeriCorps State 

and National application narratives. Finally, we compared only pre-2019 AmeriCorps State and 

National applicants’ (n = 16) and 2019 AmeriCorps State and National applicants’ (n = 11) 

scaling readiness as defined according to the application specification requirements. Because of 

the small sample sizes, percentages should be interpreted as indicators for trends and should not 

be weighed heavily. 

22 Of all the AmeriCorps State and National applicants included in the study, five organizations submitted scaling 

plans before 2019 and in 2019. These organizations are included in both samples; however, for the pre-2019 

cohorts, we used only scaling readiness data from applications submitted before 2019 and for the 2019 cohort, we 

used only scaling readiness data from the application submitted in 2019.
Planned Scaling Activities Report Mathematica

A.15

Over-time scaling readiness, as defined by the scaling readiness framework: full sample

• Intervention readiness. Looking across the intervention readiness condition, we find that 

organizations that submitted applications for funding prior to 2019 more frequently specified 

all three intervention readiness components, than organizations that submitted applications in 

2019 (Table A.9a). However, we also found that the patterns of specification of intervention 

readiness components were similar between the pre-2019 cohort and the 2019 cohort. In both 

instances, personnel needs and completion criteria were the least frequently specified 

element, while identification of content, intensity, mode of service delivery, setting, and 

target population were specified by nearly all or all organizations (results not shown). We 

also found that 2019 organizations more frequently specified CQI processes and their data 

systems than did the pre-2019 cohort (results not shown). The increase in specification of 

data systems aligns with our expectation that given the addition of the description of “culture 

that values learning,” organizations in 2019 may have provided more information about 

performance monitoring and use of data. Furthermore, by specifying the use of data for 

learning and decision making, organizations may have also described CQI procedures, even 

though AmeriCorps State and National did not specifically require specification of CQI. 

Table A.9a. Intervention readiness across cohorts, by scaling readiness framework

Intervention readiness condition

Described all three 

intervention 

readiness 

components

Described two of 

the three 

conditions only

Described one 

condition only

Did not describe 

any intervention 

readiness 

conditions

Percentage of pre-2019 

organizationsa

33% 29% 38% 0%

Percentage of 2019 organizationsb 0% 81% 18% 0%

a The analysis includes 21 organizations and presents percentages of organizations that provided a full or limited 

description of each condition. See Figure II.1 note for additional information on organizations.

b The analysis includes 11 organizations and presents percentages of organizations that provided a full or limited 

description of each condition. See Figure II.1 note for additional information on organizations.

• Organizational readiness. Looking across the organizational readiness condition, we find 

that organizations that submitted applications for funding prior to 2019 more frequently 

specified both organizational readiness components, than organizations that submitted 

applications in 2019 (Table A.9b). However, patterns of specification of each organizational 

readiness component was also similar across cohorts. Organizations funded in 2019 specified 

at similar rates leadership and stakeholder support of scaling and innovation and learning. 

However, 2019 organizations described having past implementation challenges and how to 

address those challenges to a lesser extent (results not shown). 2019 organizations’ 

specifications of implementation infrastructure were also similar to those of pre-2019 

organizations. Most organizations specified the financial resources required for scaling and 

descriptions of sufficient personnel, and fewer organizations specified materials, physical 

space, and having a human resource system to support implementation (results not shown). 
Planned Scaling Activities Report Mathematica

A.16

Table A.9b. Organizational readiness across cohorts, by scaling readiness framework

Organizational readiness condition

Described both 

organizational readiness 

conditions

Described one condition 

only

Did not describe 

either 

organizational 

readiness 

condition

Percentage of pre-2019 

organizationsa

14% 34% 52%

Percentage of 2019 organizationsb 0% 27% 73%

a See footnote for Table A.9a. 

b See footnote for Table A.9a.

• Overall readiness. Overall, we found that 2019 organizations did not demonstrate 

intervention readiness or organizational readiness at the same rate as pre-2019 organizations 

(Table A.9c). No 2019 organizations specified all three intervention readiness conditions or 

both organizational readiness conditions, which means that no 2019 organizations specified 

all scaling readiness conditions. In contrast, among the pre-2019 organizations, two 

organizations did. 

Table A.9c. Overall scaling readiness across cohorts, by scaling readiness framework

Overall scaling readiness

Described all scaling 

readiness conditions

Described one set of 

scaling readiness 

conditions

Did not describe 

either set of 

scaling readiness 

conditions

Percentage of pre-2019 

organizationsa

10% 29% 62%

Percentage of 2019 organizationsb 0% 0% 100%

a See footnote for Table A.9a.

b See footnote for Table A.9a.

Over-time scaling readiness as defined by AmeriCorps: full sample

When we used only the variables that AmeriCorps required for specification, we found that 2019 

organizations had much lower intervention readiness specification than pre-2019 organizations 

(9 percent compared to 43 percent), similar rates of organizational readiness specification (36 

percent compared to 38 percent), and lower overall readiness (9 percent compared to 29 percent) 

(see Tables A.10a,b, and c). The primary driver of this stark decline in intervention readiness 

appears to be related to a decline in the percentage of organizations that specified personnel 

needs in 2019 (down to 45 percent of 2019 organizations from 71 percent of pre-2019 

organizations) and a decline in specification of performance benchmarks (down to 73 percent of 

2019 organizations from 95 of pre-2019 organizations). Though the decline in detail of personnel 

is unexpected because the changes in the NOFO do not appear to relate to personnel, the change 

in specification of performance benchmark is expected, as the 2019 NOFO required less detailed 

specification of performance monitoring benchmarks. 
Planned Scaling Activities Report Mathematica

A.17

Table A.10a. Full sample intervention readiness across cohorts, by AmeriCorps State and 

National reporting requirements

Intervention readiness condition

Described all three 

intervention 

readiness 

components

Described two of 

the three 

conditions only

Described one 

condition only

Did not describe any 

intervention 

readiness conditions

Percentage of all pre-2019 

organizationsa

43% 29% 29% 0%

Percentage of all 2019 

organizationsb

9% 73% 18% 0%

a The analysis includes 21 organizations and presents percentages of organizations that provided a full or limited 

description of each condition. See Figure II.1 note for additional information on organizations.

b The analysis includes 11 organizations and presents percentages of organizations that provided a full or limited 

description of each condition. All the organizations that submitted applications in 2019 were AmeriCorps State and 

National organizations. See Figure II.1 note for additional information on organizations.

Table A.10b. Full sample organizational readiness across cohorts, by AmeriCorps State 

and National reporting requirements

Organizational readiness condition

Described both 

organizational readiness 

conditions

Described one 

condition only

Did not describe 

either organizational 

readiness condition

Percentage of all pre-2019 

organizationsa

38% 62% 0%

Percentage of all 2019 

organizationsb

36% 64% 0%

a See footnote of Table A.10.a b See footnote of Table A.10.a

Table A.10c. Full sample overall scaling readiness across cohorts, by AmeriCorps State 

and National reporting requirements

Overall scaling readiness

Described all scaling 

readiness conditions

Described one set of 

scaling readiness 

conditions

Did not describe 

either set of scaling 

readiness conditions

Percentage of all pre-2019 

organizationsa

29% 24% 48%

Percentage of all 2019 

organizationsb

9% 27% 64%

a See footnote of Table A.10.a

b See footnote of Table A.10.a

Over-time scaling readiness, as defined by AmeriCorps: AmeriCorps State and National 

only sample 

When we looked at over-time readiness of just the AmeriCorps State and National organizations, 

according to only those items that should be specified per AmeriCorps State and National 

reporting requirements, we found again that 2019 organizations had lower specification of all 

scaling readiness conditions compared to pre-2019 organizations (Table A.11a, b, and c). This 
Planned Scaling Activities Report Mathematica

A.18

indicates that overall, by using the scaling readiness framework and solely the AmeriCorps State 

and National reporting requirements, AmeriCorps State and National organizations in 2019 

provided less specification of components that indicate scaling readiness than did applicants 

from previous cohorts. The lack of observed changes in specification could be due to the 

restricted page limit, as organizations had two fewer pages for the main narrative. However, the 

results of the analysis of scaling readiness over time should be interpreted with caution as the 

small sample sizes make it challenging to assess the changes in specification across cohorts.

Table A.11a. AmeriCorps State and National sample intervention readiness across 

cohorts, by AmeriCorps State and National reporting requirements

Intervention readiness condition

Described all three 

intervention 

readiness 

components

Described two 

of the three 

conditions only

Described 

one condition 

only

Did not describe any 

intervention 

readiness conditions

Percentage of AmeriCorps State 

and National pre-2019 

organizationsa

56% 25% 19% 0%

Percentage of AmeriCorps State 

and National 2019 

organizationsb

9% 73% 18% 0%

a The analysis includes 16 AmeriCorps State and National organizations and presents percentages of organizations 

that provided a full or limited description of each condition. 

b The analysis includes 11 AmeriCorps State and National organizations and presents the percentages of 

organizations that provided a full or limited description of each condition. Because all 2019 applicants were submitting 

AmeriCorps State and National applications, these are the same 11 organizations included in the whole sample 

analysis. Thus, the percentages for the 2019 whole-sample analysis are the same as those for the 2019 AmeriCorps

State and National-only analysis.

Table A.11b. AmeriCorps State and National sample organizational readiness across 

cohorts, by AmeriCorps State and National reporting requirements

Organizational readiness condition

Described both 

organizational 

readiness conditions

Described one 

condition only

Did not describe 

either organizational 

readiness condition

Percentage of AmeriCorps State and 

National pre-2019 organizationsa

50% 50% 0%

Percentage of AmeriCorps State and 

National 2019 organizationsb

36% 64% 0%

a See footnote of Table A.11.a b See footnote of Table A.11.a
Planned Scaling Activities Report Mathematica

A.19

Table A.11c. AmeriCorps State and National sample overall scaling readiness across 

cohorts, by AmeriCorps State and National reporting requirements

Overall scaling readiness

Described all scaling 

readiness conditions

Described one set of 

scaling readiness 

conditions

Did not describe 

either set of scaling 

readiness conditions

Percentage of AmeriCorps State and 

National pre-2019 organizationsa

38% 31% 31%

Percentage of AmeriCorps State and 

National 2019 organizationsb

9% 27% 64%

a See footnote of Table A.11.a b See footnote of Table A.11.a
APPENDIX B: 

RUBRIC FOR INTERVENTION AND SCALING ACTIVITIES
This page has been left blank for double-sided copying.
Planned Scaling Activities Report Mathematica

B.3 

I. INTERVENTION REVIEW SECTION

The intervention review section of the rubric is structured to extract information from documents 

submitted by AmeriCorps organizations that report evaluation findings for AmeriCorps-funded 

interventions. The information extracted through the use of this rubric section describes the 

characteristics of the intervention when it was evaluated. It contains six subsections. The first 

subsection collects information about the context in which an intervention was implemented, and 

each of the next five subsections collects information on one of the five conditions needed for 

scaling. 

A. Context for the intervention

Table B.I.1 provides the first subsection of the rubric, which includes three types of questions 

about the context in which the intervention unfolds: 

1. Description of the intervention, according to AmeriCorps nomenclature. The nomenclature 

includes the intervention’s (1) focus area, (2) topic area, (3) intended outcome domain(s), and 

(4) intended outcome(s). The appendix provides a list of predetermined response options for 

classifying the intervention per the nomenclature, with the “other” response indicating that 

the predetermined areas do not adequately describe the intervention. 

2. Local area context, including contextual features that could affect implementation of the 

intervention, including any regional, state, and local contextual features; demand for the 

intervention at the local level; and partners.

3. Measures of implementation, including the percentage of participants who complete the 

intervention and cost per participant. The measures require a definition of components 

(participant, completer, and costs). 

Table B.I.1. Questions about context for the intervention

Question Response options Additional guidance

I. Description of the intervention 

1. What is the 

intervention’s focus 

area?

Only one focus area. Focus areas include disaster services, economic 

opportunity, education, environmental stewardship, 

healthy futures, nonprofit organizational capacity, and 

veterans and military families.

2. What is (are) the 

intervention’s topic 

area(s)?

Up to two topic areas listed 

in Table 1 in the appendix.

The topic area provides greater specificity about the 

intervention’s broad goals than about the focus area. For 

example, an intervention may fall into the “economic 

opportunity” focus area. If it helps low-income people 

obtain employment, however, it would fall into the 

“employability” topic area.
Planned Scaling Activities Report Mathematica

B.4 

Question Response options Additional guidance

3. What is (are) the 

intervention’s 

outcome domain(s)?

Up to two outcome domains 

listed in Table 1 in appendix

that fall within the topic area 

selected.

The outcome domain provides greater specificity than the 

topic area. The intervention aimed at helping low-income 

people obtain employment may do so by providing 

industry skill training, which is the intervention’s outcome 

domain. The veterans and military families, disaster 

services, and nonprofit organizational capacity focus 

areas and some of the education topic areas do not have 

outcome domains. 

4. What is (are) the 

intervention’s 

outcome(s)?

Up to four outcomes listed 

in Table 1 in appendix. 

Outcomes are the targets that the intervention seeks to 

affect. In the example of the industry skill training 

intervention, an outcome might be certifications/skills 

obtained by participants. Outcomes may span several 

outcome domains within a given topic area. 

II. Local area context

1. Might the regional, 

state, and local context 

facilitate or challenge 

implementation of the 

intervention?

Yes, contextual 

considerations might affect 

implementation.

No, contextual 

considerations are not 

mentioned.

Regional, state, and local contextual factors can 

challenge or facilitate implementation of an intervention. 

Such factors may include community stakeholders, the 

political environment, or the local economy. For example, 

an affordable housing intervention that was implemented 

during a local economic downturn could see increased 

demand for its services.

Question 2 is skipped with a “no” response.

2. What are the major 

features of the 

regional, state, and 

local context that 

could have affected 

implementation of the 

intervention?

[open-ended] Describe any features of the regional, state, or local 

context that may have been salient in implementing the 

intervention. 

3. Does evidence exist 

for local demand for 

the intervention? 

Yes (full), local demand 

exists for the intervention.

Yes (limited), local demand 

exists, but details are not 

available.

No, local demand for the 

intervention is not 

mentioned.

Local demand may be identified by conducting a needs 

assessment, landscape analysis, or informal data 

collection through attendance at community meetings or 

conversations with key stakeholders. 

4. Do key partners play 

a role in 

implementing the 

intervention?

Yes, implementation 

partners are identified.

No, partners are not 

identified.

The organization may use partners to deliver services or 

assist with implementation of the intervention. 

Question 5 is skipped with a “no” response. 

5. What role do the key 

partners play in 

implementing the 

intervention?

[open-ended] Describes the partners or contractors that assisted with 

implementation of the intervention, including the role 

played by each.

III. Measures of implementation

1. What is the intended 

overall completion rate 

for participants in the 

intervention?

[open-ended] Intended intervention completion rate during a specific 

time period. Interventions often set forth completion goals 

for participants. 
Planned Scaling Activities Report Mathematica

B.5 

Question Response options Additional guidance

2. What is the actual 

overall completion rate 

for the intervention?

[open-ended] Actual rate of participants’ completion of the intervention 

during a specific time period. 

3. Is cost-per-participant 

information discussed?

Yes, cost per participant is 

discussed.

No, cost per participant is 

not mentioned. 

Cost-per-participant information may be presented as 

part of a cost-benefit analysis.

Question 4 is skipped with a “no” response.

4. What is the cost per 

participant for the 

intervention?

[open-ended] The average cost of delivering the intervention per 

participant, including the year the costs were captured 

and information about the resources reflected in the 

calculation (the cost of personnel time, materials, space 

rental, partner services, among other items).

B. Well-specified intervention

This subsection addresses the first condition of the scaling readiness framework—a wellspecified intervention. It includes questions in six areas (Table B.I.2). The first area describes the 

intervention, and the next four describe a different dimension of each core element. The last area 

describes what it takes a participant to complete the intervention.

1. Core elements, including identification of the intervention by its name. Core elements are 

activities that were part of the intervention and were intended to achieve the intervention’s 

ultimate outcome domain and outcomes. 

2. Mode of service delivery, including how each element was designed to be delivered to 

participants. 

3. Intensity, including how often and for how long each element was offered to participants 

and how much of each element each participant received.

4. Personnel needs, including the intervention’s personnel structure. A personnel structure 

includes the job titles of personnel who delivered the intervention, the number of individuals 

involved in delivery of the intervention, reliance on AmeriCorps national service 

participants, and qualifications of personnel. 

5. Setting, including location (geography) and venue (the place where activities occurred, such 

as a community center, home, nonprofit organization, park, school). 

6. Definition of completion, including criteria for being considered a participant in the 

intervention and what a participant needed to do to be considered a “graduate” or completer 

of the intervention. 

In some cases, the rubric refers to a standardized, name-brand intervention (for example, Teach 

For America). A standardized intervention adheres to a specific model or curriculum in terms of 

the scope and sequence of the intervention’s content. It spells out the same type and amount of 

services that each participant should receive. In a nonstandardized intervention, each 

participant’s need determines the type and amount of services delivered; as a result, each 

participant experiences the intervention differently. 
Planned Scaling Activities Report Mathematica

B.6 

Table B.I.2. Questions about specifying the intervention

Question Response options Additional guidance

I. Core elements

1. What is the name of 

the intervention?

[open-ended] Use of a standardized, name-brand intervention rather 

than the local intervention name. 

2. What are the 

designed core 

elements of the 

intervention?

[open-ended] Core elements are the set of activities or curricula that 

result in participants’ receipt of services, such as a 

description of services, the content area of the 

intervention, the existence of a standardized intervention 

model, the services or curriculum comprising the 

intervention, and the content area of the intervention to 

be delivered. A standardized intervention is considered 

to have a clearly specified intervention. Examples of core 

elements are tutoring, facilitator-led classes or 

workshops, one-on-one coaching, case management, 

electronic or telephone communication with participants, 

and building the capacity of an organization or partner 

organization. 

The description must align with the selected categorical 

responses in Table A.1 in the appendix. 

3. Were all of the 

intervention’s core 

elements 

implemented?

Yes, all core elements were 

implemented.

No, some core elements 

were not implemented.

Question 4 is skipped with a “yes” response.

4. Which core elements 

of the intervention 

were implemented? 

[open-ended] Identify all core elements implemented.

5. Were any 

purposeful changes 

made to the 

intervention core 

elements 

implemented as 

compared to an 

earlier version of the 

model (including 

adaptations and 

modifications)?

Yes, intervention core 

elements were purposefully 

changed. 

No, intervention core 

elements were not 

purposefully changed. 

Organizations may make changes to the intervention 

core elements prior to implementation, referred to as 

purposeful changes. Purposeful changes could extend to 

the intervention content, service delivery mode, intensity, 

and personnel needs (for example, a model designed for 

classroom training of service providers may have 

changed to one-on-one training). Implementing a “2.0” 

version is also a purposeful change. Differences between 

intended and actual implementation are not considered 

purposeful changes if, for example, the actual duration 

was shorter than intended, but the intended duration did 

not differ from the intervention model. 

Question 6 is skipped with a “no” response. 

6. What purposeful 

adaptations or 

modifications did the 

organization make?

[open-ended] Describes all purposeful changes made to the 

intervention core elements. 
Planned Scaling Activities Report Mathematica

B.7 

Question Response options Additional guidance

7. Were any planned

changes made to the 

intervention during 

the implementation 

period?

Yes, planned changes were 

made to the intervention 

during implementation. 

No, planned changes were 

not made to the intervention 

during implementation. 

During implementation, an organization may decide to 

make changes to an intervention, referred to as planned 

changes. Planned changes may include modifications to 

activities or services, mode of delivery, source of 

delivery, setting, or intensity. For example, an 

organization may have decided to offer an intervention in 

a new setting, extend the period of service delivery, or 

change the originally planned activities in order to test 

whether such changes would increase participant 

outcomes. 

Question 8 is skipped with a “no” response.

8. What were the 

planned changes to 

the intervention and 

the reasons for 

them?

[open-ended] Describes all planned changes to intervention core 

elements.

II. Mode of service delivery

1. What is the mode of 

delivery for each 

element?

[open-ended] The service delivery model specifies how services are 

delivered, who delivers services, and the mode through 

which services are delivered. Organizations should 

provide a description of the way in which each core 

element is delivered. For example, peer educators or 

teams of facilitators may deliver services in groups or 

one-on-one meetings and through the use of electronic 

communication or in-person meetings.

III. Intensity 

1. What is the intended

frequency of delivery 

of each element?

[open-ended] For each element implemented, describes how often the 

element is intended to be offered during a specific time 

period. 

2. What is the 

frequency with which 

each element was 

actually 

implemented (on 

average)?

[open-ended] For each element implemented, describes how often the 

element was actually offered during a specific time 

period. 

3. What is the intended

duration of 

attendance for each 

element?

[open-ended] For each element implemented, includes the intended 

length of time participants were supposed to receive an 

element during a specific time period.

4. What is the actual

duration of 

attendance for each 

element (on 

average)?

[open-ended] For each element implemented, describes the actual 

length of time participants received an element during a 

specific time period.

5. What is the intended

number of hours of 

service receipt for 

each session of each 

element?

[open-ended] For each element implemented, describes how many 

hours participants were supposed to receive an element 

during a specific time period.
Planned Scaling Activities Report Mathematica

B.8 

Question Response options Additional guidance

6. What is the actual 

number of hours of 

service receipt for 

each session of each 

element (on 

average)?

[open-ended] For each element implemented, describes the number of 

hours that participants actually received an element 

during a specific time period. 

7. What is the intended 

total number of hours 

of programming for 

each element?

[open-ended] For each element implemented, describes the total 

number of hours of services that participants were 

scheduled to receive. 

8. What is the actual 

total number of hours 

of programming for 

each element (on 

average)?

[open-ended] For each element implemented, describes the total 

number of hours of services that participants actually 

received, on average, during a specific time period. 

IV. Personnel needs

1. How many individuals 

deliver the 

intervention (overall 

and by intervention 

element)? 

[open-ended] Describes the number of individuals, by job title, who 

directly delivered the services across all locations, both 

overall and by element. For example, if an after-school 

reading intervention used 3 national service participants

to lead classroom sessions and 9 classroom teachers to 

provide one-on-one tutoring across three locations, the 

response would state, “12 personnel overall: 3 

participants (classroom sessions) and 9 teachers (oneon-one tutoring).”

2. Who delivers the 

services per 

intervention element?

[open-ended] For each core element, lists job titles of individuals who 

directly delivered services, including several job titles as 

necessary. For example, if an after-school reading 

intervention used national service participants to lead 

classroom sessions and classroom teachers to provide 

one-on-one tutoring, the titles of both the participants and 

teachers are listed, along with the service element 

delivered by each group.

3. Does the 

intervention’s 

personnel structure 

include AmeriCorps

service participants? 

Yes, AmeriCorps 

participants are included in 

the personnel. 

No, these groups are not 

mentioned.

Question 4 is skipped with a “no” response.

4. What role do 

AmeriCorps

participants play?

[open-ended] Describes the role played by AmeriCorps participants. 

For example, “participants provided tutoring services for 

an after-school intervention.”

5. What qualifications 

are the required or 

preferred as a 

prerequisite for the 

personnel involved in 

delivering the 

intervention?

[open-ended] Describes qualifications for each job title involved in 

delivering the intervention. Qualifications could include 

experience or expertise in delivering the needed services 

or education/training. For example, a nutrition education 

intervention requires educators to hold a degree in 

nutrition and demonstrate at least two years of teaching 

experience. 
Planned Scaling Activities Report Mathematica

B.9 

Question Response options Additional guidance

6. Does the organization 

intend or plan for a 

different personnel

structure or personnel

qualifications or 

training from what 

was originally 

intended or planned?

Yes, personnel plans differ 

from the personnel as 

implemented.

No, personnel plans do not 

differ from the personnel as 

implemented.

Describes how the intended personnel plan, including the 

job titles of those who delivered the intervention, the 

number of individuals involved in service delivery, 

whether the personnel included AmeriCorps national 

service participants, personnel qualifications, and training 

plans for personnel, differed from the plan that was 

implemented. 

Question 7 is skipped with a “no” response.

7. How does the 

personnel structure 

differ from what was 

originally intended or 

planned?

[open-ended] Describes the differences between the planned 

personnel structure and the implemented personnel

structure. Includes the job titles of those who delivered 

the intervention, the number of individuals involved in 

service delivery, and whether the personnel included 

AmeriCorps service participants. 

V. Setting 

1. What are the 

geographic location(s) 

of the intervention, 

and what sites are 

included in 

implementation?

[open-ended] For all locations, describes the geographic region, 

location, and whether the intervention was implemented 

in a rural or urban setting (or both). For example, an 

intervention implemented in two regions of the United 

States, three cities or areas, and several school districts 

and schools within each district would state, "Northeast 

and Southern regions, including New York, NY 

(Brooklyn): three school districts, 10 schools; 

Washington, DC: one school district, 8 schools; eight 

rural counties in southwestern Virginia (two school 

districts, 4 schools).” 

2. In what venue(s) are 

the intervention core 

elements 

implemented?

[open-ended] The implementation setting is where an intervention 

physically takes place, such as community center, home, 

nonprofit organization, park, school, and residential 

facilities. Organizations may implement core elements in 

different settings. For example, an intervention may have 

included job training at an American Job Center and job 

coaching on site with employers; both the American Job 

Center and employer settings, along with the job training 

and job coaching activities, are listed.

3. Did the evaluation 

enroll participants 

from a subset of 

intervention 

locations? 

Yes, the locations for the 

evaluation were a subset of 

places where the 

intervention was 

implemented. 

No, the locations for the 

evaluation and 

implementation were the 

same.

The evaluation of the intervention may be conducted with 

participants from a subset of the locations in which the 

intervention was implemented. For example, an 

intervention may have been implemented in two cities, 

but the evaluation included participants in only one city.

Question 4 is skipped with a “no” response.

4. What locations were 

included in the 

evaluation?

[open-ended] Describes geographic locations indicating where the 

evaluation occurred and the number of locations. 
Planned Scaling Activities Report Mathematica

B.10

Question Response options Additional guidance

VI. Definition of completion

1. Is intervention 

participant defined?

Yes, participant is defined.

No, participant is not 

defined.

Definition of a participant describes the minimum amount 

of services needed to be received in order to be 

considered a “participant” in the intervention. For 

example, individuals may enroll in an intervention and be 

considered a participant even if they never receive any 

services. Alternatively, participants may be defined as 

only those individuals who enroll and receive at least one 

service through the intervention.

Question 2 is skipped with a “no” response.

2. What is the definition 

of an intervention 

participant?

[open-ended] Describes the amount of services an individual must 

receive in order to be considered a “participant.”

3. Is completion of the 

intervention defined?

Yes, completion is defined.

No, completion is not 

defined.

Definition of completion indicates what participants must 

do to be considered “completers” or “graduates.” For 

example, participants may have to complete a course 

and obtain a certificate in order to complete an 

intervention.

Question 4 is skipped with a “no” response. 

4. What is the definition 

of intervention 

completion?

[open-ended] Describes what participants must do to be considered 

“completers” or “graduates.”
Planned Scaling Activities Report Mathematica

B.11

C. Well-defined target population

This subsection addresses the second condition of the scaling readiness framework—a welldefined target population. It includes questions about the target population along three major 

dimensions (Table B.I.3): 

1. Intervention as it was designed, including the characteristics of the target population as the 

intervention was originally designed. 

2. Intervention as it was implemented, including the characteristics of the population that was 

eligible to participate in the intervention as it was implemented.

3. Evaluation sample for the intervention, including the participants in the evaluation sample. 

Table B.I.3. Questions about defining the target population

Question Response options Additional guidance

I. Intervention as it was designed

1. What is the target 

population for the 

intervention as it was 

designed?

[open-ended] Describes the characteristics of the population that the 

intervention was originally designed to serve. Includes 

risk factors, age or grade ranges, participants’ locations, 

gender, and so forth. In the case of several target 

populations, describes each population. 

II. Intervention as it was implemented

1. Who is eligible to 

participate in the 

intervention as it was 

implemented by the 

organization?

[open-ended] Describes who is eligible to participate in the intervention 

as implemented. Includes risk factors, age or grade 

ranges, participants’ locations, gender, and so forth, 

along with inclusion and exclusion criteria and 

information for several target populations. 

2. Does the intervention 

as it is implemented 

enroll participants 

primarily from a 

specific racial or 

ethnic group?

Yes, participants are 

primarily from one of the 

prespecified racial or ethnic 

categories.

No, participants are not 

primarily from a racial or 

ethnic category.

Racial/ethnic groups include African American, Hispanic, 

and other. An even distribution of African American, 

European American, and Hispanic participants is 

considered a “no” response. 

3. Does the intervention 

as it is implemented 

enroll participants 

primarily from a 

certain age group?

Yes, participants are 

primarily from one of the 

prespecified age group(s). 

No, participants are not 

primarily from a certain age 

group(s).

Age categories include 0-5, 6-9, 10-7, 18-24, 25 55, and 

56+. The intervention may be designed to serve 

participants in a specific age category, but, during 

implementation, participants from a different age 

category might enroll. Age group categories include all 

ages in that range. For example, if an intervention 

enrolled 4- to 6-year-old children, the response would be 

"0-5 and 6-9." 

4. Does the intervention 

as it is implemented 

enroll participants 

primarily from any of 

the key populations?

Yes, participants are 

primarily from one of the 

prespecified key 

populations.

No, participants are not 

primarily from a key 

population.

Key populations include low-income 

households/individuals, developmentally disabled 

individuals, homeless individuals, parents/caregivers of 

young children, parolees, and veterans/military families. 
Planned Scaling Activities Report Mathematica

B.12

Question Response options Additional guidance

III. Evaluation sample for the intervention

1. Does the evaluation 

sample differ from 

the target population 

of the intervention as 

it is implemented?

Yes, the evaluation sample 

and target population differ 

for the intervention as 

implemented. 

No, the evaluation sample 

and target population do not 

differ for the intervention as 

implemented.

Differences may arise if the evaluation sample was a 

subset of the population served by the intervention. For 

example, an intervention may have targeted 1,000 

students in grades 9 through 12 in Chicago and Detroit, 

but the evaluation may have considered the outcomes of 

only 250 of those students in all grades and in both cities.

Although the evaluation sample might be representative 

of the population it served, it does not include all of those 

served. 

Questions 2 through 5 are skipped with a “no” response.

2. How does the 

evaluation sample 

differ from the target 

population for the 

intervention as it is 

implemented?

[open-ended] Describes how the evaluation sample differs from the 

population intended to receive the intervention. 

3. Does the evaluation 

measure outcomes 

of participants 

primarily from a 

particular racial or 

ethnic group?

Yes, the evaluation 

measured outcomes of 

participants primarily from a 

prespecified racial or ethnic 

group(s). 

No, the evaluation did not 

measure outcomes of 

participants primarily from a 

racial or ethnic group(s). 

Racial/ethnic groups include African American, Hispanic, 

and other. An even distribution of African American, 

European American, and Hispanic participants is 

considered a “no” response.

4. Does the evaluation 

measure outcomes 

of participants 

primarily from a 

certain age group?

Yes, the evaluation 

measured outcomes of 

participants from a certain 

age group(s). 

No, the evaluation did not 

measure outcomes along 

the age dimension.

Age categories include 0-5, 6-9, 10-17, 18-24, 25-55, and 

56+. 

5. Does the evaluation 

measure outcomes 

of participants from 

any of the key 

populations? 

Yes, the evaluation 

measured outcomes of 

participants from a 

prespecified key 

populations.

No, the evaluation did not 

measure outcomes of 

participants from the key 

populations.

Key populations include low-income 

households/individuals, developmentally disabled 

individuals, homeless individuals, parents/caregivers of 

young children, parolees, and veterans/military families. 
Planned Scaling Activities Report Mathematica

B.13

D. Implementation supports

This section of the rubric addresses the third condition of the scaling readiness framework— 

implementation supports (Figure I.2 in the main report). It includes questions in six major areas, 

each of which describes the supports that help ensure quality in implementation (Table B.I.4). 

1. Implementation monitoring team, including how the organization ensures fidelity to the 

intervention model (whether an organization delivered an intervention as intended). It also 

includes whether implementation issues arose, and whether the organization made any 

purposeful changes to the implementation supports. 

2. Performance procedures, including performance goals and benchmarks for how personnel 

deliver the intervention. 

3. CQI, including the ongoing, systematic process of identifying, describing, and analyzing 

strengths and challenges during implementation, along with the collection and use of data to 

improve the intervention’s processes. 

4. Preservice and inservice training for the personnel, including initial and ongoing training 

for those who deliver the intervention as well as the identification of those responsible for the 

delivery of training. 

5. Communication systems, including the elements that facilitate high quality communication 

among intervention leaders, personnel, and partners. 

6. Data systems, including efforts to track, measure, and store information about 

implementation and to use the information to help make decisions for monitoring and CQI. 

Table B.I.4. Questions about implementation supports

Question Response options Additional guidance

I. Implementation monitoring team

1. Is an implementation 

monitoring team 

described?

Yes (full), an implementation 

monitoring team is described, 

with details provided. 

Yes (limited), an implementation 

monitoring team is described, but 

with no details provided.

No, an implementation monitoring 

team is not mentioned.

A monitoring team ensures that implementation 

takes place as planned. Team members may play 

other roles in the organization, such as supervisors, 

project managers, and project directors, or they 

may be other personnel dedicated exclusively to 

ensuring implementation of the intervention takes 

place with fidelity to the model. 

Question 2 is skipped with a “no” response.

2. Which personnel

make up the 

implementation 

monitoring team? 

[open-ended] Describes the team members, including their titles 

and responsibilities, who are monitoring 

implementation of the intervention with fidelity to the 

model. 
Planned Scaling Activities Report Mathematica

B.14

Question Response options Additional guidance

3. Does a process exist 

for monitoring service 

delivery to assess 

fidelity to the 

intervention as 

planned?

Yes, steps exist for monitoring 

fidelity.

No, steps to monitor fidelity are 

not mentioned.

A process for monitoring fidelity to the intervention 

model ensures that services are being delivered as 

intended. The process for monitoring 

implementation fidelity could include collection of 

service delivery data through observations during 

site visits or regular reviews of service data entered 

into a data system. References to processes for 

assessing whether the “intervention is delivered as 

intended” are considered in assessing fidelity. 

Discussions of monitoring quality or other facets of 

implementation are not considered.

Question 4 is skipped with a “no” response.

4. How is 

implementation 

fidelity monitored?

[open-ended] Describes the steps taken to monitor 

implementation with fidelity, including identification 

of the individual (by job title) responsible for 

assessing fidelity, frequency of implementation 

monitoring, processes for managing/supervising 

personnel involved in service delivery, and 

frequency of personnel management or supervisory 

meetings.

5. Are issues or 

challenges noted 

regarding 

implementation 

fidelity? 

Yes, issues were noted. 

No, issues regarding fidelity were 

not mentioned.

Fidelity issues include variation among instructors 

in the manner in which they deliver the curriculum 

across classes or cohorts and unplanned 

modifications to the intervention during the 

evaluation, such as changing an implementation 

site or adjusting the curriculum to meet the target 

population’s unexpected needs. 

Question 6 is skipped with a “no” response.

6. What issues or 

challenges are noted 

regarding 

implementation 

fidelity?

[open-ended] Describes any issues or challenges reported with 

implementation of the intervention with fidelity.

7. Who was responsible 

for supervising 

personnel involved in 

service delivery? 

[open-ended] Describes, by job title, who supervised the 

personnel involved in service delivery. If more than 

one personnel member responsible for supervising 

the individuals involved in service delivery, identifies 

each title and who was supervised. 

8. How were personnel

involved in service 

delivery supervised?

[open-ended] Describes the frequency and format of supervision 

for each job title. For example, supervision may 

take place on a weekly, monthly, or as needed 

basis and may be in the form of one-on-one 

meetings or small team meetings. Supervision 

extends to check-in meetings, case note reviews, 

administrative data analysis, communication, and 

supports for supervisors. 
Planned Scaling Activities Report Mathematica

B.15

Question Response options Additional guidance

9. Did any 

implementation 

supports 

purposefully change 

from an earlier 

version of the 

intervention model 

before the start of 

implementation?

Yes, the implementation supports 

changed from an earlier version 

of the model before the start of 

implementation.

No, the implementation supports 

did not change from an earlier 

version of the model before the 

start of implementation.

Organizations may make changes to 

implementation supports before the start of 

implementation. Purposeful changes to 

implementation supports may include changes 

related to the following: the personnel structure, 

personnel training, supervision and performance 

management, implementation monitoring, 

communication processes, and data systems. 

Question 10 is skipped with a “no” response.

10. What purposeful

changes were made 

to the intervention’s 

implementation 

supports?

[open-ended] Describes implementation supports that 

purposefully differ from an earlier design of the 

intervention. Changes may include any support 

related to the following: steps to ensure fidelity, the 

personnel structure, personnel training, supervision 

and performance management, monitoring, 

communication, and data systems. 

II. Performance procedures

1. Are performance 

benchmarks

established for 

personnel involved in 

service delivery 

described?

Yes (full), performance 

benchmarks established for 

personnel involved in service 

delivery are described, with 

details provided.

Yes (limited), performance 

benchmarks established for 

personnel involved in service 

delivery are described, but with 

no details provided.

No, performance benchmarks 

established for personnel

involved in service delivery were 

not mentioned.

Performance benchmarks are specific goals that

personnel involved in service delivery must meet as 

related to delivery of the intervention, such as the 

number of participants contacted per personnel

member or the number of participants served per 

personnel member. 

Question 2 is skipped with a “no” response. 

2. What were the 

performance

benchmarks 

established for 

personnel involved in 

service delivery? 

[open-ended] Describes the performance benchmarks 

established for personnel involved in service 

delivery, such as the number of participants 

contacted per personnel member or the number of 

participants served per personnel member. If 

different performance goals are set for specific job 

titles, goals are listed by job title; for example, “case 

managers are required to contact 12 participants 

per day.” 

3. Are procedures for 

monitoring 

achievement of 

performance 

benchmarks 

described?

Yes, procedures for monitoring 

achievement of performance 

benchmarks are described, with 

details provided.

Yes (limited), procedures for 

monitoring achievement of 

performance benchmarks are 

described, but with no details 

provided.

No, procedures for monitoring 

achievement of performance 

benchmarks were not mentioned.

Procedures for monitoring achievement of 

performance benchmarks include how benchmarks 

are measured and collected, who reviews progress 

toward benchmarks, and the frequency of 

performance monitoring.

Question 4 is skipped with a “no” response.
Planned Scaling Activities Report Mathematica

B.16

Question Response options Additional guidance

4. What were the 

performance 

benchmarks 

monitoring 

procedures?

[open-ended] Describes the procedures for monitoring 

achievement of performance benchmarks including 

how benchmarks are measured and collected, who 

reviews progress toward goals, and how frequently 

benchmarks for goals are captured and reviewed. If 

different performance monitoring procedures are 

set for specific job titles or roles, monitoring 

procedures are listed by job title.

III. Continuous quality improvement (CQI)

1. Are CQI processes 

for the intervention 

described?

Yes (full), CQI processes are 

described, with details provided.

Yes (limited), CQI processes are 

described, but with no details 

provided.

No, CQI processes are not 

mentioned.

Continuous quality improvement processes refer to 

procedures for continuously assessing the quality of 

the intervention as implemented to improve 

implementation practice. CQI includes regularly 

testing the intervention and making adjustments as 

needed, with continual retesting of the modifications 

to ensure that the intervention is succeeding as 

planned. 

Question 2 is skipped with a “no” response.

2. What were the CQI 

processes for the 

intervention?

[open-ended] Describes the process for testing the intervention, 

including how frequently CQI data is collected. 

3. Were data analyzed 

to support CQI for 

the intervention?

Yes (full), data were analyzed 

and used for CQI, with details 

provided.

Yes (limited), data were analyzed 

and used for CQI, but with no 

details provided.

No, the analysis and use of data 

for CQI was not mentioned.

CQI requires the collection of data on performance 

benchmarks and use of the data to provide ongoing 

personnel development training and the delivery of 

technical assistance to partners. Data analysis to 

support CQI includes examining personnel

achievement of performance benchmarks. 

Question 4 is skipped with a “no” response.

4. How were data 

analyzed to support

CQI for the 

intervention?

[open-ended] Describes how performance data were used to 

continuously improve intervention implementation.

IV. Preservice and inservice training for personnel

1. Is the initial training 

to deliver the 

intervention received 

by personnel

described?

Yes (full), initial personnel training 

to deliver the intervention is 

described, with details provided.

Yes (limited), initial personnel

training to deliver the intervention 

is described, but with no details 

provided.

No, initial personnel training to 

deliver the intervention is not 

mentioned.

Training for service delivery given to personnel

before implementation begins includes training on 

intervention content (such as a training on a 

curriculum used in the intervention), methods for 

service delivery (such as motivational interviewing), 

and client processing procedures (such as intake 

procedures).

Question 2 and 3 are skipped with a “no” response.
Planned Scaling Activities Report Mathematica

B.17

Question Response options Additional guidance

2. What initial training 

did personnel receive 

to deliver the 

intervention?

[open-ended] Describes, for each job title, initial personnel

training for delivery of the intervention. Specifies 

when training occurred and training content and 

format (such as classroom, online). For example, a 

"summer training institute" for teachers occurs in 

person (through classroom sessions and practice 

teaching) in the summer before teachers begin 

teaching. 

3. Who delivered the 

initial training?

[open-ended] Describes, by job title, who delivered the initial 

training (for example, supervisors, personnel

involved in service delivery, and partners). 

Separately identifies people with different job titles 

who delivered different types of training. For 

example, Teach For America personnel might 

deliver a "summer training institute" for teachers. 

4. Is the ongoing 

training to deliver the 

intervention received 

by personnel

described?

Yes (full), ongoing personnel

training to deliver the intervention 

is described, with details 

provided.

Yes (limited), ongoing personnel

training to deliver the intervention 

is described, but with no details 

provided.

No, ongoing personnel training to 

deliver the intervention is not 

mentioned.

Training for personnel involved in service delivery 

during implementation and throughout the service 

delivery period includes refresher trainings on 

intervention content, methods for service, and client 

processing procedures (such as intake procedures).

Question 5 and 6 are skipped with a “no” response.

5. What is the ongoing 

training needed to 

deliver the 

intervention?

[open-ended] Describes, by job title, the ongoing training that 

helps service providers deliver the intervention. 

Details include when and how often training 

occurred, content (for example, curriculum content, 

use of data systems), and format (for example, 

classroom, online). 

6. Who delivered the 

ongoing training?

[open-ended] Describes, by job title, who delivered the ongoing 

training (for example, supervisors, service 

providers, and partners). 

V. Communication systems

1. Is a communication 

system to support 

coordination among 

personnel and 

partners the 

described?

Yes (full), a communication 

system is described, with details 

provided.

Yes (limited), a communication 

system is described, but with no 

details provided.

No, a communication system is 

not mentioned.

Communication systems support coordination 

among personnel and partners and specify the 

frequency and the expected duration of 

communication, the parties responsible for 

communication, and the expected duration of 

communication. Systems may vary in the extent to 

which they specify and standardize communication. 

For example, a detailed plan may state that, for the 

first six months of the intervention, service providers 

must meet as a group with their supervisor for one 

hour each week to discuss topics related to 

recruitment and engagement of participants; for the 

next six months, meetings must take place monthly 

in the form of hour-long sessions to discuss service 

provision. 

Question 2 is skipped with a “no” response.
Planned Scaling Activities Report Mathematica

B.18

Question Response options Additional guidance

2. What was the 

communication 

system for supporting 

coordination among 

personnel and 

partners?

[open-ended] Describes the communication system, including the 

frequency and duration of the communication 

between personnel and with any partners, and 

which individuals are included in the communication 

system.

3. Is a process for 

ensuring the 

communication 

system functioned as 

intended described?

Yes (full), a process for ensuring 

the communication system 

functioned is described, with 

details provided.

Yes (limited), a process for 

ensuring the communication 

system functioned is described, 

but with no details provided.

No, a process for ensuring the 

communication system 

functioned is not mentioned

Processes to ensure a communication system 

functions as intended include obtaining feedback 

from personnel regarding the communication 

system and tracking the communication that takes 

place. Details include how feedback was obtained 

regarding the functioning of the communication 

system. For example, a communication system may 

have included logs of how often personnel met 

together\ or used regular check-ins with personnel

eliciting feedback on how easily they feel they are 

able to get in contact with each other or key 

intervention partners.

Question 4 is skipped with a “no” response.

4. What was the 

process for ensuring 

the communication 

system functioned as 

intended?

[open-ended] Describes the process for ensuring the 

communication system functioned as intended.

VI. Data systems

1. Is a data system to 

support data 

collection, analysis, 

and decision making 

described?

Yes (full), a data system is in 

place, with details provided.

Yes (limited), a data system is 

mentioned, but with no details 

provided.

No, a data system is not 

mentioned.

Data systems (for example, a management 

information system or a standardized Excel 

spreadsheet) capture enrollment and participation 

data and generate reports on participation trends. 

Such systems support data collection, analysis, and 

decision making. A detailed description specifies 

the information captured by the system, the 

organization’s capacity to analyze recorded data, 

and how the organization uses the data to support 

its decision-making processes.

Question 2 is skipped with a “no” response.

2. What was the data 

system developed to 

support data 

collection, analysis, 

and decision 

making?

[open-ended] Describes the data system developed to support 

data collection, analysis, and decision making.

3. Is a process for 

ensuring data quality 

described?

Yes (full), a process for ensuring 

data quality is described, with 

details provided.

Yes (limited), a process for 

ensuring data quality is 

described, but with no details 

provided.

No, a process for ensuring data 

quality is not mentioned.

Processes for ensuring that the data collected are 

of high quality, such as checking for the 

completeness, accuracy, consistency, and 

timeliness of the data. For example, there is a plan 

to check participation data entered into a data 

system with paper records of attendance or case 

note file reviews.

Question 4 is skipped with a “no” response.
Planned Scaling Activities Report Mathematica

B.19

Question Response options Additional guidance

4. What was the 

process for ensuring 

the quality of the 

data collected 

through the data 

system?

[open-ended] Describes the process for ensuring that the data 

collected through the data system was of high 

quality.

5. Is a data system to 

support data 

collection, analysis, 

and decision making 

described?

Yes (full), a data system is in 

place, with details provided.

Yes (limited), a data system is 

mentioned, but with no details 

provided.

No, a data system is not 

mentioned.

Data systems (for example, a management 

information system or a standardized Excel 

spreadsheet) capture enrollment and participation 

data and generate reports on participation trends. 

Such systems support data collection, analysis, and 

decision making. A detailed description specifies 

the information captured by the system, the 

organization’s capacity to analyze recorded data, 

and how the organization uses the data to support 

its decision-making processes.

Question 2 is skipped with a “no” response.
Planned Scaling Activities Report Mathematica

B.20

E. Enabling context

This subsection addresses the fourth condition of the scaling readiness framework—the enabling 

context. It includes three aspects of the context in which organizations provide services (Table 

B.I.5): 

1. Support for the intervention, including support from organizational leadership and partner 

agencies for implementing and evaluating the intervention. 

2. Innovation and learning, including earlier activities that the organization might have 

undertaken in an effort to introduce new practices or improve the intervention and the 

organization’s ability to understand the reason for any successes. 

3. Improvement in response to challenges, including the identification of any challenges that 

organizations have encountered during implementation of the intervention and the solutions 

to those challenges. 

Table B.I.5. Questions about the enabling context

Question Response options Additional guidance

I. Leaders, key stakeholders, and partners support for the intervention

1. Is support from 

organizational leaders 

for the intervention 

described?

Yes (full), support from organizational 

leaders for the intervention is 

described, with details provided.

Yes (limited), support from 

organizational leaders for the 

intervention is described, but with no 

details provided.

No, support from organizational leaders 

for the intervention is not mentioned. 

Describes organizational leaders and their 

commitment to the intervention. For example, 

organization leaders are described as 

engaging in planning activities or making 

implementation of the intervention a priority 

for the organization. 

Question 2 is skipped with a “no” response.

2. How was support from 

organizational leaders 

for the intervention 

demonstrated?

[open-ended] Describes how support was demonstrated by 

organizational leaders.

3. Is support from 

stakeholders and/or 

partners for the 

intervention described?

Yes (full), support from stakeholders 

and/or partners for the intervention is 

described, with details provided.

Yes (limited), support from 

stakeholders and/or partners for the 

intervention is described, but with no 

details provided.

No, support from stakeholders and/or 

partners for the intervention was not 

mentioned.

Describes stakeholders’ and/or partners’ 

commitment to the intervention. Stakeholders 

and partners include service delivery partners 

or leadership at implementation sites. Support 

may be demonstrated by securing space for 

service delivery at local implementation sites 

or removing responsibility for non-intervention 

related workload for personnel involved in 

service delivery. 

Question 4 is skipped with a “no” response.

4. How was support from 

organization 

stakeholders and/or 

partners for the 

intervention 

demonstrated?

[open-ended] Describes how support was demonstrated by 

stakeholders and/or partners.
Planned Scaling Activities Report Mathematica

B.21

Question Response options Additional guidance

II. Innovation and learning

1. Is there any 

description of the 

organization’s earlier 

efforts to be 

innovative?

Yes (full), efforts to be innovative are 

described, with details provided.

Yes (limited), efforts to be innovative 

are described, but with no details 

provided.

No, efforts to be innovative were not 

mentioned.

Describes organizational efforts to identify, 

develop, and implement new ways of meeting 

community needs. For example, 

organizational innovation could include 

offering new products or services or 

identifying novel ways of delivering regularly 

available services. 

Question 2 is skipped with a “no” response.

2. What were the 

innovative efforts 

described?

[open-ended] Describes organization’s efforts to be 

innovative.

3. Is there a description of 

the organization’s 

efforts to improve its 

interventions?

Yes (full), efforts to improve its 

interventions are described, with details 

provided.

Yes (limited), efforts to improve its 

interventions are described, but with no 

details provided.

No, efforts to improve its interventions

are not mentioned.

Describes the way in which organizations 

have improved interventions. Improvement 

may be demonstrated by a description of how 

the organization advanced its practices and 

enhanced the current service array. For 

instance, the evaluated intervention may have 

improved upon a previous version of the 

intervention by making changes to its 

recruitment strategies. 

Question 4 is skipped with a “no” response. 

4. What were the 

improvement efforts 

described?

[open-ended] Describes organization’s efforts to improve 

interventions.

5. Is there any discussion 

of successes regarding 

implementation of the 

intervention?

Yes (full), implementation successes 

are discussed, with details provided.

Yes (limited), implementation 

successes are discussed, but with no 

details provided.

No, implementation successes were 

not mentioned.

Describes successes related to implementing 

intervention core elements, recruiting or 

serving the target population(s), supporting 

implementation, and creating an enabling 

context. 

Question 6 is skipped with a “no” response. 

6. What were the 

implementation 

successes of the 

intervention and the 

reasons for those 

successes?

[open-ended] Describes the implementation successes and 

the reasons for those successes. 

III. Improvement in response to challenges

1. Is there any discussion 

of challenges regarding 

implementation of the 

intervention?

Yes, challenges were discussed.

No, challenges were not discussed.

Challenges could be related to intervention 

components, target population(s), 

implementation supports, and the enabling 

context. 

Question 2 is skipped with a “no” response. 

2. What were the 

challenges regarding 

implementation of the 

intervention and 

solutions to those 

challenges?

[open-ended] Includes the implementation challenges and 

the solutions to those challenges, if any. 
Planned Scaling Activities Report Mathematica

B.22

F. Implementation infrastructure

This section of the rubric addresses the fifth condition of the scaling readiness framework— 

implementation infrastructure (Figure I.2 in the main report). It includes questions in four major 

areas, each of which describes the organization’s infrastructure in support of implementation of 

the intervention (Table B.I.6). 

1. Financial resources, including how the organization demonstrates that it or its partners 

successfully supported implementation of the intervention with dedicated financial resources. 

2. Sufficient personnel, including whether the organization dedicated the personnel needed 

either to implement or scale the intervention. 

3. Materials, including the standardization of materials and tailoring of materials to meet 

participants’ needs. 

4. Physical space, including the availability of space necessary for service delivery. 

5. Human resource system, including a human resource system that is critical in typically 

overseeing three highly important, recognized implementation drivers—hiring, training, and 

ongoing supervision. 

Table B.I.6. Questions about implementation infrastructure

Question Response options Additional guidance

I. Financial resources

1. Does the organization 

provide funding for 

implementation of the 

intervention?

Yes (full), adequate funding is in place, 

with details provided.

Yes (limited), adequate funding is in 

place, but with no details provided. 

No, funding is not mentioned.

Describes the amount and source of funds 

and, if applicable, partners’ funding for 

implementation of the intervention.

Question 2 is skipped with a “no” response. 

2. What funding was 

provided by the 

organization for 

implementation of the 

intervention?

[open-ended] Describes the funding provided by the 

organization for implementation of the 

intervention.

II. Sufficient personnel

1. Does the organization 

provide personnel to 

implement the 

intervention (including 

dedicated supervisors 

and service providers)?

Yes (full), the organization provides 

dedicated personnel, with details 

provided.

Yes (limited), the organization provides 

dedicated personnel, but with no details 

provided.

No, dedicated personnel are not 

mentioned.

Describes the number of personnel needed 

for implementation, whether personnel

worked exclusively on intervention 

implementation or also had nonintervention 

responsibilities, and, if applicable, partners’ 

role in providing personnel for the 

intervention.

Question 2 is skipped with a “no” response.

2. How many personnel

were provided by the 

organization to 

implement the 

intervention?

[open-ended] Describe the number of personnel needed 

for implementation.
Planned Scaling Activities Report Mathematica

B.23

Question Response options Additional guidance

III. Materials 

1. Is there a description of 

materials needed for 

the intervention?

Yes (full), intervention materials are 

described, with details provided.

Yes (limited), intervention materials are 

mentioned, but with no details provided.

No, needed intervention materials are 

not mentioned.

Describes the materials needed to 

implement the intervention, such as 

workbooks, culturally sensitive documents, 

or other handouts.

Question 2 is skipped with a “no” response. 

2. What were the 

materials needed for 

the intervention?

[open-ended] Describes the intervention materials needed 

for implementation.

IV. Physical space

1. Is there a description of 

the physical space 

needed for the 

intervention?

Yes (full), the physical space is 

described, with details provided.

Yes (limited), the physical space is 

described, but with no details provided.

No, the physical space is not mentioned.

Describes the physical space needed for 

implementation, such as the size or number 

of classrooms needed to fit the target 

number of participants or the availability of 

private meeting spaces for one-on-one

service delivery.

Question 2 is skipped with a “no” response.

2. What physical space 

was needed for 

implementation of the 

intervention?

[open-ended] Describes the physical space needed for 

implementation.

V. Human resource system

1. Is a human resource 

system in place to hire, 

supervise, and develop 

the personnel? 

Yes (full), a human resource system is 

in place, with details provided.

Yes (limited), a human resource system 

is in place, but with no details provided.

No, a human resource system is not 

mentioned.

A human resource system supports 

implementation through processes and 

procedures for hiring personnel who meet 

qualifications for implementation with fidelity; 

for defining a supervisory structure (including 

the identification of who reports to whom, the 

frequency of supervisory meetings, 

mechanisms for reporting on personnel

progress); and for providing personnel

development (including planned training for 

personnel and opportunities for performance 

monitoring and improvement). A clearly 

defined human resource system specifies 

how these supports are routinized and 

whether procedural guidelines are developed 

to standardize the supports.

Question 2 is skipped with a “no” response.

2. What was the human 

resource system put in 

place for the 

intervention?

[open-ended] Describes the human resource system put in 

place for the intervention.

II. SCALING READINESS SECTION 

The scaling readiness section of the rubric is used to assess an intervention’s readiness to be 

scaled and the organization’s ability to support that scaling as described in the organization’s 

narrative application for funding or in the organization’s scaling plan. This section is used 
Planned Scaling Activities Report Mathematica

B.24

primarily to assess the degree to which the organization specifies details critical for assessing 

that the intervention and the organization will be able to successfully implement the intervention 

and to what degree the intervention proposed for scaling differs from the intervention that was 

evaluated (and described using the previous section of the rubric that was applied to the 

intervention’s evaluation report). The section includes six subsections. One subsection collects 

background information about the approach to scaling and the context in which implementation 

will occur, and the next five subsections collect information on each of the five conditions 

needed for scaling. (Figure I.2 in the main report). 

A. Scaling approach and its context

Table B.II.1 provides the first subsection of the rubric, which includes two types of questions 

about the organization’s approach to scaling and the context in which it occurs: 

1. Scaling approach planned for the intervention. Research from implementation science

includes three forms of scaling. (1) Expansion extends the intervention to more people in the 

same target population in the same location. Successful expansion requires the intervention 

and the organization to serve a larger number of participants with the same service quality 

and in a manner demonstrating fidelity to the model’s design. (2) Replication extends the 

intervention to the same target population, but in a new location. Successful replication 

requires the intervention and the organization to maintain service quality and fidelity to the 

intervention in the new location. (3) Adaptation extends the intervention to a new target 

population. Successful adaption requires the organization to change the intervention in a way 

that maintains service quality. AmeriCorps also funded some AmeriCorps State and National 

organizations to sustain or deepen the intervention services instead of scaling the 

intervention. Some organizations received funds to sustain services, which means that the

intervention will continue serving the same target population in the current location without 

any purposeful changes to it. Some organizations received funds to deepen services, which 

means that the intervention will serve the same target population in the current location, but 

with enhanced services (for example, more hours of job coaching). 

2. Local context and demand associated with the geographic area where the intervention is 

proposed for scaling. Local contextual factors include any regional, state, and local features 

that may affect implementation of the intervention, and local demand factors include 

evidence of demand for the intervention in the local area. 
Planned Scaling Activities Report Mathematica

B.25

Table B.II.1. Questions about scaling approach and local context

Question Response options Additional guidance

I. Scaling approach

1. What is the 

organization’s proposed 

intervention?

[open-ended] Describes the proposed intervention, 

including its core components and 

activities and plans for implementation. 

2. How does the 

organization selfcategorize its scaling 

approach? 

Expansion

Replication

Adaption

Sustaining services

Deepening services

Not described

Describes the self-categorization of the 

scaling approach.

3. Based on the 

description of the 

proposed intervention, 

what is the scaling 

approach? 

Expansion

Replication

Adaption

Sustaining services

Deepening services

Describes the scaling approach as defined 

by implementation science.

4. Will the intervention 

proposed for scaling be 

implemented by the 

organization that 

developed it?

Yes, the intervention proposed for scaling 

will be implemented by the organization 

developing it.

No, the intervention proposed for scaling 

was developed by a different organization.

Describes whether the scaled intervention 

was developed by the implementing 

organization or by another entity. 

II. Local context and demand

1. Does the organization 

provide evidence of 

demand in the local 

area for the intervention 

that is being scaled? 

Yes (full), evidence exists for local demand 

for the intervention being scaled, with 

details provided.

Yes (limited), demand exists for the 

intervention being scaled, but with no 

details provided.

No, local demand is not mentioned or does 

not exist.

Means of illustrating local demand include 

the results of a needs assessment, a 

landscape analysis specifying the scope of 

the intervention, or informal data collection 

via attendance at community meetings or 

conversations with key stakeholders. The 

results of such efforts must motivate the 

proposed scaling approach. 

2. Does the organization 

provide evidence that 

the local, regional, or 

state context will be 

conducive to scaling the 

intervention?

Yes (full), the context will be conducive to 

scaling, with details provided. 

Yes (limited), the context will be conducive 

to scaling, but with no details provided.

No, the context is not mentioned or not 

conducive to scaling.

Evidence exists that key community 

partners will value the intervention and that 

stakeholders at the local, regional, or state 

level will not impede implementation. Such 

evidence might include memorandum of 

understanding, grants, contracts or more 

formal arrangements, as well as tacit 

relationships, such as board memberships.

B. Well-specified intervention

This subsection addresses the first condition of the scaling readiness framework—the wellspecified intervention—as described in the organization’s plan for scaling. It includes overall 

questions on the core elements of the intervention proposed for scaling and their dimensions for 

service delivery, as well as the definition of participants completing the intervention (subsection 

B.I.2 describes these dimensions). The subsection asks questions about five dimensions of 

intervention specification (Table B.II.2): (1) core elements (services, the intervention’s content 
Planned Scaling Activities Report Mathematica

B.26

area, and whether a standardized intervention model is used), (2) mode of service delivery, (3) 

intensity, (4) personnel needs, (5) setting, and (6) definition of completion. For each, questions 

(1) categorize the specificity of the proposed intervention elements, (2) identify differences 

between the proposed intervention element and the evaluated intervention element (as described 

in the intervention evaluation reports and captured in subsection B.I.2 of the rubric), and, if 

applicable, (3) report the rationale for differences between the proposed intervention element and 

evaluated intervention element. If the organization proposes to expand to a new population or 

replicate the intervention, it may not make any changes to the intervention itself. 

Table B.II.2. Questions about specifying the intervention proposed for scaling

Question Response options Additional guidance

I. Core elements 

1. Are the core elements 

of the intervention well 

specified?

Yes (full), core elements are well 

specified, with details provided.

Yes (limited), core elements are 

specified, but with no details provided.

No, core elements are not mentioned.

Core elements are the set of activities or 

curricula that result in participants’ receipt of 

services, such as a description of services, 

the content area of the intervention, the 

existence of a standardized intervention

model, the services or curriculum comprising 

the intervention, and the content area of the 

intervention to be delivered. A standardized 

intervention is considered to have a clearly 

specified intervention. 

2. Do the proposed core 

elements differ from 

those in the evaluation? 

Yes (full), proposed core elements differ 

from those evaluated, with details of the 

differences provided.

Yes (limited), proposed core elements 

differ from those evaluated, but with no 

details provided.

Do not know; proposed core elements 

are not defined.

No, proposed core elements for the 

evaluation and intervention are the 

same. 

The proposed core elements of the 

intervention for scaling may differ from the 

core elements of the intervention that were 

evaluated. Changes to the intervention could 

include the addition of services, the 

modification of existing services, the 

discontinuation of some services, the 

addition of content, or the use of a new 

curriculum, all described in sufficient detail to 

allow an external audience to replicate them.

Question 3 is skipped with a “don’t know” or 

“no” response.

3. Does a rationale or 

support exist for the 

proposed change(s) to 

the core elements? 

Yes, a rationale or support for the 

change(s) is provided.

No, a rationale or support is not 

provided. 

Organizations may provide a reason for 

differences between the proposed 

intervention core elements and the evaluated 

core elements. Support might refer, for 

example, to research showing that a 

proposed change has positive impacts on 

outcomes.
Planned Scaling Activities Report Mathematica

B.27

Question Response options Additional guidance

II. Mode of service delivery

1. Is the service delivery 

model well-specified? 

Yes (full), the service delivery model is 

well-specified, with details provided.

Yes (limited), the service delivery model 

is specified, but with no details provided. 

No, the service delivery model is not 

mentioned.

The service delivery model specifies how 

services are delivered, who delivers services, 

and the mode through which services are 

delivered. For example, services may be 

delivered in groups or one-on-one meetings, 

by peer educators or teams of facilitators, 

and the through the use of electronic 

communication or in-person meetings. A 

standardized intervention clearly specifies 

the service delivery model. 

2. Does the proposed 

service delivery model 

differ from the service 

delivery model 

implemented during the 

evaluation? 

Yes (full), the proposed service delivery 

model differs from the intervention 

evaluated, with details provided.

Yes (limited), the proposed service 

delivery model differs from the 

intervention evaluated, but with no 

details provided.

Do not know; the service delivery model 

is not defined.

No, the proposed and evaluation service 

delivery models are the same. 

The proposed service delivery model for the 

intervention for scaling may differ from the 

service delivery model of the intervention that 

was evaluated. Changes to the service 

delivery model could include the type of 

delivery (such as face-to-face meetings or 

electronic communications) and the nature of 

delivery (such as in a group or one-on-one 

meetings). 

Question 3 is skipped with a “don’t know” or 

“no” response.

3. Does a rationale or 

support exist for the 

proposed change(s) to 

the service delivery 

model? 

Yes, a rationale or support for the 

change(s) is provided.

No, a rationale or support for the 

proposed change(s) is not provided.

Organizations may provide a reason for 

differences between the proposed 

interventions’ service delivery mode and the 

evaluated service delivery mode. Support 

could include research showing that a 

proposed change has positive impacts on 

outcomes.

III. Intensity 

1. Is the intensity of the 

intervention wellspecified?

Yes (full), the intensity is well-specified, 

with details provided.

Yes (limited), the intensity is specified, 

but with no details provided.

No, the intensity is not mentioned.

Describes the total number of hours of 

intervention programming participants will 

receive. This is based on how long an 

intervention will last (duration) and the

amount of services participants will receive 

(dosage), including the frequency of 

intervention interaction and hours of service. 

2. Does the proposed 

intensity of the scaled 

intervention differ from 

the intensity of the 

intervention evaluated?

Yes (full), the proposed intensity differs 

from the intervention evaluated, with 

details of the differences provided.

Yes (limited), the proposed intensity 

differs from the intervention evaluated, 

but with no details provided.

Do not know; the intensity is not defined.

No, the proposed and evaluated 

intensity is the same.

The proposed intensity of the intervention for 

scaling may differ from the intensity of the 

intervention that were evaluated. Differences 

may include changes to how long an 

intervention lasts, how many times a 

participant and a service provider meet, and 

how many hours of service content a 

participant receives.

Question 3 is skipped with a “don’t know” or 

“no” response.
Planned Scaling Activities Report Mathematica

B.28

Question Response options Additional guidance

3. Does a rationale or 

support exist for the 

proposed change(s) to 

the intensity of the 

intervention? 

Yes, a rationale for the change(s) is 

included, with details provided.

No, a rationale for the change(s) to 

dosage or duration is not mentioned.

Organizations may provide a reason for 

differences between the proposed 

interventions’ intensity and the evaluated 

intensity. Support might include research 

showing that a proposed change has positive 

impacts on outcomes.

IV. Personnel needs

1. Are the qualifications 

clearly specified for the 

personnel involved in 

delivering the 

intervention?

Yes (full), personnel qualifications are 

clearly specified, with details provided.

Yes (limited), personnel qualifications 

are specified, but with no details 

provided.

No, personnel qualifications are not 

mentioned.

Clearly specified qualifications for each job 

title involved in delivering services for the 

scaled intervention. Qualifications could 

include required training or education and 

experience levels of the personnel delivering 

the intervention services. 

2. Do the proposed 

personnel qualifications 

after scaling differ from 

the qualifications of 

personnel delivering 

services as part of the 

evaluation? 

Yes (full), the proposed personnel

qualifications differ from the intervention 

evaluated, with details of the differences 

provided.

Yes (limited), the proposed personnel

qualifications differ, but with no details of 

the differences provided.

Do not know; the personnel

qualifications are not defined.

No, the proposed and evaluation 

personnel qualifications are the same.

The proposed personnel of the intervention 

for scaling may differ from the personnel of 

the intervention that were evaluated. 

Differences in personnel qualifications may 

include education, experience, or various 

levels of training.

Question 3 is skipped with a “don’t know” or 

“no” response.

3. Does a rationale or 

support exist for the 

proposed change(s) to 

the personnel

qualifications? 

Yes, a rationale or support exists for the 

personnel qualifications change(s).

No, a rationale for the proposed 

change(s) is not mentioned.

Organizations may provide a reason for 

differences between the proposed 

interventions’ personnel and the personnel of 

the evaluated intervention. A rationale could 

include research showing that a proposed 

change has positive impacts on outcomes. 

V. Setting 

1. Is the implementation 

setting clearly 

specified?

Yes (full), the implementation setting is 

clearly specified, with details provided.

Yes (limited), the implementation setting 

is specified, but with no details provided.

No, the implementation setting is not 

mentioned.

Implementation setting includes the 

geographic location of implementation and 

where an intervention takes place. The 

description of the implementation setting may 

include the city and state; whether the 

intervention is being implemented in a 

school, community-based organization, or 

workforce center; and the number of 

implementation settings. The number of 

implementation sites is not considered a 

change unless the new sites are located in a 

new city, state, or type of setting (such as a 

different implementing agency).
Planned Scaling Activities Report Mathematica

B.29

Question Response options Additional guidance

2. Does the proposed 

implementation setting 

differ from the 

evaluation’s 

implementation setting?

Yes (full), the proposed setting differs 

from the evaluation setting, with details 

provided.

Yes (limited), the proposed setting 

differs from the evaluation setting, but 

with no details provided.

Do not know; the setting is not 

described.

No, the proposed and evaluation setting 

are the same.

The proposed implementation setting of the 

intervention for scaling may differ from the 

implementation setting of the intervention 

that was evaluated. Differences in the 

implementation setting might include 

delivering services in a new city, state, 

region, or urban/rural area as well as 

switching from a community-based to a 

school-based intervention. 

Question 3 is skipped with a “don’t know” or 

“no” response.

3. Does a rationale or 

support exist for the 

proposed change(s) to 

the implementation 

setting?

Yes, a rationale exists for the change(s). 

No, a rationale is not provided for the 

proposed change(s).

Organizations may provide a reason for 

differences between the implementation 

setting of the proposed intervention and the 

setting of the evaluated intervention. A 

rationale could include research showing that 

a proposed change has positive impacts on 

outcomes.

VI. Definition of completion

1. Are the criteria for 

completion wellspecified?

Yes (full), the criteria for completion are 

well-specified, with details provided.

Yes (limited), the criteria for completion 

are specified, but with no details 

provided.

No, the criteria for completion are not 

mentioned. 

Criteria for completion indicate what 

participants must do to be considered 

“completers” or “graduates” for the 

intervention for scaling. To sufficiently specify 

the completion criteria, there must also be a 

definition of a “participant,” including the 

minimum amount of services needed to be 

received in order to be considered a 

“participant.” For example, to be considered 

a participant, a youth must attend at least 1 

case manager meeting; to be considered a 

completer or graduate, participants must 

attend 10 case manager meetings.

2. Do the criteria for 

completion differ from 

the criteria described in 

the evaluated 

intervention?

Yes (full), the proposed completion 

criteria differ from the criteria for the 

intervention evaluated, with details 

provided.

Yes (limited), the proposed completion 

criteria differ from the criteria for the 

intervention evaluated, but no details 

are provided.

Do not know; the completion criteria are 

not described.

No, the proposed and evaluated criteria 

for completion are the same.

The completion criteria of the intervention for 

scaling may differ from the completion 

criteria of the evaluated intervention. 

Changes to the criteria for completion might 

include requiring a new minimum amount of 

services to be attended to be considered a 

participant or a change in the number of 

services received to be considered a 

graduate of the intervention.

Question 3 is skipped with a “don’t know” or 

“no” response.

3. Does a rationale or 

support exist for the 

proposed change(s) to 

the participant 

completion definition?

Yes, a rationale or support for the 

change(s) is provided.

No, a rationale or support for the 

proposed change(s) is not provided.

Organizations may provide a reason for 

differences between the completion criteria 

of the proposed intervention and the 

completion criteria of the evaluated 

intervention. A rationale could include 

research showing that a proposed change 

has positive impacts on outcomes.
Planned Scaling Activities Report Mathematica

B.30

C. Well-defined target population

This subsection of the rubric addresses the second condition of the scaling readiness 

framework—well-defined target population (Figure I.2 in the main report). It includes questions 

discussed in subsection B.I.3 about the target population proposed for scaling (Table B.II.3) and 

describes differences between the proposed target population after scaling and the population 

studied in the evaluation. If the organization proposes to expand the population size or replicate 

the intervention, it may not make any changes to the target population. 

Table B.II.3. Questions about the target population proposed for scaling

Question Response options Additional guidance

1. Is the target population 

well-specified?

Yes (full), the target population is 

clearly specified, with details provided.

Yes (limited), the target population is 

specified, but with no details provided.

No, the target population is not 

mentioned.

Describes who is eligible to participate in the 

intervention, including inclusion and exclusion 

criteria. For example, a proposed intervention 

to serve unemployed adults should include 

examples of eligible age ranges (for example, 

18-64), length of unemployment, and any other 

inclusion or exclusion criteria such as no felony 

conviction.

2. Does the proposed 

target population differ 

from the evaluation 

sample? 

Yes (full), the proposed and 

evaluation populations differ, with 

details of the differences provided.

Yes (limited), the proposed and 

evaluation populations differ, but with 

no details of the differences provided.

Do not know; the populations are not 

defined.

No, the proposed and evaluation 

populations are the same. 

Changes to the target population include age 

requirements (for example, expanding the 

eligible age range from 11- to 13-year-old 

children to 10- to 14-year-old children), 

personal characteristics (for example, risk 

factors or education levels), or eligibility (for 

example, veterans or youth). 

Question 3 is skipped with a “don’t know” or 

“no” response. 

3. Does a rationale or 

support exist for the 

proposed change(s) to 

the target population?

Yes, a rationale exists for the 

change(s), with details provided.

No, no rationale for the change(s) is 

mentioned.

A rationale could include research showing that 

a proposed change has had positive impacts 

on outcomes in other studies or that 

modifications could be based on the 

recommendations. 

D. Implementation supports

This section of the rubric addresses the third condition of the scaling readiness framework—

implementation supports (Figure I.2 in the main report). It includes questions about the 

implementation supports discussed in subsection B.I.4 that are available for the scaled 

intervention (Table B.II.4): (1) implementation monitoring team, (2) performance procedures, (3) 

CQI, (4) preservice and inservice training for the personnel, (5) communication system, and (6) 

data system.
Planned Scaling Activities Report Mathematica

B.31

Table B.II.4. Questions about implementation supports for the intervention proposed for 

scaling

Question Response options Additional guidance

I. Implementation monitoring team

1. Is a team proposed to 

monitor implementation 

of the scaled 

intervention? 

Yes (full), a team is proposed, with 

details provided.

Yes (limited), a team is described, 

but with no details provided.

No, a team is not mentioned.

A monitoring team ensures that implementation 

takes place as planned. Team members may 

play other roles in the organization (such as 

supervisors, project managers, and project 

directors) or they may be other personnel

dedicated exclusively to ensuring that the 

intervention is implemented with fidelity to the 

model. Details include specific information about 

members of the implementation team. 

2. Does a process for 

monitoring fidelity to the 

intervention model exist 

for the scaled 

intervention?

Yes (full), a process for monitoring 

fidelity to the intervention model is 

described, with details provided.

Yes (limited), a process for 

monitoring fidelity to the intervention 

is described, but with no details 

provided.

No, a process for monitoring fidelity 

is not mentioned. 

A process for monitoring fidelity to the 

intervention model ensures that services are 

being delivered as intended. The process for 

monitoring implementation fidelity could include 

collecting service delivery data through 

observations during site visits or regular reviews 

of service data entered into a data system. 

Details include identification of the individual (by 

job title) responsible for assessing fidelity, 

frequency of implementation monitoring, 

processes for managing/supervising personnel, 

and frequency of personnel management or 

supervisory meetings.

II. Performance procedures

1. Do performance 

benchmarks exist for 

personnel involved in 

service delivery of the 

scaled intervention?

Yes (full), performance benchmarks 

are described, with details provided.

Yes (limited), performance 

benchmarks are described, but with 

no details provided.

No, performance benchmarks are 

not mentioned.

Performance benchmarks are specific goals that 

personnel must meet as related to delivery of the 

scaled intervention, such as such as the number 

of participants contacted per personnel member 

or the number of participants served per 

personnel member. 

2. Do procedures for 

monitoring achievement 

of performance 

benchmarks exist for 

the scaled intervention?

Yes (full), procedures for monitoring 

achievement of performance 

benchmarks are described, with 

details provided.

Yes (limited), procedures for 

monitoring achievement of 

performance benchmarks are 

described, but with no details 

provided.

No, procedures for monitoring 

achievement of performance 

benchmarks are not mentioned. 

Procedures for monitoring achievement of 

performance benchmarks include how 

benchmarks are measured and collected, who 

reviews progress toward benchmarks, and the 

frequency of performance monitoring.
Planned Scaling Activities Report Mathematica

B.32

Question Response options Additional guidance

III. Continuous quality improvement (CQI)

1. Are CQI processes 

proposed for the scaled 

intervention? 

Yes (full), CQI processes are 

proposed, with details provided. 

Yes (limited), CQI processes are 

proposed, but with no details 

provided.

No, CQI processes are not 

mentioned.

CQI processes refer to procedures for 

continuously assessing the quality of the 

intervention as implemented to improve 

implementation practice. CQI includes regularly 

testing the intervention and making adjustments 

as needed, with continual retesting of the 

modifications to ensure that the intervention is 

succeeding as planned. Details include a 

description of the data collected throughout this 

process. 

2. Does a plan exist to 

analyze data to support 

CQI for the scaled 

intervention?

Yes (full), data analysis to support 

CQI are described, with details 

provided.

Yes (limited), data analysis to 

support CQI are described, but with 

no details provided.

No, data analysis to support CQI is 

not mentioned.

CQI requires the collection of data on 

performance benchmarks and use of the data to 

provide ongoing personnel development training 

and the delivery of technical assistance to 

partners. Describes plans for using data to 

support CQI includes examining personnel

achievement of performance benchmarks. 

IV. Preservice and inservice training for personnel

1. Do plans exist to 

provide initial training to 

personnel to deliver the 

scaled intervention? 

Yes (full), initial training plans are 

described, with details provided.

Yes (limited), initial training plans 

are described, but with no details 

provided.

No, initial training plans are not 

mentioned.

Training for service delivery given to personnel

before implementation begins includes training on 

intervention content (such as a training on a 

curriculum used in the intervention), methods for 

service delivery (such as motivational 

interviewing), and client processing procedures 

(such as intake procedures).

2. Do plans exist to 

provide ongoing training 

to personnel to 

implement the scaled 

intervention?

Yes (full), ongoing training plans are 

described, with details provided.

Yes (limited), ongoing training plans 

are described, but with no details 

provided.

No, ongoing training plans are not 

mentioned.

Training for service delivery given to personnel

during implementation and throughout the service 

delivery period, includes refresher trainings on 

intervention content, methods for service, and 

client processing procedures (such as intake 

procedures).

V. Communication system

1. Does a communication 

system exist to support 

coordination among 

personnel and partners 

for the scaled 

intervention?

Yes (full), a communication system 

exists, with details provided.

Yes (limited), a communication 

system is mentioned, but with no 

details provided.

No, a communication system is not 

mentioned.

Communication systems support coordination 

among personnel and partners and specify the 

frequency and expected duration of 

communication and the parties responsible for 

communication. Systems may vary in the extent 

to which they specify and standardize 

communication. For example, a detailed plan 

may state that, for the first six months of the 

intervention, service providers must meet as a 

group with their supervisor for one hour each 

week to discuss topics related to recruitment and 

engagement of participants; for the next six 

months, meetings must take place monthly in the 

form of hour-long sessions to discuss service 

provision. 
Planned Scaling Activities Report Mathematica

B.33

Question Response options Additional guidance

2. Do processes exist for 

ensuring the 

communication system 

is functioning as 

intended for the scaled 

intervention?

Yes (full), processes for ensuring 

the communication system is 

functioning as intended are 

described, with details provided.

Yes (limited), processes for 

ensuring the communication system 

is functioning as intended are 

described, but with no details 

provided.

No, processes for ensuring the 

communication system is 

functioning as intended are not 

mentioned. 

Processes to ensure a communication system 

functions as intended include obtaining feedback 

from personnel regarding the communication 

system and tracking the communication that 

takes place. Details include how feedback was 

obtained regarding the functioning of the 

communication system. For example, a 

communication system may have included logs of 

how often personnel met together or used regular 

check-ins with personnel eliciting feedback on 

how easily they feel they are able to get in 

contact with each other or key intervention 

partners.

VI. Data system

1. Does a data system 

exist to support data 

collection, analysis, and 

decision making for the 

scaled intervention?

Yes (full), a data system is 

described, with details provided.

Yes (limited), a data system is 

described, but with no details 

provided.

No, a data system is not mentioned.

Data systems capture enrollment and 

participation data and generate reports on 

participation trends. Such systems support data 

collection, analysis, and decision making. Details 

include the information captured by the system, 

the organization’s capacity to analyze recorded 

data, and how the organization uses the data to 

support its decision-making processes.

2. Does a process exist for 

ensuring the data 

quality for the scaled 

intervention?

Yes (full), a process for ensuring 

the data quality is described, with 

details provided.

Yes (limited), a process for ensuring 

the data quality is described, but 

with no details provided.

No, a process for ensuring the data 

quality is not mentioned.

Processes for ensuring that the data collected are 

of high quality, such as checking for the 

completeness, accuracy, consistency, and 

timelines of the data. For example, there is a plan 

to check participation data entered into a data 

system with paper records of attendance or case 

note file reviews.

E. Enabling context

This subsection addresses the fourth condition of the scaling readiness framework—the enabling 

context. It includes questions about the implementation supports discussed in subsection B.I.5 

that are available for the scaled intervention (Table B.II.5). Support for scaling pertains to the 

organization’s leaders and key stakeholders, as well as to the organization’s culture and its focus 

on innovation, learning, and improvement. The enabling context involves questions in three 

major areas about the system in which organizations provide services: (1) organizational 

leadership and partner support, (2) innovation and learning and (3) improvements in response to 

challenges.
Planned Scaling Activities Report Mathematica

B.34

Table B.II.5. Questions about the enabling context for the organization proposing the 

scaling

Question Response options Additional guidance

I. Leaders, key stakeholders, and partners support for the intervention

1. Is support from 

organizational leaders 

for scaling the 

intervention described?

Yes (full), support from organizational 

leaders for scaling the intervention is 

described, with details provided.

Yes (limited), support from 

organizational leaders for scaling the 

intervention is described, but with no 

details provided.

No, support from organizational 

leaders for scaling is not mentioned.

Describes organizational leaders and their 

commitment to scaling the intervention. For 

example, organization leaders are part of the 

team that is overseeing the AmeriCorps grant 

and the scaling plan for the intervention.

Question 2 is skipped with a “no” response.

2. Is support from 

stakeholders and/or 

partners for scaling the 

intervention described?

Yes (full), support from stakeholders 

and/or partners for scaling the 

intervention is described, with details 

provided.

Yes (limited), support from 

stakeholders and/or partners for 

scaling the intervention is described,

but with no details provided.

No, support from stakeholders and/or 

partners for scaling the intervention is 

not mentioned.

Describes stakeholders and/or partners’ 

commitment to scaling the intervention. 

Stakeholders and partners include service 

delivery partners or leadership at 

implementation sites. Support may be 

demonstrated by securing space for service 

delivery or removing responsibility for nonintervention related workload for service 

providers so they can focus on scaling the 

intervention.

II. Innovation and learning

1. Has the organization 

previously participated 

in efforts to be 

innovative? 

Yes (full), efforts to be innovative are 

described, with details provided.

Yes (limited), efforts to be innovative 

are described, but with no details 

provided.

No, efforts to be innovative were not 

mentioned.

Describes organizational efforts to identify, 

develop, and implement new ways of meeting 

community needs. For example, organizational 

innovation could include offering new products 

or services or identifying novel ways of 

delivering regularly available services. 

2. Has the organization 

previously participated 

in activities to improve 

its interventions? 

Yes (full), efforts to improve its 

interventions were described, with 

details provided.

Yes (limited), efforts to improve its 

interventions were described, but with 

no details provided.

No, efforts to improve its interventions

were not mentioned.

Describes the way in which organizations have 

improved interventions. Improvement may be 

demonstrated by a description of how the 

organization advanced its practices and 

enhanced the current service array. For 

instance, the evaluated intervention may have 

improved upon a previous version of the 

intervention by making changes to its 

recruitment strategies. 

III. Improvements in response to challenges

1. Did the organization 

previously face 

challenges in 

supporting intervention 

implementation?

Yes (full), the organization faced 

challenge(s), with details provided.

Yes (limited), the organization faced 

challenges, but with no details 

provided.

No, no challenges are described.

Describes earlier challenges experienced by 

the organization in providing organizational 

support for implementation of the intervention. 

For example, organizational leaders might 

have changed or a previous leader might not 

have endorsed the intervention and thus 

refused to dedicate personnel to its 

implementation. 

Question 2 is skipped with a “no” response. 
Planned Scaling Activities Report Mathematica

B.35

Question Response options Additional guidance

2. Did the organization 

make improvements to 

address earlier 

challenges in providing 

organizational support 

for the scaled 

intervention?

Yes (full), the organization made 

improvements to address 

challenge(s), with details provided.

Yes (limited), the organization made 

improvements to address challenges, 

but with no details provided.

No, improvements to address 

challenges were not mentioned.

Describes improvements made to address 

challenges that may include a change in 

organizational structure or the identification of 

new funding sources.

F. Implementation infrastructure

This subsection addresses the fifth condition of the scaling readiness framework—

implementation infrastructure. It includes questions about the implementation infrastructure

discussed in subsection B.I.6 that are available for the scaled intervention (Table B.II.6): (1) 

financial resources, (2) sufficient personnel to implement the intervention, (3) materials, (4) 

physical space, and (5) a human resource system. 

Table B.II.6. Questions about implementation infrastructure for the organization 

proposing the scaling

Question Response options Additional guidance

I. Financial resources

1. Will the organization 

provide funding for the 

scaled intervention? 

Yes (full), funding for scaling exists, with 

details provided.

Yes (limited), funding after scaling exists, 

but with no details provided.

No, funding for scaling is not mentioned.

Describes how the organization and, if 

applicable, its partners will provide funding 

to implement the intervention after scaling. 

Details include the amount and source of 

funds and the plan to continue providing 

funds in the future.

II. Sufficient personnel

1. Will the organization 

provide dedicated 

personnel to implement 

the scaled intervention? 

Yes (full), personnel plans are described, 

with details provided.

Yes (limited), personnel plans exist, but 

with no details provided.

No, personnel plans are not mentioned.

Describes the number of personnel who 

will work on implementation (as compared 

to the number required by the intervention 

model), whether personnel will work 

exclusively on intervention implementation, 

and, if applicable, partners’ role in 

providing personnel to implement the 

intervention.

III. Materials 

1. Will the organization 

have the materials 

needed for the scaled 

intervention?

Yes (full), materials are described, with 

details provided.

Yes (limited), materials are mentioned, 

but with no details provided.

No, materials are not mentioned.

Describes how the organization will obtain 

materials needed for implementation after 

scaling, such as workbooks, culturally 

sensitive documents, or other handouts. 
Planned Scaling Activities Report Mathematica

B.36

Question Response options Additional guidance

IV. Physical space

1. Will the organization 

have the physical space 

needed for the scaled 

intervention?

Yes (full), the physical space is 

described, with details provided.

Yes (limited), the physical space is 

mentioned, but with no details provided.

No, the physical space is not mentioned.

Describes how the organization has or will 

obtain the physical space needed for 

implementation after scaling, such as 

classrooms to fit the target number of 

participants or private meeting spaces for 

one-on-one service delivery.

V. Human resource system

1. Is a human resource 

system in place to hire, 

supervise, and develop 

the personnel for the 

scaled intervention? 

Yes (full), a human resource system is in 

place, with details provided.

Yes (limited), a human resource system 

is mentioned, but with no details 

provided.

No, a human resource system is not 

mentioned.

Describes a human resource system to 

support implementation after scaling. 

Details specify how the human resource 

system supports processes and 

procedures for hiring appropriate 

personnel, describe a supervisory 

structure, and provide for personnel

development, including a description of 

how supports are routinized and 

standardized. 
 

This page has been left blank for double-sided copying.
Mathematica

Princeton, NJ • Ann Arbor, MI • Cambridge, MA 

Chicago, IL • Oakland, CA • Seattle, WA

Tucson, AZ • Woodlawn, MD • Washington, DC 

EDI Global, a Mathematica Company

Bukoba, Tanzania • High Wycombe, United Kingdom

mathematica.org