PIONEERING AN

EVIDENCE-BASED

INVESTMENT MODEL
The Social Innovation Fund: Pioneering an Evidence-Based Investment Model 

January 2015

Authors 

Christopher Spera, Ph.D. 

Adrienne DiTommaso, M.P.A. 

Mary Morris Hyde, Ph.D. 

Lily Zandniapour, Ph.D. 

Acknowledgements 

The authors would like to acknowledge and thank the following reviewers for their comments and suggestions in writing 

this paper: Ms. Kirsten Breckenridge, Dr. Martha Moorehouse, Ms. Lois Nembhard, and Dr. Nicole Vicinanza. They 

would also like to thank JBS International for their support of the SIF evaluation work under the direction of CNCS. 

Citation 

Corporation for National and Community Service, Office of Research and Evaluation. (2014). The Social Innovation Fund:

Pioneering an Evidence-Based Investment Model. (by Christopher Spera, Adrienne DiTommaso, Mary Morris Hyde, and 

Lily Zandniapour). Washington, DC: Author.

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. Upon request, this 

material will be made available in alternative formats for people with disabilities. 
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 2

Introduction 

The Social Innovation Fund (SIF) is a groundbreaking, evidence-based approach to finding, improving and growing 

promising community solutions. This paper is part of a series designed to share information, best practices, and lessons 

learned from the development and implementation of the SIF. It provides an overview of the SIF initiative, an explanation 

of the SIF’s innovative learning strategy, a description of SIF-supported interventions (programs that address social and 

community challenges) and evidence of their effectiveness as well as the rigorous process used to evaluate SIF-funded 

programs.1

1

 A glossary of key terms can be found at the end of this paper. 

What is the Social Innovation Fund? 

The SIF, a key White House initiative and program of the Corporation for National and 

Community Service (CNCS), is one of six federal tiered innovation and evidence 

initiatives that prioritizes evaluation and building of evidence. The SIF combines public 

and private resources to grow the impact of innovative, community-based solutions that 

have compelling evidence of improving the lives of people in low-income communities 

throughout the United States. The SIF invests in three priority areas: economic 

opportunity, healthy futures, and youth development.

CNCS launched the SIF in 2010, following President Obama’s signing of the Serve America 

Act. The SIF invests in promising programs that address pressing social and community 

challenges and uses a rigorous process to evaluate each program, building evidence of its 

impact and effectiveness. The evaluations create a robust evidence base that demonstrates 

whether SIF-funded programs are achieving intended outcomes and impact on 

communities. The evidence base also promotes an understanding of how programs achieve success and ways to improve 

them. 

By developing a strong evaluation process, providing resources to support quality evaluations, and encouraging the SIF’s 

intermediary grantmaking organizations and subgrantees to do the same, the SIF enables stakeholders to improve their 

own evaluation capacity and processes. This transparent approach helps achieve another mission-critical goal: sharing best 

practices and lessons learned from evidence-based grant-making with the broader social sector. 

CNCS funds SIF grants through annual Congressional appropriations, with $70 million earmarked for the SIF in 20142. 

2

 In prior years, the SIF congressional appropriation totaled approximately $50 million annually. 

Using a competitive process, CNCS selects highly successful intermediaries to implement the program, with responsibility 

for selecting nonprofits for SIF funding and overseeing their work. 

Key Takeaway 

A total of 20 nonprofit 

grantmaking intermediaries 

were funded from 2010- 

2012, with 226 sub-grantees 

receiving grants in the first 

three years. To date, the first 

three groups receiving SIFfunding have sponsored 87 

programs, with 77 program 

evaluations underway. 

Multiplying Federal Funding 

The SIF’s grantmaking model uses a unique formula to sustain funding from non-federal sources. It almost triples the 

initial federal investment in a two-step process. Intermediaries selected through a competitive process must match federal 

grants, dollar for dollar, by raising additional private funds. The intermediaries, in turn, re-grant SIF funding to 

subgrantees in one of the SIF’s three focus areas: economic opportunity, youth development (from early childhood 

through transitions to adulthood), and healthy futures. Subgrantees also are required to match the federal investment, 

one-to-one, creating a powerful multiplier effect. 

Between FY 2010 to FY2012, the SIF awarded a total of $137.7 million in federal grants. Matches multiplied the potential 

reach and effectiveness of programs. The 20 selected intermediaries and their 226 nonprofit subgrantees raised an 
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 3

additional $350 million in funding for promising community programs3. 

3 Detailed funding information is available on the CNCS website: http://www.nationalservice.gov/programs/social-innovation-fund/funded-organizations

Table 1: Number of SIF Intermediaries and Subgrantees by Year 

Number of SIF Intermediaries and Subgrantees, by Year

2010 2011 2012 Total

Intermediaries 11 5 4 20

Subgrantees 160 49 17 226

Source: Corporation for National & Community Service

Improving Community Results 

The SIF represents a new way of doing business for the federal government. Targeted federal investment serves as the 

catalyst to deepen and broaden the impact of promising solutions already delivering results. The SIF’s unique operating 

model is built on the premise that local communities know what works to address local challenges and increases their 

ability to transform more lives. By mobilizing public-private partnerships, requiring high standards of evidence of 

programs’ effectiveness and delivering an 80 percent match to taxpayer dollars, the SIF accelerates the flow of resources to 

programs tackling some of the nation’s most intractable problems. More than 174,000 people have benefitted from initiatives 

that open up employment and educational opportunities, increase student achievement, and improve individuals’ access 

to medical care. 

Grantees pursue local and national strategies to scale up their programs and evaluate the impact of their solutions on 

communities. In addition to demanding evaluations that capture program effectiveness, the SIF shares results, lessons 

learned and best practices. Insights help inform decisions about how to allocate limited funds and resources, including the 

criteria for selecting programs and growing them, as well as ways to continually improve programs. By systematically 

capturing and sharing what works, the SIF generates critical knowledge for other organizations in the social sector. 

Figure 1. How the SIF Improves Community Outcomes

 

Public/Private Partnership

Intermediaries and the nonprofits 

they fund must match Federal 

SIF $ with private sector $

High-Performing Organizations & Promising Programs 

Resources are allocated to 

organizations with capacity to 

identify and grow nonprofits and 

implement evidence-based 

programs

Evaluation and Evidence

Rigorous evaluation studies and 

results are used to guide 

program improvement efforts 

and grantmaking practices 
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 4

Learning What Works 

The SIF is a learning initiative, rooted in program data and evaluations that highlight which 

programs are working, under what conditions and who benefits. Just as important, the 

scientifically based process identifies which programs are not yielding results, enabling 

improvements and redesigns. The SIF’s learning strategy is framed by three key 

knowledge-building efforts: a Knowledge Initiative, a national assessment, and 

comprehensive program evaluations. 

Knowledge Sharing: CNCS designed the SIF’s Knowledge Initiative to document and share SIF lessons and best 

practices with those in its portfolio and other nonprofits, funders and government agencies. It nurtures a culture of 

learning that advances the use of SIF-style grantmaking for economic opportunity, youth and health programs. 

National Assessment: A systematic assessment of the SIF’s groundbreaking funding practices will review how 

successfully the SIF and its grantees4 have implemented required activities: 







4

 Grantees under the SIF are grantmaking organizations that identify, select, and build the capacity of nonprofit organizations (subgrantees) that address community needs in the areas 

of economic opportunity, youth development and healthy futures. 

Creating public-private funding partnerships, 

Using evidence and comprehensive evaluations, and 

Conducting open competition for extending subgrants to community-based organizations. 

The assessment also will help determine an organization’s ability and capacity to meet funding requirements and 

the level of support needed to successfully administer grants. Finally, it will assess trajectories for achieving 

program growth and sustainability. 

The results allow others in the field to clearly understand how the SIF works and the implications of public-private 

funding models and evidence-based programs for the larger field of social innovation. 

Evaluating Every Program 

Nonprofits often lack the expertise, resources, or infrastructure to evaluate their efforts, demonstrate impact and take their 

programs from “promising” to “proven.” While many grant programs do not provide support or funding for evaluations, 

the SIF provides both. Evaluations identify opportunities for greater effectiveness and help ensure that federal dollars are 

invested in programs with evidence of results. 

The first three groups of subgrantees are in the process of conducting 87 evaluations. They vary in design, scope and 

outcomes, but they all include steps required to achieve the subgrantees long-term goals and a statistical outcome analysis, 

with most evaluations including a comparison group. The first results arrived in 2013 and the rest will be staggered over 

the next four years, with plans to make them public on the CNCS website. Evaluations of subgrantee clusters with similar 

programs make it possible to compare effective practices, helping to achieve similar outcomes. 

Key Takeaway 

SIF is a learning initiative, 

and much can be learned 

from both the successes 

and challenges. 
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 5

Building the Evidence Base for Social Innovation 

“CNCS expects grantees, to the extent practicable, to fund subgrantees with rigorous evidence of their 

impact. CNCS recognizes, however, that in many parts of the country, and in many fields, such 

evidence will not yet be available. In these areas, the Corporation will prioritize intermediaries that 

are prepared to build portfolios that, over time, are most likely to demonstrate strong evidence of 

strong impact.” 

– 2010 Social Innovation Fund Notice of Funding Opportunity 

The SIF is unique in the breadth and diversity of the programs it funds, allowing it to support the development of innovative 

solutions. During the application process, subgrantees select a prime focus from one of the SIF’s three targeted areas, or a 

geographic focus, or both. The overwhelming majority of programs – 72 percent – target Youth Development, while 16 

percent focus on Healthy Futures, and 12 percent increase Economic Opportunities (see Table 2).5 

5

 All funded programs are expected to undergo rigorous evaluation over time. A subset of programs has approved evaluation plans in place and is implementing them; others are in 

various stages of plans development and review. 

Once funding is in place, the evaluation process begins. Rigorous evaluations are the cornerstone of the SIF’s evidencebased model. They document and assess whether a program’s approach is more effective, costs less or leads to better results 

for communities. As a condition of SIF funding, both intermediary grantmakers and subgrantees are required to conduct 

evaluations. Intermediaries must demonstrate a track record of using evaluations to make programming decisions. 

Subgrantees conduct comprehensive program evaluations to strengthen their evidence base. All programs have to meet a 

minimum –“preliminary”—evidence threshold to qualify for SIF funding.

By investing in evaluations, the SIF is building robust evidence about the most effective ways to increase economic 

opportunity, boost youth development and promote healthy futures. 

Table 2: SIF Programs by Focus Area and Year 

Number of SIF Interventions per Issue Area, by Cohort Year

2010 2011 2012 Total (% of Total)

Healthy Futures 11 2 1 14 (16%)

Economic Opportunity 9 1 0 10 (12%)

Youth Development 29 21 13 63 (72%)

Total 49 24 14 87 Interventions

(56%) (28%) (16%) (100%)

Source: Corporation for National & Community Service

Note: Nine intermediaries required all subgrantees to implement the same program. These intermediaries submitted one subgrantee evaluation plan (SEP). 

 
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 6

Evaluating the Evidence – How the Process Works 

CNCS supports subgrantees through the evaluation process to ensure their success. The first step is preparing and 

submitting a SIF Evaluation Plan (SEP) to CNCS. Intermediaries receive a detailed SEP guidance document that helps them 

navigate the requirements. They also partner with an independent evaluator to build evidence supporting the program’s 

effectiveness and, ultimately, improve performance. To bolster their evidence, subgrantees have the option of providing 

additional information about prior evaluations and research. Because programs represent a variety of approaches, sizes 

and stages, there is no one-size-fits-all evaluation for SIF-funded programs. 











 

Five Types of Program Evaluation 

SIF-funded programs take different approaches to evaluating their impact, depending on their size, approach, stage and other 

variables. To find the right fit, CNCS works with its intermediaries and subgrantees to develop portfolio evaluation strategies and 

individual program evaluation plans. Independent third-party evaluators conduct evaluations. Almost all of them document program 

participation, engagement, quality and satisfaction. To accommodate such diversity in building an evidence base, research questions 

and evaluation strategies vary, generating rich data that help to improve existing programs and to inform future decisions. 

The five types of evaluations include: 











Causal impact: Document program outcomes and impact and the link to results (whether participants benefitted from the 

program). The process is designed to help programs achieve moderate or strong effectiveness by the end of their funding cycle. 

Cost: Explore program costs, cost per participant, return on investment and related issues. 

Commitment to SIF model: Determine how adhering to the SIF model correlates with effectiveness, including implementation 

and the implications when a program with strong evidence of effectiveness is adapted to a new setting or delivered to a different 

target population. 

Feasibility: Examine the implementation process in a new situation or for a new target population; assess the feasibility of 

conducting an experimental or quasi-experimental design study. 

Indicators: Assess the outcome when a program is expanded or revised for a larger population of users. 

After submitting the evaluation plan, a review team assesses the evidence, guided by criteria outlined in the 2012 funding 

requirements: 

Review the subgrantees self-assessment of its evidence and prior studies. 

Determine the methodology of prior studies as well as the number of program sites and populations. 

Search for studies on the program not included in the plan or similar programs to review the methodology, number 

of sites, and populations. 

Document whether other programs (non-intermediaries and subgrantees) have performed studies of the program 

model, using a single site QED study, a single site RCT, a multi-site RCT, a PSM or time series design, or multiple 

RCTs. 

Determine if evidence from non-subgrantee evaluations shows that the program worked. 

The review team also explores how the program model was adapted--directly, with modifications, or loosely based on prior 

studies -- and key results from other program implementers. Another consideration: Did the subgrantee operate the 

program model prior to SIF funding, and if so, did it conduct an outcome study of the program model, and what type?6

6

 Outcome studies include a single site QED study (matched or short time series), a single site QED study (multivariate matched, PSM, or time series), a single site RCT, a multisite RCT, 

PSM, a time series design, or multiple RCTs 
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 7

Organizing Results into an Evidence Framework 

The SIF organizes evaluation results into a framework with three levels of evidence: Preliminary, moderate and strong. 

(For more detail, see “A Three-Tiered Framework for Evidence.”)







Preliminary evidence indicates a reasonable hypothesis with credible research that shows promising results.

Moderate evidence demonstrates that the program achieves the desired outcome for a limited population.

Strong evidence documents the success of the program, including for larger populations.

 

A Three-Tiered Evidence Framework 

The SIF organizes evidence into three categories. The framework is very similar to those used by other federal evidence-based 

programs. 

Preliminary evidence: To receive funding, a program must meet the preliminary evidence criteria. This means it has evidence 

based on a reasonable hypothesis and is supported by credible research findings. Examples of research that meet the standards 

include: outcome studies that track participants through a program and measure participants’ response at the end of the program; 

and, third-party pre- and post-test research that determines whether participants have improved on an intended outcome. 

Moderate evidence levels demonstrate if the program achieves the desired outcome. Results only apply to a limited population. 

Previous studies on the program used designs that support causal conclusions (studies with high internal validity) but have moderate 

external validation or vice versa—studies that only support moderate causal conclusions but have broad general applicability. 

Programs may use randomized controlled trials or quasi-experimental designs when random sampling is not possible. Examples 

include: at least one well-designed and well-implemented experimental or quasi experimental study supporting the effectiveness of 

the practice study or program, with small sample sizes or other conditions of implementation or analysis that limit general application; 

or correlational research with strong statistical controls for selection bias and for discerning the influence of internal factors. 

Strong evidence establishes a cause-and-effect from the program and can rule out other explanations for the outcomes. This is 

accomplished by conducting multiple (or multi-site) randomized controlled trials or well-designed and implemented quasiexperimental design approaches. Evidence from previous studies on the program use designs that can support causal conclusions 

( i.e., studies with high internal visibility) and that, in total, include enough of a range in participants and settings to support scaling 

up to the state, regional, or national level (i.e., studies with high external validity). The key distinction between “strong” and 

“moderate” is that “strong” levels of evidence have both internal and external validity and the results can be applied to general 

populations. Examples of strong evidence include: one well-implemented experimental study or well-designed and well-implemented 

quasi-experimental study that supports the effectiveness of the practice, strategy, or program; or one large, well-designed and 

implemented randomized controlled, multisite trial that supports the effectiveness of the practice, strategy or program. 

The three-tiered evidence model provides a roadmap for grantees, showing them how they can improve. If successful, three-quarters 

of the evaluations will allow programs to demonstrate a moderate or strong level of evidence.

To date, CNCS has received a total of 108 subgrantee evaluation plans covering 87 programs,7 with 77 plans winning 

approval (see Figure 2 and 3). The findings provide a snapshot of programs at the outset of SIF funding and serve as a 

baseline. 

Almost three-fourths of the programs are in the preliminary level, the early stage of evidence building. Only eight 

programs-- 9%-- earned a “strong” evidence level designation. The evaluation results set the stage for improvements and 

will guide future decisions, such as approval for additional grants to “strong” programs, enabling substantial growth of 

their services to communities in need. 

7

 Due to the dynamic nature of these evaluations, SEPs frequently change and must be re-evaluated. As a result, the number of SEPs remains in flux.
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 8

Figure 2: Entry Level of Evidence by Subgrantee 

•

•

•

62 Interventions 

(71%) 

Preliminary 

Non-Experimental Design

17 Interventions 

(20%) 

Moderate 

Quasi-Experimental Design or Small/Single 

Site Randomized 

Controlled Trials 

8 Interventions 

(9%) 

Strong 

Multiple or Multi-site Well-Designed 

Randomized Control Trial Designs 

Note: An intermediary using one evaluation to cover multiple subgrantees is counted as one. Eight intermediaries are using this approach. 

Designing Evaluation Studies to Advance the Evidence Base 

Subgrantees use a variety of study designs to gauge a program’s effectiveness and impact as well as building on prior 

evidence. More than one-third – 36 percent – are conducting randomized controlled trials while almost half – 47 percent – 

selected a quasi-experimental design. All studies explore the implementation process, critical for program success and 

replication. 

Questions probe an array of factors: Which populations are served, the extent of outreach and engagement in services, 

dosage, and how services are delivered and staffed. They also look at broader organizational factors such as what enables 

or impedes implementation, effectiveness of performance management systems, and how closely the program adhered to 

the implementation plan. A number of studies address cost-related questions. Understanding implementation challenges 

and cost factors is essential for successfully replicating the program. 

Evaluations also offer specifics about how programs are addressing the SIF’s three targeted areas: 

Youth development – school readiness, school attendance, academic performance and graduation rates 

Economic opportunity – employment and job placements rates, training credentials, income earnings, hours 

worked and benefits, pursuit of a college degree 

Healthy futures – access to health care, reductions in risky behaviors 
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 9

Figure 3: Evaluation Study Designs 

Building a Storehouse of Knowledge 

The SIF’s pioneering funding model provides a scarce resource for those delivering social programs: A scientific approach 

to measuring results in a field with few agreed-upon measures. While evidence gathering still is in the early stages, the SIF 

model already is building a valuable store of knowledge. By comparing evidence-based results of SIF-funded programs 

with previous studies, as well as applicable control groups, the findings offer a robust assessment of which grassroots 

programs are most effectively delivering change. 

The evaluations serve as a powerful knowledge-building tool for intermediaries and their subgrantees in search of ways to 

continually improve. Over the course of the funding cycle, evaluations will allow them to discover what works and what 

does not by assessing core elements: the implementation process, adherence to program models, feasibility of replicating 

models at different sites, program outcomes and impact. 

With the launch of the Knowledge Initiative and National Assessment, the SIF fosters its potential to promote new 

grantmaking practices. Comprehensive evaluations over the next two to five years will identify the strongest program 

models for better serving people and generating more impact per dollar. As the SIF evidence base grows richer, we will 

continue sharing insights and lessons learned with social policymakers and program creators. 

Gauging the SIF’s impact involves answering five mission-critical questions: 

•

•

•

•

•

What programs contribute to improving results in the three focus areas?

What novel program elements identified in evaluations help advance knowledge and practices in targeted 

areas?

How do evidence-based initiatives like the SIF model inform evaluation practices in the public and private 

sectors?

How do they advance public and private grantmaking strategies?

What are the implications of collaborative public-private funding partnerships for community problemsolving? And what is the collective impact of these efforts?

The findings will help answer the overriding question: Is the SIF’s breakthrough investment model accomplishing its bold 

goal: Expanding community-driven solutions and changing the game to generate more powerful results for Americans. 
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 10

Social Innovation Fund: Glossary of Key Terms 

SIF: 

The Social Innovation Fund (SIF), a key White House initiative and program of the 

Corporation for National and Community Service, combines public and private resources to 

grow the impact of innovative, community-based solutions that have compelling evidence of 

improving the lives of people in low-income communities throughout the United States.

CNCS: 

Corporation for National and Community Service (CNCS) is an independent federal agency 

with the mission to improve lives, strengthen communities, and foster civic engagement 

through service and volunteering. The agency administers the SIF. 

Focus Areas: Programmatic areas that address similar issues or challenges. The three focus areas that SIF 

supports are youth development; healthy futures; economic opportunity. 

Evidence-based 

Initiatives: 

In this context, a federal initiative that seeks to promote programs rooted in science and 

research. There currently are six federally funded evidence-based initiatives, of which SIF is 

one. 

Intermediary: 

A non-profit, grant-making organization that can apply for SIF funds. If chosen through 

CNCS’ grant-making processes, this organization will select a number of subgrantees to 

participate in its SIF portfolio and to which it will disburse SIF funds. An intermediary is 

required to match SIF funds one-to-one. 

Subgrantee: 

A non-profit organization that implements a program aimed at addressing a social or 

community challenge. These organizations are chosen to receive SIF funds through an 

intermediary organization, and join other such organizations as a part of the intermediary’s 

SIF portfolio. Subgrantees are required to conduct a program evaluation that is rigorous and 

builds upon the existing body of evidence for the program’s intervention. Subgrantees are 

required to match intermediary funds one-to-one. 

SIF Cohort: 

A group of intermediaries chosen in a particular funding year. Currently, there are three SIF 

cohorts, one in each of the following years: 2010, 2011, and 2012. The term also encompasses 

each intermediary’s subgrantees: 2010 cohort of 11 intermediaries, 154 subgrantees; 2011 

cohort of 5 intermediaries 48 subgrantees, and; 2012 cohort of four intermediaries, that have 

selected a number of subgrantees but are currently still in process to select more. 

Scaling Up: A term that means increasing the size and reach of a subgrantees intervention or program. 

Matching Funds:

Federal requirements of dollar match by a non-federal entity in order to be eligible to receive 

grant funding. 

Intervention:

A program’s activity or model that addresses a social or community challenge. The 

intervention is what is evaluated in the SIF. 

Body of Evidence: A collection of science-based studies or research that support the effectiveness of a subgrantees 

program or intervention. 

Level of Evidence: 

A particular location along a continuum of programmatic evidence, ranging from anecdotal 

information (participant stories) to rigorous causal studies. For the purposes of SIF, the 

continuum is broken up into three distinct segments: preliminary, moderate, and strong. A 

subgrantee is assessed after being selected by an intermediary, or upon “entry” to SIF, and 

again after completing the grant cycle, or upon “exit.” 
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 11

Subgrantee Evaluation 

Plan (SEP): This term signifies the evaluation plan developed by subgrantees.

Unified Subgrantee 

Evaluation Plan 

(UniSEP): 

A single subgrantee evaluation plan that an intermediary applies to the evaluation of multiple 

subgrantees’ programs in cases where multiple subgrantees are implementing an intervention 

across multiple sites. 

Entry Level of 

Evidence: 

A designation of either preliminary, moderate, or strong based on CNCS’ assessment of the 

initial body of evidence behind a program’s intervention. 

Target Level of 

Evidence: 

A designation of either preliminary, moderate, or strong based on CNCS’ assessment of the 

outcomes of the executed evaluation that was designated by the SEP upon exit from the grant 

program. 

Preliminary Level of 

Evidence: 

Interventions with all other types of outcome studies (e.g., pre-post test studies, studies 

monitoring outcomes throughout an intervention) were designated as “preliminary.” 

Interventions that were based on reasonable hypotheses supported by research findings (e.g., 

a body of literature that supports the use of the general type of intervention, but not the specific 

program as conducted by the grantee/subgrantee) were also designated as having preliminary 

evidence. 

Moderate Level of 

Evidence: 

Interventions were designated as having “moderate” evidence if they had at least one welldesigned and well-implemented experimental or quasi-experimental study, or multiple 

examples of correlational research with statistical controls supporting the effectiveness of the 

program. 

Strong Level of 

Evidence: 

Interventions were designated as having “strong” evidence if they had “(1) more than one 

well-designed and well-implemented experimental study or well-designed and wellimplemented quasi-experimental study that supports the effectiveness of the practice, 

strategy, or program; or (2) one large, well-designed and well-implemented randomized 

controlled multisite trial that supports the effectiveness of the practice, strategy, or program.” 

Internal Validity: The extent to which a study can support causal conclusions by reducing systematic error or 

biases. 

External Validity: The extent to which a study’s results can be generalized to locations, contexts or populations 

beyond those actually in the study itself. 

Experimental Design: 

Experimental design studies using random control trials or RCTs randomly assign program 

participants to two distinct groups: the treatment group, which receives program services, and 

the control group, which does not. The control group is called the “counterfactual,” 

representing the condition in which the program or intervention is absent. Random 

assignment ensures that the treatment and control groups are initially similar and do not differ 

on background characteristics or other factors. Thus, random assignment creates an evaluation 

design where any observed differences between the two groups after the program intervention 

takes place can be attributed to the intervention with a high degree of confidence. 

Random Assignment: 

A process that uses randomly generated numbers or other approaches to assign study units to 

groups in ways that are unaffected by the characteristics of the study units. With random 

assignment, any differences between the groups at pre-test can be attributed only to chance. 

The use, or lack of use, of this process differentiates experimental designs from nonexperimental designs. 
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 12

Quasi-Experimental 

Design: 

A design that forms a counterfactual group by means other than random assignment. This 

approach is used for conducting impact evaluations where observed changes in the treatment 

group are compared with a comparison group (as counterfactual representing an absence of 

intervention) to assess and estimate the impact of the program on participants. However, 

groups formed in these designs typically differ for reasons other than chance, and these 

differences may influence the impact estimate. There are different types of approaches used 

in quasi-experimental designs such as those using Propensity Score Matching (PSM), 

Regression Discontinuity, Interrupted Time Series (ITS) and others. 

Propensity Score

Matching

A statistical matching approach that is sometimes employed in quasi-experimental design 

studies for the purposes of developing a comparison group. This approach is based on a 

predicted probability of group membership (e.g., intervention vs. control) using measured 

characteristics of study units as predictors. The predicted probabilities are typically obtained 

from logistic regression. 

: 

Regression 

Discontinuity Design: 

This is a specific quasi-experimental design approach that is used for evaluating causal effects 

of interventions. Under this approach assignment to a treatment is determined at least partly 

by the value of an observed covariate lying on either side of a fixed threshold. The intervention 

and control group are formed using a well-defined cutoff score. The group below the cutoff 

score receives the intervention and the group above does not, or vice versa. For example, if 

students are selected for a program based on test scores, those just above the score and just 

below the score are expected to be very similar except for participation in the program, and 

can be compared with each other to determine the program’s impact. 

Interrupted Time 

Series: 

This is a specific quasi-experimental design approach that is used for evaluating causal effects 

of interventions. Under this approach multiple observations are obtained prior to the 

intervention to establish a baseline. Multiple observations are also obtained after the 

intervention. Effects are demonstrated when the observations after the intervention deviate 

from expectations derived from baseline projections. 

Non-Experimental 

Design: 

The term is a catch-all category that refers to a range of research and evaluation studies that 

do not fall under the experimental or quasi-experimental research designs. They include 

process and outcomes evaluations, spanning pre-post test designs without comparison 

groups, case studies, cost effectiveness, cost benefit analysis or other cost studies, feasibility 

studies, rapid assessments, situational and contribution analysis, developmental evaluation, 

strategic learning, systems change studies, and others. 
The Social Innovation Fund: Corporation for National & Community Service 

Pioneering an Evidence-Based Investment Model Office of Research & Evaluation 

nationalservice.gov/SIF 13

About the Social Innovation Fund (SIF) 

The SIF is a federal program intended to foster innovation to transform lives and communities. A program of the 

Corporation for National and Community Service (CNCS) launched in 2010, it is one of six Obama Administration 

“tiered-evidence initiatives” embodying the principles of social innovation. As a program, it leverages federal funds 

through public-private collaborations by granting money to highly successful intermediary grant-makers who in turn 

find, improve, and grow promising community solutions with evidence of successful outcomes in youth development, 

economic opportunity, and health. 

The SIF is characterized by the unique interplay of six key elements: 

1. It relies on intermediary grant-making institutions to implement the program – they take on the role of finding, 

selecting, monitoring, supporting, evaluating and reporting on the nonprofit programs implementing 

community-based interventions. 

2. It is a tier-based evidence program that requires all funded programs/interventions to demonstrate at least 

preliminary evidence of effectiveness, or funding “what works.” 

3. The SIF requires that all programs or interventions implement a rigorous evaluation that will build on their level 

of evidence 

4. SIF intermediaries are charged with scaling evidence-based programs – increasing their impact within their 

community or to communities across the country – and as such, grapple with a field-wide challenge of how best 

to successfully and efficiently do so. 

5. The SIF leverages public-private partnerships to effect large-scale community impact in ways that either a 

traditional federal grant investment or a philanthropic grant investment could not achieve on its own. This 

includes its unique leveraged funding model to support nonprofit programs. 

6. The SIF is committed to improving the effectiveness of nonprofits, funders, and other federal agencies by 

capturing learning and best practices and promoting approaches that will generate the greatest impact for 

individuals and communities. 
Corporation for National and Community Service 

1201 New York Avenue, NW 

Washington, DC 20525 

TEL: (202) 606-5000 

TTY: (202) 606-3472 

NationalService.gov