National Service Systematic Review 

and Synthesis of National Service 

Literature 

Submitted: August 19, 2015 

Submitted to: 

Adrienne DiTommaso 

Corporation for National and Community Service 

1201 New York Avenue, NW 

Washington, DC 20525 

Submitted by: 

JBS International, Inc. 

Aguirre Division 

555 Airport Blvd, Suite 400 

Burlingame, CA 94010-2002 
Table of Contents 

Executive Summary ................................................................................................................................... 1

Introduction to the National Service Literature Review ...................................................................... 3

Research Questions ............................................................................................................................... 3

Methods and Procedures for Completing the Literature Review ................................................... 3

Evolution of the Evidence Base for National Service ....................................................................... 4

Organization of the Systematic Review and Synthesis Report ....................................................... 6

Overview of Documents Reviewed .................................................................................................... 6

Focus Areas Targeting Beneficiary Outcomes ..................................................................................... 10

Education .............................................................................................................................................. 10

Economic Opportunity ....................................................................................................................... 16

Environmental Stewardship .............................................................................................................. 19

Disaster Services .................................................................................................................................. 21

Healthy Futures ................................................................................................................................... 23

Impact on National Service Participants (“Member Development”) ............................................... 28

Skills Relevant to Professional Development .................................................................................. 28

Civic Engagement ................................................................................................................................ 33

Impact on Organizations and Communities (Organizational Capacity and “Strengthening 

Communities”) ......................................................................................................................................... 38

Analysis ................................................................................................................................................. 39

Conclusions ............................................................................................................................................... 40

Strength of Evidence Base .................................................................................................................. 40

Learnings, Best Practices, and Key Resources ................................................................................. 40

Areas for Future Research .................................................................................................................. 42

Appendix A: Search and Review Processes ......................................................................................... 45

Systematic Search Process .................................................................................................................. 45

Systematic Review and Data Extraction ........................................................................................... 45

Appendix B: Annotated Bibliographies ................................................................................................ 47

Education .............................................................................................................................................. 47

Economic Opportunity ....................................................................................................................... 65
Environmental Stewardship ..............................................................................................................68

Disaster Services ..................................................................................................................................70

Member Development ........................................................................................................................81

Organizational Capacity.....................................................................................................................94
National Service 

Synthesis Report

August 19, 2015

1

Executive Summary

This report provides an overview of the full body of empirical evidence1 on national service 

identified through a systematic search and review process. JBS worked collaboratively with the 

Corporation for National and Community Service (CNCS) to develop the search and review 

protocols that formed the basis for the body of literature discussed in this report (see Appendix 

A). The sources identified and reviewed for this report all have in common a focus on national 

service programs and interventions intended to effect positive change in five strategic focus 

areas (education, economic opportunity, environmental stewardship, disaster services, and 

healthy futures2), in two impact areas for member development (skills relevant to professional 

development and civic engagement), or in an impact area addressing organizational capacity 

and sustainability.

Each of the five focus areas and the three impact areas comprises a section in the main body of 

the report. Within each section, the report identifies and briefly describes each of the sources 

that were examined in detail as offering the best sources of evidence for that focus or impact 

area. Each section also answers the following three research questions, as they pertain 

specifically to the focus areas:

1. What do the outcome/impact evaluations on national service programs generally say 

about the effectiveness of those interventions?

2. What is the strength of the evidence base for national service?

3. What are the learnings, best practices, and key resources that can inform the work of 

CNCS, its grantees, and other stakeholder groups?

 

1 Empirical evidence serves to either support or counter a theory or hypothesis. According to the 

AmeriCorps State and National 2015 Notice of Funding Opportunity (NOFO), the rigor of evidence is 

defined as strong, moderate, preliminary, or pre-preliminary. Strong evidence supports causal 

conclusions for the program with the highest level of confidence (i.e., very high internal validity). 

Moderate evidence supports causal conclusions for the program with moderate confidence (i.e., steps are 

taken to increase internal validity). Preliminary evidence can demonstrate improvement in program 

participants over time on one or more intended outcomes or an implementation (process) evaluation used 

to improve program operations. Pre-preliminary evidence includes quantitative or qualitative data from 

program staff, program participants, or beneficiaries that have been used for program improvement, 

performance measurement reporting, and/or tracking. An example of pre-preliminary evidence would be 

gathering feedback from program participants following their service year.

2 The veterans and military families focus area were excluded from the project because there is relatively 

little literature on it.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

2

The following fourth research question pertains to all the focus areas and is addressed once in 

the concluding section of the report:

4. What does the academic literature and research say about national service in general 

and particular interventions found within national service?

Key findings of this report include the following:

 Substantial variation exists in the strength and depth of evidence across the five focus 

and three impact areas.

 In education, research shows that highly structured, well-implemented, intensive, oneon-one tutoring models are effective in promoting academic achievement, especially at 

the early grades and for children from disadvantaged backgrounds.

 Economic opportunity and healthy futures represent two additional areas with a 

sufficient evidence base to allow tentative observations about the efficacy of selected 

program models.

 Environmental stewardship and disaster services both have very limited evidence, 

leaving considerable room to expand the evidence base.

 The two domains of member development reviewed contain extensive literature that 

includes studies employing well-implemented causal designs. In the area of member 

skills relevant to professional development, national service participation brings 

substantial benefits to members through the soft- and hard-skills acquired during 

service, the expansion of social networks, and the clarification of personal goals. In the 

area of civic engagement, national service programs continue to successfully promote 

civic engagement as an ongoing part of alumni’s self-identities.

 In the area of organizational capacity and sustainability, the VISTA program has 

demonstrated its ability to increase the organizational capacity of host organizations, 

particularly with regard to strengthening volunteer recruitment.

An annotated bibliography for each section, including a description of each source, is provided 

in Appendix B.

Figure 2 in the introduction of this report provides an overview of the documents reviewed for 

possible inclusion in the systematic review and synthesis report. Table 1 in the introduction 

summarizes our examination of the literature found.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

3

Introduction to the National Service Literature Review

This report presents a synthesis of systematically collected literature on national service. The 

report was developed based on an extensive and systematic literature review of the overall 

evidence base for national service, including research and evaluation reports written between 

1990 and early 2015 relating to national service programs. The purpose of this report is to 

describe the overall evidence base for national service as well as identify the specific strategies 

and interventions supported by the strongest evidence.

The introduction to this report identifies the research questions, the methods used to select the 

literature sources, the breadth of national service literature and the overall level of evidence.

Research Questions

This report addresses the following research questions for each focus area:

1. What do the outcome/impact evaluations on national service programs generally say 

about the effectiveness of those interventions?

2. What is the strength of the evidence base for national service?

3. What are the learnings, best practices, and key resources that can inform the work of 

CNCS, its grantees, and other stakeholder groups?

The fourth research question pertains to all the focus areas and is addressed once in the 

concluding section of the report.

4. What does the academic literature and research say about national service in general 

and particular interventions found within national service?

Methods and Procedures for Completing the Literature Review

JBS developed search and review methods and procedures that supported an extensive review 

of the literature related to evidence, practices, strategies and programs that comprise national 

service. These processes, while consistent with evidence standards and guidelines used by other 

federal agencies, were intended primarily to maximize knowledge building and utility to the 

national service field and are balanced by consideration of the state of the science and practice 

within national service. The development of the search and review protocols for the CNCS 

National Service systematic review was informed by:

 Documented search protocols used in other systematic literature reviews, systematic 

literature review repositories, and evidence-focused clearinghouses;

 Inclusion criteria and review processes for existing systematic literature review 

repositories and clearinghouses; and

 Interviews with key informants responsible for the development of a major evidence 

based practice clearinghouse and a systematic literature review repository.

See Appendix A for further information on the search and review process.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

4

In each of the following eight sections of this report, we examine the available literature, 

highlighting a selected group of studies of particular significance to that area. Items for in-depth 

review were chosen based on relevance, recency, and strength of study design. After briefly 

describing the studies and noting their contribution to the evidence base in a particular focus 

area or impact area, we provide an analysis aligned with the research questions. We also 

characterize the overall level of evidence for each focus or impact area based on selecting the 

highest level at which there were multiple studies supporting positive outcomes.

Evolution of the Evidence Base for National Service

Since the 1990’s, as CNCS has matured as an agency, there has been an evolution from requiring 

grantees to report administrative and performance measurement data to expecting greater 

accountability at the level of outcomes and impacts.3 This has created a greater emphasis on 

clarifying and measuring intended program outcomes and increasing methodological rigor to 

allow causal attribution of outcomes to program interventions. The underlying theme of this 

gradual, progressive change, which has also been reflected in the broader public and non-profit 

sector, has been greater program accountability at the level of outcomes and impacts.

This change is clearly reflected in the results of the search and review of the national service

literature, as the number of reports, particularly addressing impacts, has grown. Figure 1, which 

draws upon the complete body of literature collected and found eligible for review for this

project4, shows that the proportion of impact studies increased from a low of about seven

percent in the early and mid-1990s to 18 and 25 percent in the five-year periods between 2006-

2010 and 2011-2014, respectively. Especially in the last decade, there has been steady 

improvement in the quantity and quality of available evidence to support the effectiveness of 

national service programs. Over the same period, the proportion of outcome studies and other 

studies has fluctuated around the 40 percent mark with no clearly discernable trend, although 

the decline in outcome studies from approximately 2001 onward presumably reflects programs

conducting impact studies that might have otherwise conducted outcome studies.

 

3

Impact evaluation uses study designs that can determine if the outcomes observed among program 

participants are due to having participated in the intervention. Outcome evaluation can determine if 

program participants experienced intended outcomes (changes in attitudes, knowledge, behavior, or 

conditions), but uses study designs that do not support attributing changes to the intervention. Other 

studies refers to a variety study types, including implementation studies, evaluation feasibility studies, 

program descriptions, and rigorous qualitative studies.

4 This figure includes reports from the Social Innovation Fund, which are not discussed in this report. It 

excludes web based documents from clearinghouses which are synthesis of studies, or which have no 

firm publication date.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

5

Figure 1: Study Type over Time

The increase in proportion of impact studies can likely be attributed, at least in part, to the 

convergence of two factors: (1) an increase in the evaluation capacity of national service 

grantees, particularly large AmeriCorps State and National (ASN) grantees, as they have 

matured programmatically; and (2) a growing movement within CNCS itself towards greater 

promotion of and support for higher quality evaluation. At the policy level, this is also reflected 

in the 2010 Rulemaking for AmeriCorps, introduced with the Edward M. Kennedy Serve

America Act, and specifically in the requirement that all AmeriCorps State and National 

grantees that receive an average annual Corporation grant of $500,000 or more must conduct an 

independent evaluation.5

The movement toward more systematic and rigorous evaluations has not been limited to the 

ASN program, as witnessed, for example, by the ambitious research agenda being undertaken 

by the Senior Corps program. CNCS is also investing in infrastructure to spread and support 

the use of evidence. This is currently being accomplished through a study repository, the 

Evidence Exchange, and an Evaluation Resources webpage of training and technical assistance 

materials6. These efforts point to a growing commitment within national service to mainstream 

evaluation for all programs and grantees. The following literature review points to abundant 

opportunities for research and evaluation to demonstrate the effectiveness of programs and 

interventions happening within national service and increase the knowledge base regarding 

what works. In addition, the Social Innovation Fund, which began providing reports in 2013, 

 

5 As articulated in Sections 2522.700-2522.740 of the AmeriCorps regulations, see 

http://www.nationalservice.gov/pdf/06_1128_ac_sn_evaluation_faqs.pdf

6 http://www.nationalservice.gov/impact-our-nation/research-and-reports/evidence-exchange

0%

10%

20%

30%

40%

50%

60%

1990-1994 1995-2000 2001-2005 2006-2010 2011-2014

Proportion of Studies Eligible for Review 

(N=640)

Study Publication Year

Impact

Outcome

Other
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

6

and which funds some subgrantees that are also supporting ASN and VISTA members, 

emphasizes that all funded interventions should engage in rigorous impact evaluations.

Organization of the Systematic Review and Synthesis Report

This report is organized into sections addressing eight major focus areas or “impact areas.”

Focus area refers to the CNCS focus areas of education, economic opportunity, environmental 

stewardship, disaster services, and healthy futures.

7 These focus areas figure prominently in the 

CNCS 2011-2015 Strategic Plan, which outlines strategies for addressing the Serve America Act.

Impact areas refer to additional areas of interest within national service: member development 

with a focus on skills relevant to professional development and/or civic engagement, and 

organizational capacity and sustainability. Each of these impact areas has been a topic of 

longstanding programmatic interest and has received considerable attention from researchers. 

Member development has been a core activity of all CNCS programs. Civic engagement, in 

particular, is a cornerstone goal of national service programs and one of the most important 

reasons for the existence of national service itself. Organizational capacity and sustainability are 

explicit goals of the AmeriCorps VISTA (Volunteers in Service to America) program, and, to a 

lesser extent, the National Civilian Community Corps (NCCC).

Overview of Documents Reviewed

Figure 2 provides an overview of the documents reviewed for possible inclusion in the 

systematic review and synthesis report, with a breakdown by broad study type (impact, 

outcome, other) and by study design type within the impact category (randomized control trial, 

groups formed by matching, groups formed by cutoff score, non-equivalent comparison 

groups, and other).

 

7 The veterans and military families focus area was excluded from the project because there is relatively 

little literature on it.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

7

Figure 2: Literature Summary Flow Chart

Table 1 summarizes the 639 national service focused documents identified for review. Ninetyfive (15 percent) address impacts, 249 (39 percent) address outcomes, and 295 (46 percent) 

address other types of information.8

 

8 Other refers to a variety of sources that are neither impact nor outcome reports. This may include 

implementation studies, evaluation feasibility studies, program descriptions, and rigorous qualitative 

studies, such as ethnographies.

All Reviewed 

Documents

N = 2508

National Service

N = 639 

Impact

N = 95

Randomized 

Controlled Trial 

N = 19 

Groups formed by 

Matching

N = 18

Groups formed by 

Cutoff Score

N = 2

Interrupted Time 

Series

N = 3 

Comparison groups 

– Non-equivalent

N = 53

Outcome

N = 249

Other

N = 295 

Not eligible or not 

National Service

N = 1869 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

8

Table 1: National Service Literature Located by Study Type9

Impact Outcome Other Total

Selected Eligible Selected Eligible Selected Eligible Selected Eligible

Education 15 65 1 135 0 150 16 350

Economic 

Opportunity 3 10 0 54 0 62 3 126

Environmental 

Stewardship 1 1 1 7 0 6 2 14

Disaster

Services 0 0 0 4 1 5 1 9

Healthy 

Futures 2 8 6 25 0 22 8 55

Member 

Development 

– Professional 

Skills

6 26 2 73 0 71 8 170

Member 

Development 

– Civic 

Engagement

7 26 2 113 0 109 9 248

Organizational 

Capacity and 

Sustainability

2 8 1 48 0 36 3 92

Total 36 95 13 249 1 295 50 639

There is great variation in the volume, depth, and strength of literature across the focus and 

impact areas. Interventions in education are particularly well researched. These include literacy 

tutoring programs and subject-specific interventions aimed at improving student academic 

performance. Economic opportunity and healthy futures have a sufficient evidence base to 

allow tentative observations about the efficacy of selected program models. The two domains of 

member development (civic engagement and skills relevant to professional development) 

contain extensive literature that includes some studies using causal designs. In the focus areas 

of environmental stewardship and disaster services, there are very few studies and those 

studies that exist are generally not rigorous. The body of evidence for organizational capacity 

and sustainability, while somewhat limited, contains a few important studies that address 

impact.

 

9 Documents may be classified as eligible in multiple topic areas (e.g., Education and Member 

Development – Civic Engagement). However, total numbers of eligible documents reported in the bottom 

row only count each document once; therefore, numbers in eligible columns will exceed reported totals. 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

9

Table 2 shows the number of reports examined in detail for the national service systematic 

review and synthesis by classification as impact, outcome, or other (implementation).

Table 2: Overview of Articles for National Service Systematic Review and Synthesis, by 

Study Methodology and Focus Area

Focus or Impact Area Impact Outcome Other

Education (16) 15 1 0

Economic Opportunity (3) 3 0 0

Environmental Stewardship (2) 1 1 0

Disaster Services (1) 0 0 1

Healthy Futures (8) 2 6 0

Member Development: Skills Relevant to Professional Development 

(8) 6 2 0

Member Development: Civic Attitudes and Behavior (10) 7 2 0

Organizational Capacity and Sustainability (3) 2 1 0

Total 36 13 1
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

10

Focus Areas Targeting Beneficiary Outcomes

This literature review covers five of the focus areas traditionally associated with national service 

participants’ direct service activities: education, economic opportunity, environmental 

stewardship, disaster services, and healthy futures.

The analysis of the literature addressing impact on beneficiaries reveals notable variation in the 

depth and quality of sources available across focus areas. At the extremes, the education focus 

area boasts the strongest and most extensive body of evidence overall, while disaster services 

and environmental stewardship have very limited bodies of literature that meet either a 

preliminary or pre-preliminary level of evidence. Economic opportunity and healthy futures are 

situated in the middle of the spectrum with meaningful evidence typically ranging from 

preliminary to moderate in strength, but with relatively little breadth of coverage of key subtopics within these areas. There is also variation within each focus area in terms of the types of 

interventions that have received greater or lesser attention from the research community. The 

focus areas of economic opportunity and healthy futures are particularly noteworthy for the 

variety of interventions available and populations served.

Overall, there is only one focus area, education, that has a great depth of literature addressing

program efficacy in a rigorous way. Elsewhere in the world of national service, there remains a 

wide domain for further research, especially utilizing causal designs. Many focus areas and 

associated interventions present particular challenges to evaluability. For example, in the field 

of disaster services, disasters tend to be unpredictable episodic events that run counter to the 

notion of planned and structured evaluation activities. In the focus area of environmental 

stewardship, the potential evaluand is often a natural setting that may be geographically 

dispersed and/or lack clearly delineated boundaries (e.g., a watershed or forest); non-human 

evaluands (e.g., plants and animals) are not amenable to conventional social science data 

collection methods, such as surveys and interviews; and other natural forces that are beyond 

human control (e.g., drought, flood, fire) may also contaminate results. In the focus area of 

healthy futures, randomly assigning individuals to treatment and control groups may be an

infrequent option and/or ethically problematic.

In situations like these, evaluation may be largely limited to making narrow assessments of 

participants’ knowledge, skills, or attitudes (e.g., whether individuals attending emergency 

response training demonstrate newly acquired skills; whether students exposed to 

environmental curricula gain knowledge about aspects of the environment). In reviewing the 

evidence for the focus areas, this review considers whether there are additional opportunities to 

strengthening the existing evidence base.

Education

There were 350 documents that included a focus on education. Sixty-five of these reports are 

impact evaluations, 135 are outcome evaluations, and 150 are classified as other, and typically 

describe pre- or non-experimental approaches.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

11

Sixteen documents with a primary focus on education were selected for full review; 15 impact 

evaluations (nine randomized control trials [RCT], six quasi-experimental designs [QED]) and 

one outcome study. Table 3 shows the education sources reviewed in detail for this report.

Table 3: Education

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Effects of Teach For 

America on Students: 

Findings from a National 

Evaluation (1682)

Other RCT Strong Math achievement , 

Reading achievement 

Impacts of Teach For 

America Investing in 

Innovation Scale-Up 

(2269)

Internal, nongovernment 

report

RCT Strong Reading skills (Pre-K to 

2

nd grade) , Reading 

skills overall, Math skills 

Effectiveness of Secondary 

Math Teachers from Teach 

For America and the 

Teaching Fellows 

Programs. NCEE 2013-

4015 (2270)

Government 

report (e.g., 

grey literature)

RCT Strong Math test scores 

CIS Communities in 

Schools Five Year 

National Evaluation 

Summary Report (2489)

Government 

report (e.g., 

grey literature)

RCT Strong Academic performance 

, Attendance 

Mobilizing Volunteer 

Tutors to Improve Student 

Literacy (2365)

Government 

report (e.g., 

grey literature)

RCT Strong Reading achievement 

Effect of Teach for 

America on distribution of 

student achievement in 

primary school (170)

Peer-reviewed 

journal article

RCT Moderate Math achievement , 

Reading achievement

Evaluation of Experience 

Corps Student Reading 

Outcomes (2492)

Internal, nongovernment 

report

RCT Moderate Literacy skills 

Evaluation of the College 

Possible Program Results 

from a Randomized 

Controlled Trial (2526)

Internal, nongovernment 

report

RCT Moderate Enrollment in 4year 

colleges , College 

enrollment 

Impact Evaluation of the 

Minnesota Reading Corps 

K-3 Program (2366)

Internal, nongovernment 

report

RCT Moderate Literacy assessment 

scores 

Outcome Evaluation of 

the Minnesota Reading 

Corps Pre-K Program 

(2369)

Internal, nongovernment 

report

QED: Groups 

formed by 

matching

Moderate IDGI literacy outcome 

measures 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

12

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Achieving High School 

Graduation: Citizen 

Schools’ Youth Outcomes 

in Boston (1370)

Internal, nongovernment 

report

QED: Groups 

formed by 

matching

Moderate School engagement , 

Academic achievement 

Girls Incorporated of 

Alameda County 

AmeriCorps Program 

Evaluation (1535)

Internal, nongovernment 

report

QED: 

Comparison 

group – other

Preliminary Attendance , Reading 

proficiency 

Evaluation of National 

and Community Service 

Programs Final Report 

(1471)

Internal, nongovernment 

report

QED: Groups 

formed by 

matching

Preliminary Academic performance 

2007-2008 Evaluation of 

Jumpstart in California 

(1357)

Internal, nongovernment 

report

QED: Nonequivalent 

comparison 

group

Preliminary Literacy , Socialemotional development 

NP NCAC. The National 

College Advising Corps 

Evaluation Report (2442)

Government 

report (e.g., 

grey literature)

QED: Nonequivalent 

comparison 

group

Preliminary Attitudes and behaviors 

associated with postsecondary achievement 

An Investigation of the 

America-Reads 

Mississippi Program: 

What Works and What 

Doesn’t Work (843)

Internal, nongovernment 

report

Pre-Post Preliminary Reading achievement 

The 16 studies examined in detail addressed a range of outcomes, with a heavy focus on 

academic performance. Eight studies looked specifically at literacy skills and five studies

included math skills. Two studies had a broader focus on “academic performance” or “student 

achievement.” Four studies focused on school engagement outcomes, including 

attendance/absenteeism and in-school behavior. Two studies examined college enrollment. 

Titles, references and brief summaries of all studies can be found in the annotated 

bibliographies in Appendix B.

Teach for America and Student Achievement

Teach for America (TFA) is one of the largest and the most heavily studied national service 

program that CNCS funds. Four studies analyzing data from three separate RCTs were 

reviewed for TFA. All four studies examined student achievement for students from 

disadvantaged backgrounds taught by TFA versus non- TFA teachers. Three studies looked at 

reading and math performance, while the fourth study looked only at math test scores. Overall, 

results were more promising for math achievement than for reading achievement. The three 

rigorous RCTs that formed the basis for the four TFA reports represent strong evidence. Three 

of the studies reported small but statistically significant results, indicating TFA students 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

13

outperformed other students on math test scores. The fourth study showed no differences in 

math performance. Of the three studies that included an examination of reading achievement, 

only one found a statistically significant positive impact on reading scores, and this was limited 

to children in pre-kindergarten through grade 2. The two other studies that included reading 

achievement found no difference. One study of the four studies also looked at school 

engagement measured by attendance and disciplinary incidents, but found no impact on these 

measures. One of the studies in this set took the data from an original RCT and re-ran the data 

using fixed effect quantile regression (FEQR) analysis, which constitutes a type of QED design,

representing a moderate level of evidence.10

Overall, the evidence presented suggests that TFA does not harm students’ academic 

achievement, and may have small positive impacts on math achievement. The level of evidence 

in this area is strong.11

Additional Interventions to Promote Student Achievement

Ten additional studies focused on various aspects of student academic achievement were 

examined. Six of these studies examined literacy achievement. Separate large-scale studies of 

Reading Partners and Experience Corps used RCT designs to demonstrate statistically significant 

positive impacts on reading skills for program participants in the elementary grades. Two large 

studies of the Minnesota Reading Corps, one RCT focused on students in grades K-3 and one QED

with matched comparison groups focused on Pre-K students, were included. Both studies 

found statistically significant positive effects on program participants’ literacy skills, with the 

largest effects observed among the youngest students. Another QED study looked at 668 

children in grades two and three participating in Girls Incorporated of Alameda County. This study 

found non-statistically significant gains in reading proficiency for program participants relative 

to an unmatched comparison group. Finally, an outcome study of the America Reads Mississippi 

Program provides preliminary evidence showing improved pre-to-post standardized reading 

test scores for students involved in the program.

Some of the studies looking at various aspects of student achievement examined outcomes 

other than (or in addition to) improved literacy skills. The Reading Partners RCT mentioned 

above found statistically significant positive outcomes for math skills. A QED of the California 

Jumpstart program found statistically significant improvements for pre-kindergarten program 

participants on school readiness indicators for literacy and social-emotional development 

relative to a matched comparison group. The level of evidence in this area is moderate.

 

10 This study is entitled “The effect of Teach for America on the distribution of student achievement in 

primary school”.

11 The overall level of evidence is determined based on selecting the highest level at which there were 

multiple studies supporting positive outcomes.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

14

Student Engagement

Three QED studies examined school engagement for various grades. The Girls Incorporated

study cited above found that program participants had higher rates of school attendance than 

non-participants. An evaluation of Citizen Schools found that 8th graders who participated in the 

program were more likely to stay in high school and ultimately graduate. An RCT evaluation of 

case-managed students with chronic absenteeism in the Communities in Schools program found 

that students had reduced absenteeism and modestly improved academic performance.

Evidence in this area is moderate.

Post-secondary Education Preparation and Prospects for Success

Two reports examined interventions designed to promote college enrollment and success. The 

first study is an RCT of the College Possible program, which aims to increase applications and 

enrollment in post-secondary institutions by providing high school juniors and seniors with 

SAT and ACT test preparation and assistance with enrollment and financial aid applications. 

The study found that while program participants were not more likely than the control group to 

apply to post-secondary institutions, when they did so they were more likely to apply to a fouryear college. The second study looked at college enrollment for students mostly from minority 

backgrounds served by the National College Advising Corps (NCAC). This QED, which did not 

establish baseline equivalence, found that seniors who met with an NCAC adviser were more 

likely to take a series of preparatory steps and to ultimately apply to a college or university 

(particularly a 4-year institution) and receive an acceptance. The level of evidence in this area is 

preliminary.

Analysis

Effectiveness of the Interventions

What do the outcomes/impact evaluations of national service programs generally say about the 

effectiveness of those interventions?

Education is the most-studied area in national service and therefore has one of the strongest 

evidence bases within national service. Some of the most common program designs applied in 

national service for supporting student academics have been studied using rigorous methods 

and found to be effective. These interventions, when properly implemented, have generally 

demonstrated positive results for students from diverse backgrounds. Education interventions 

with positive outcomes for service recipients share several characteristics, including intensive, 

individualized one-on-one or small group support. Examples include Reading Partners, 

Experience Corps, and Minnesota Reading Corps.

TFA has been the subject of many studies, including studies employing causal designs. These 

studies have built up a substantial body of evidence for mixed results regarding the 

effectiveness of TFA versus non- TFA teachers in fostering student academic achievement. 

Results for literacy have been approximately equal for students taught by TFA and non- TFA

teachers, while results for math have been somewhat better for students taught by TFA teachers.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

15

Strength of Evidence Base

What is the strength of the evidence base for national service?

Within the education focus area, the strength of the evidence base varies across different 

interventions and program areas. The strongest evidence base exists for intensive interventions 

designed to improve subject-specific academic skills, with one-on-one literacy tutoring being 

the best example. There are fewer studies looking at interventions addressing student 

engagement and post-secondary education preparation, and these studies as a whole do not 

attain the same level of rigor seen in the more familiar areas of student achievement. These 

interventions, although somewhat newer to national service, show promise both as new areas in 

which national service has the potential to make valuable contributions to student success and

as areas for expanding the knowledge base around the state of educational practice. In 

particular, traditional tutoring programs can also include interventions that address student 

engagement, and thus achieve multiplicative effects on student academic performance, 

graduation, and other indicators of student success.

In sum, given the varying strength of evidence across the entire education field, the level of 

evidence is moderate. The relative abundance of research on education interventions means that

the impacts and outcomes of education interventions are better understood than in other focus 

areas. The proportion of rigorous, well-implemented study designs capable of generating 

moderate or strong evidence is highest in education, particularly for intensive literacy and math 

tutoring. Some of the most compelling evidence within this specific category is found in the 

early grades (K-3). There is also a growing body of literature of moderate strength 

demonstrating that school engagement interventions can improve attendance and graduation.

There is a narrow but promising evidence base for college preparatory programs like College 

Possible and NCAC. Both programs showed a meaningful degree of success in promoting college 

enrollment and related outcomes, although only one study (College Possible) provided causal 

evidence to support the intervention.

Learnings, Best Practices, and Key Resources 

What are the learnings, best practices, and key resources that can inform the work of CNCS, its 

grantees, and other stakeholder groups?

The following learnings, best practices, and key resources emerge from the literature on 

education:

 Highly structured, well-implemented, intensive, one-on-one tutoring models are 

effective in promoting academic achievement, especially at the early grades and with 

children from disadvantaged backgrounds. These programs can also produce peripheral 

benefits, such as increased school engagement. Minnesota Reading Corps and other welldocumented models exist to support extension of existing efforts.

 Future interventions in education should capitalize on the lessons learned from 

extensive evaluation efforts by carefully implementing proven service models. Existing 

programs should be rigorously evaluated for effectiveness and should provide feedback 

for program improvement. Other models, such as programs that have not undergone 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

16

impact evaluation, should undergo more rigorous evaluations. It may also be possible to 

graft other promising program components from school engagement programs onto 

traditional tutoring programs to enhance the overall effectiveness of these programs in 

boosting student achievement.

 CNCS can continue to showcase successful interventions in the education focus area and 

prioritize rigorous evaluation of promising new interventions. This can be done through 

a number of electronic means, including the Evaluation Resources page of the CNCS 

Resources website and the Evidence Exchange. Effective practices from these programs 

that can be adopted (or adapted) in part or in whole to other national service grantees

and programs, and contexts can be disseminated to the field through practice briefs and 

implementation guides.

The large body of relatively strong sources in education reflects the strong emphasis on 

education programming within national service, as well as the generally advanced state of the 

education field. Grantees in education tend to have more resources and greater evaluation 

experience/capacity than grantees in other fields. The larger social and political significance of 

education in a merit-based society is a background issue driving the policy and research 

agendas around education. Quantitative measurement of educational outcomes (e.g., through

standardized testing) also means these outcomes are readily amenable to evaluation. The 

wealth of available evidence in education allows practitioners to focus attention on replicating 

well-established models and exploring the contextual nuances of program implementation. The 

role of national service in education will likely continue to be one of supporting implementation 

of successful interventions in high-need contexts and contributing to the knowledge base 

regarding the moderating influence of situation and context.

Economic Opportunity

There were 126 documents identified that included a focus on economic opportunity. Ten of 

these evaluations were impact, 54 were outcome, and 62 were classified as other, and typically 

describe pre- or non-experimental approaches.

Three documents with a primary focus on aspects of economic opportunity were selected for 

full review. All three were impact evaluations. Although one impact study attained a strong

level of evidence and two impact studies attained moderate levels of evidence, the economic 

opportunity literature consists largely of studies that offered preliminary evidence. Table 4 

shows the economic opportunity sources reviewed in detail for this report.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

17

Table 4: Economic Opportunity

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Sustained Gains: Year 

Up’s Continued Impact on 

Young Adults’ Earnings 

(2293)

Internal, nongovernmental 

report

RCT Strong Average annual earnings 

The Effects of AmeriCorps 

on Participant Readiness 

for Education and 

Employment Success 

(2384)

Internal, nongovernmental 

report

QED – Nonequivalent 

groups

Preliminary Acquisition of basic 

work skills , confidence 

in ability to obtain an 

education , 

responsibility for 

educational success

Rebuilding social 

organization in lowincome neighborhoods: 

Influence of habitat for 

humanity (2144)

Thesis or 

dissertation

QED – Nonequivalent 

groups

Preliminary Social cohesion of 

neighborhoods 

The three economic opportunity studies examined in detail for this review addressed outcomes

intended to support economic self-sufficiency, including employment, earnings, and job 

stability; acquisition of basic work skills; and social cohesion of low-income neighborhoods. 

Beneficiary populations encompassed by these studies include low-income individuals and 

families, unemployed and discouraged workers, ASN members from disadvantaged 

backgrounds, and recent Habitat for Humanity homeowners.

Employment and Earnings

One RCT examined the ability of young adults from low-income and disadvantaged 

backgrounds participating in the Year Up ASN program to experience sustained gains in 

earnings three to four years after program participation. At follow-up the study found that Year 

Up graduates earned higher wages than individuals in the control group. The study also found 

that Year Up graduates were more likely to have worked in one of the program's two target 

occupations (information technology and financial operations) at some point since graduating 

from the program.

Basic Work Skills

A QED study assessed the effect of Washington AmeriCorps programs on participants' 

readiness for educational and employment success. Programs included in the study sought to 

improve members' basic work skills. The study found significant positive change in readiness 

for education and employment. Threats to internal validity render the level of evidence 

preliminary.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

18

Social Cohesion

The last study examined how the presence of multiple Habitat for Humanity12 homeowners 

contributed to feelings of neighborhood social cohesion. The QED study with non-equivalent 

groups found modest evidence for greater social cohesion in neighborhoods (blocks) consisting 

of multiple Habitat homeowners. The study achieved a preliminary level of evidence.

Analysis

Effectiveness of the Interventions

What do the outcomes/impact evaluations of national service programs generally say about the 

effectiveness of those interventions?

The studies examined in this section support the proposition that national service and SIF 

interventions addressing the specific human and social capital needs of targeted populations 

have the potential to put beneficiaries on the path to long-term economic self-sufficiency. Some 

interventions, such as home ownership, hold the possibility of more diffuse social and economic 

benefits insofar as these interventions contribute to the stability of neighborhoods and provide a 

firm foundation upon which individuals can pursue other personal goals. These studies also 

provide tentative support to the notion that program participants' success is further enhanced 

by the sense of self-efficacy they may initially gain through program participation, particularly 

through achieving the kind of personal financial and educational objectives these interventions 

are intended to support. 

Strength of Evidence Base

What is the strength of the evidence base for national service?

Evidence for the effectiveness of national service interventions addressing economic 

opportunity varies somewhat by outcome area. Evidence on employability outcomes is 

moderate, while evidence on financial literacy and housing is preliminary. There is preliminary 

evidence to support the effectiveness of interventions providing sustained career training in 

promoting employability and economic self-sufficiency, although the available evidence is 

confined to specific high-demand fields (e.g., information technology, financial operations). In 

the area of work experience programs, paid and unpaid work models with programs lasting up 

to six months or a year showed moderate evidence for impacts on adults. Outcomes for similar 

programs directed at youth are mixed.

Evidence for employability outcomes can be strengthened by incorporating stronger designs 

(i.e., well-designed and implemented RCTs) into existing programs. It should also be feasible to 

subject financial literacy interventions to random-assignment designs that embargo control 

group members from program participation, or QED designs in which comparison group 

individuals are allowed to engage in “treatment as usual.” Further research to strengthen the 

 

12 Habitat for Humanity has many affiliates that include ASN members (and in some cases VISTA 

members) who work alongside other community volunteers or work indirectly to support home 

construction.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

19

evidence base for the effects of Habitat home ownership should build on the existing body of 

qualitative and small-scale quantitative studies to create standardized quantitative measures of 

success that can be used by multiple programs. 

The absence of a critical mass of literature in the economic opportunity focus area reflects a 

number of factors, including the variety of interventions placed under the broad and somewhat 

eclectic heading of economic opportunity. Resources for program evaluation are not 

particularly abundant, and the small, grassroots organizations that often undertake these 

interventions are typically not well resourced. 

Learnings, Best Practices, and Key Resources

What are the learnings, best practices, and key resources that can inform the work of CNCS, its 

grantees, and other stakeholder groups?

The range of interventions in economic opportunity is broader than can be encompassed by the 

current review. However, based on the studies and outcome areas examined, a number of 

specific learning, best practices, and key resources emerged from the literature on economic 

opportunity.

 Workforce development programs are more likely to produce successful outcomes for 

program participants when they focus on opportunities in strong sectors of the local 

economy and involve employers in program design and implementation. These 

programs also succeed when they combine soft-skills training addressing workplace 

standards of attitude, behavior, dress, and communication, with technical job skills. 

Long-term success of workforce program graduates can be boosted by providing 

program participants with multiple opportunities to receive support and guidance, and 

by supporting program graduates to continue their education.

 For youth, linking occupational learning with paid or unpaid work experience

placements can have a positive impact on employment, earnings, and academic 

outcomes.

Environmental Stewardship

The search and review process turned up very few studies with a focus on environmental 

stewardship outcomes. There were 14 documents with a primary focus on environmental 

stewardship. One source was an impact study, seven were outcome studies, and six were 

classified as other (e.g., pre- or non-experimental approaches). Most of the documents assessed 

outcomes within the framework of performance measurement rather than as methodologically 

rigorous impact evaluations. The studies examined the efforts of AmeriCorps members to 

improve specific aspects of the local environment by promoting tree planting or educating 

students on environmental systems. Two documents addressing these aspects of environmental 

stewardship were selected for full review, including one impact evaluation and one outcome 

evaluation. Table 5 shows the environmental stewardship sources reviewed in detail for this 

report.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

20

Table 5: Environmental Stewardship

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Our City Forest 

Evaluation Report (1489)

Internal, nongovernment 

report

QED: Nonequivalent 

comparison 

group

Preliminary Tree survival and health

AmeriCorps Watershed 

Stewards Project 

Evaluation Study Final 

Report (1346)

Internal, nongovernment 

report

Outcome 

only 

Preliminary Watershed education 

and outreach

Both studies were commissioned by ASN grantees in California and were conducted in 2009. 

The first study used a QED group design to compare the health of trees planted and cared for 

by community volunteers trained by AmeriCorps members serving with Our City Forest (OCF)

in San Jose, California. The treatment group consisted of a random sample of OFS trees versus a 

random sample of trees planted and tended by the city. Researchers compared survival rates of

random samples of trees in both groups over a period of approximately two years, and found 

statistically significant positive results for trees planted and cared for by OCF community 

volunteers.

The second study provided information on the performance measurement results for the 

Watershed Stewards Project, an ASN grantee in Northern California. The report included an 

outcome performance measure looking at knowledge changes in K-12 students participating in 

the program’s Real Science Watershed Education Program. The single-group pre-post study 

showed that a majority of students increased knowledge of watershed processes.

Analysis

Effectiveness of the Interventions

What do the outcomes/impact evaluations of national service programs generally say about the 

effectiveness of those interventions?

The outcomes examined for environmental stewardship are limited to demonstrable 

improvement in specific environmental conditions (i.e., tree survival and health) and promotion 

of environmental knowledge/skills in the community (i.e., the ability of community residents to 

actuate knowledge on proper tree care, and increased knowledge among K-12 students about 

watershed processes). In these limited areas, the preliminary evidence suggests that national 

service programs have the potential to create positive change for the environment. However, 

the evidence base is too scant to generalize about the effectiveness of these interventions. There 

are a number of potential reasons for the paucity of evidence in the field of environmental 

stewardship. As noted earlier, the potential evaluand is often a natural setting that may be 

geographically dispersed and/or lack clearly delineated boundaries (e.g., a watershed or forest); 

non-human evaluand (e.g., plants and animals) are not amenable to conventional social science 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

21

data collection methods, such as surveys and interviews; and other natural forces that are 

beyond human control (drought, flood, fire, etc.) may also contaminate results.

Strength of Evidence Base

What is the strength of the evidence base for national service?

Evidence in the very limited body of literature for environmental stewardship is preliminary. 

None of the interventions examined for this project has undergone a rigorous evaluation using 

a causal design. No particular intervention or class of intervention can be recommended at this 

time. It is recommended that greater resources be devoted to more systematic and rigorous 

assessment of the most popular environmental stewardship interventions found in national 

service to begin to assess their merits.

Existing models showed preliminary evidence of success, but need to be replicated, particularly 

in varying context (e.g., different communities, interventions designed to ameliorate different 

environmental conditions) to determine their overall viability and to identify how well these 

models work and how they can be refined, improved, and successfully adapted.

Learnings, Best Practices, and Key Resources

What are the learnings, best practices, and key resources that can inform the work of CNCS, its 

grantees, and other stakeholder groups?

The QED study conducted by the ASN grantee in San Jose, CA provides a promising example 

of using a causal design to assess environmental stewardship outcomes. The OCF study is a 

useful example of how an environmental program can approach questions of causality. 

Although studies using random assignment to treatment (e.g., tree care, stream cleaning, or 

park areas improvement) would provide the strongest evidence, the currently very limited 

evidence base for environmental national service programs could be expanded using similar 

approaches such as random sampling or statistical matching of treated and untreated areas or 

locations.

Inclusion of implementation components may be especially appropriate in evaluations of 

environmental stewardship interventions. This is because fidelity of implementation (e.g., in the 

context of environmental remediation efforts) can factor heavily into the success or failure of 

such interventions. Also, since quality control monitoring is often an integral part of many welldesigned environmental stewardship efforts, in most cases it would be relatively easy to 

incorporate an implementation component into a study design.

Disaster Services

There were nine documents that primarily focused on disaster services. The search and review 

process turned up very few studies with a focus on disaster services, and the documents that 

were found typically did not focus on outcomes. Instead, these reports were mainly descriptive 

and sought to offer examples to the field for possible replication. Aspects of program design 

were discussed, but were not necessarily described in sufficient detail to serve as a how-to 

guide for replication in other settings. Table 6 shows the disaster services source reviewed in 

detail for this report.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

22

Table 6: Disaster Services

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

AmeriCorps NCCC’s Role 

in Homeland Security

(1375)

Internal, nongovernment, 

report

Other—case

study

PrepreliminaryNot applicable

The one study examined, “AmeriCorps NCCC’s Role in Homeland Security”, was a case study 

of disaster preparedness projects carried out by two AmeriCorps NCCC teams addressing the

specific subarea of disaster services known as homeland security13. The study sought to assess 

the nature and effectiveness of AmeriCorps NCCC teams’ activities in carrying out homeland 

security-related projects and identify roles in which AmeriCorps NCCC teams could be 

successful in helping communities achieve homeland security objectives. Researchers concluded 

that NCCC teams could help communities “jump-start” disaster preparedness by injecting 

substantial human resources into a location on relatively short notice. The study also 

recommended the creation of a specialized “disaster-preparedness and homeland security” 

track in the NCCC program that eventually led to the creation of FEMA Corps, a 1,600-member 

strong branch of NCCC solely devoted to disaster preparedness, response, and recovery.

Analysis

Effectiveness of the Interventions

What do the outcomes/impact evaluations of national service programs generally say about the 

effectiveness of those interventions?

Existing studies in the disaster services focus area are limited to descriptive case studies with 

information about service project implementation and potential best practices. These studies are 

intended to provide practitioners with some degree of how-to guidance for program 

implementation. Since these studies did not engage in assessing outcomes, they do not speak to 

the effectiveness of these interventions.

Strength of Evidence Base

What is the strength of the evidence base for national service?

Evidence in disaster services is pre-preliminary, with no studies currently available that address

impacts or outcomes. The paucity of evidence in the field of disaster services can be explained 

by several factors. As noted earlier, disasters are inherently unpredictable and episodic events 

that pose challenges to the careful planning and implementation typically required for 

 

13 Homeland security addresses a specific subset of human-instigated disasters related to terrorism and 

national security. The field of disaster services encompasses the full range of disasters, including natural 

disasters and human-instigated disasters not connected to security threats (e.g., transport accidents, oil 

spills).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

23

evaluation. Data collection in the aftermath of a disaster can pose serious logistical challenges 

for researchers and put undue strain on human subjects.

Nonetheless, evaluation of each of the four phases of disaster services (mitigation, 

preparedness, response, and recovery) is possible. Feasible approaches to evaluating selected 

disaster services efforts exist—specifically in the area of disaster preparedness. For example, 

surveys could be administered to trained responders (including volunteers) to test their 

knowledge of emergency procedures. Their performance could also be evaluated during drills 

and simulated emergencies. These approaches, and possibly others, could be adapted for use in 

the national service context.

Learnings, Best Practices, and Key Resources

What are the learnings, best practices, and key resources that can inform the work of CNCS, its 

grantees, and other stakeholder groups?

The learnings, best practices, and key resources identified by the NCCC case study are as 

follows:

 Effective integration of spontaneous volunteers into disaster response efforts requires 

the preparation of written plans that dovetail with existing emergency plans and that 

incorporate local and state actors into the management of volunteers.

 NCCC teams are well positioned to serve as key resources to “jump-start” disaster 

preparedness efforts.

Healthy Futures

There were 55 documents primarily that included a focus on healthy futures. Eight of the 

sources were impact, 25 were outcome, and 22 were classified as other, and typically describe 

pre- or non-experimental approaches.

Table 7 shows the healthy futures sources reviewed in detail for this report.

Table 7: Healthy Futures

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Making a Difference: An 

Impact Study of Big 

Brothers Big Sisters (1373)

Internal, nongovernment 

report

RCT Strong Relationships with 

family , relationships 

with friends , selfconcept , social and 

cultural enrichment, 

Antisocial activities

Senior Companion 

Quality of Care 

Evaluation Final Report 

(1805)

Internal, nongovernment, 

report

QED –

Groups 

formed by 

matching

Moderate Quality of life , client 

satisfaction with services 

, reduction in needs for 

assistance with unmet 

ADLs
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

24

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Providing Quality of Life 

Through Intentional 

Community Living: 

Outcomes for the 

Camphill AmeriCorps 

Education Award 

Program (1351)

Internal, nongovernment, 

report

Single group 

– outcome 

only

Preliminary Quality of life (physical, 

psycho-social, spiritual) 

Public Report on the 

Results of the 2005 

Independent Living 

Survey of the Senior 

Companion Program 

(1376)

Other–

Hardcopy 

document 

available at 

CNCS

Single group 

– outcome 

only

Preliminary Satisfaction , 

responsive to needs , 

quality of life (improved 

QoL , peace of mind , 

maintain overall 

physical health) , 

companionship 

Evaluation of the 

SCP/AoA Joint Initiative 

for the Vulnerable Elderly 

Program (1810)

Internal, nongovernment 

report

Single group 

– outcome 

only

PrepreliminaryPsycho-social well-being 

of SCP clients 

AmeriCorps Members 

Increase Enrollment in 

Medicaid/CHIP and 

Preventive Care 

Utilization at a 

Community Health Center 

(838)

Peer-reviewed 

journal article

Comparison 

group – other

Preliminary Medicaid/CHIP 

enrollment , 

subsequent health care 

utilization 

In Service to Community -

Giving the Story a Name: 

The Community 

HealthCorps Impact 

Evaluation (1804)

Internal, nongovernment 

report

Single group 

– outcome 

only

PrepreliminaryImpact on individuals 

and families served 

Wisconsin Farm to School: 

One Year Evaluation 

Report (1360)

Internal, nongovernment 

report

Single group 

pre-post

Preliminary Knowledge and 

attitudes about fruit and 

vegetables , diet 

behaviors 

The eight studies examined in detail included two impact evaluations and six outcome studies. 

These studies addressed a range of outcomes, including social relationships, quality of life, 

access to health care, healthy dietary habits, general health, and survival. Beneficiary 

populations encompassed by these studies included youth mentored by Big Brothers and Big 

Sisters, homebound and/or frail elderly, low-income individuals and families, students age 6-12, 

national service participants with disabilities, and adult volunteers. The set of studies examined 

in detail included older studies covering a time span from the mid-1990s through 2014. The 

subsections below describe studies according to outcomes addressing quality of life 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

25

(particularly for seniors), health care access and utilization, resiliency against violence and 

drug/alcohol abuse, healthy diet, and the health benefits of volunteering.

Quality of Life

Four studies looked at various quality of life outcomes. Three studies looked at Senior Corps 

Program participants, while the fourth study looked at individuals with disabilities served by 

an ASN grantee.

One study, the “Senior Companion Quality of Care Evaluation Final Report”, was a welldesigned and implemented QED examining the experience of frail elderly individuals served by 

Senior Companions compared to individuals on a waiting list and another group receiving 

other services. The study found positive physical and psychosocial outcomes for the treatment 

group, although these results dissipated after nine months.

Another study, “Public Report on the Results of the 2005 Independent Living Survey of the 

Senior Companion Program”, was largely descriptive, but did include results assessing SCP 

clients’ self-reported quality of life. The report noted that 90 percent of SCP clients reported 

improvements in quality of life, including improvement in eating habits and maintaining 

overall physical health. The study did not address whether these outcomes were sustained.

The third quality of life study, “Evaluation of the SCP/AoA Joint Initiative for the Vulnerable 

Elderly Program”, was a single-group outcome study that looked at the psychosocial well-being 

of SCP clients. The study found that the program helped homebound clients maintain 

independence in various activities of daily living.

The fourth quality of life study, “Providing Quality of Life Through Intentional Community 

Living: Outcomes for the Camphill AmeriCorps Education Award Program”, was an outcome 

evaluation of individuals with physical and/or mental disabilities served by ASN members in 

the Camphill Association. The study found that participants experienced an adequate to 

excellent quality of life and benefited particularly from the creation of a supportive social fabric 

through program participation.

As a whole, these four studies suggested that meaningful short-term quality of life 

improvements were feasible under individualized service models such as the Senior 

Companion Program and programs like Camphill. One-on-one models of service could also 

make a difference in the lives of persons with disabilities. To the extent it was examined, 

sustainability of positive outcomes was found to be a challenge particularly for elderly clients. 

One common theme running through all four studies was the importance of psychosocial 

factors for physical and mental well-being. In particular, reducing social isolation tended to 

promote positive outcomes. Evidence is this area is preliminary.

Health Care Access and Utilization

The first of two studies examining health care access and utilization was a QED using

statistically equivalent convenience samples of low-income families with uninsured children 

presenting for health care services at two community health center clinics. One clinic utilized 

AmeriCorps members (sponsored by the Association for Utah Community Health) to provide 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

26

case management (treatment), while the other clinic did not have case management by 

AmeriCorps members (comparison). The QED attained only a preliminary level of evidence due 

to selection bias and attrition. Administrative data showed that treatment group children were 

much more likely to successfully enroll in Medicaid/CHIP. However, there were no differences 

in health care utilization for enrolled children in the treatment and control groups.

The second study in health care access and utilization was an outcome evaluation reporting 

results from a 2010 study of the Community HealthCorps (ASN). The study, which attained a prepreliminary level of evidence, consisted largely of routinely documented performance 

measurement data. The study drew upon surveys of health center staff and 

administrators/program coordinators to conclude that members facilitated access to, and 

utilization of, health care by low-income individuals and families. Evidence in this area is 

preliminary.

Resiliency Against Violence and Drug/Alcohol Abuse

The one study examined in this area compared the experiences of youth mentored through the 

Big Brothers Big Sisters (BBBS) program to a control group of youth placed on a waiting list for 

12 months. The RCT provided strong evidence of positive results. BBBS participants were less 

likely than their control group counterparts to initiate drug and alcohol use, engage in violent 

behavior, and skip school. BBBS youth were also more likely to show modest gains in grades, 

exhibit trust towards their parents, and report improved relationships with their peers. Several 

of these effects were stronger for minority youth than for white youth. Researchers did not find 

statistically significant improvements in self-concept or in the number of social and cultural 

activities in which BBBS participants engaged.

Healthy Diet

This single group pre-post outcome study attained a preliminary level of evidence. The study

found that students aged 6-12 that were exposed to information about healthy food choices and 

given more opportunities to try fruits and vegetables increased knowledge, favorable attitudes, 

and consumption of fruits and vegetables. These improvements tended to increase 

incrementally with years of involvement in the school-based program. The study exhibits a 

preliminary level of evidence.

Analysis

Effectiveness of the Interventions

What do the outcomes/impact evaluations of national service programs generally say about the 

effectiveness of those interventions?

The diverse array of outcomes and impacts considered in this section limit the ability to 

generalize about the effectiveness of particular interventions in the focus area of healthy futures. 

However, the following observations were made:

 A number of interventions are potentially effective in addressing the health and wellbeing needs of a variety of service recipients, especially low-income individuals and 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

27

families, individuals with disabilities, and frail and homebound seniors. These benefits 

extend to healthcare access, youth behaviors, and quality of life factors.

 Psychosocial factors appear to contribute to quality of life outcomes, including physical 

health outcomes. Reducing social isolation may be particularly important in promoting 

positive outcomes.

 Short-term positive results may give way to no results or ambiguous results with the 

passage of time, particularly for service populations with ongoing needs, lack of 

alternative social supports, or high risk of health decline.

 Impact evaluations, while very few in number, present some evidence to support the 

effectiveness of interventions in this focus area. Outcome evaluations suggest the 

potential existence of real and meaningful improvements for service recipients in a 

variety of areas. These outcome studies point to the need to pursue impact evaluations 

in these programmatic areas.

Strength of Evidence Base

What is the strength of the evidence base for national service?

The evidence base for healthy futures interventions is preliminary and limited in scope to a 

relatively small number of interventions and service models.

The most rigorous study reviewed, “Making a Difference: An Impact Study of Big Brothers Big 

Sisters”, found strong evidence for mentoring to reduce substance abuse and improve familial 

relationships among at-risk youth.

Prioritization of the research agenda in healthy futures, particularly for impact evaluations, 

should match the allocation of national service resources to different types of interventions in 

this focus area. Given the relatively strong emphasis of national service on youth mentoring, 

supportive services for homebound and frail seniors, and increasing health care access, 

interventions in these areas could be considered for more rigorous research.

Community based organizations operating healthy futures programs and services have a long 

tradition of focusing on client services and expanding the range of service provided to meet 

client needs. This is well illustrated by local agencies serving the elderly, which have expanded 

their repertoire of services to include nutrition, caregiving, prevention of elder abuse, health 

and wellness, intergenerational programming, transportation, and making communities more 

livable for individuals with disabilities. The diversity of interventions and beneficiaries presents 

a challenge to standardized program evaluation, and organizations in the field have not always 

prioritized evaluation due in part to the perceived intrinsic value of providing much-needed 

services. National service, as a key sponsor of many healthy futures initiatives, is well placed to 

organize large-scale research studies that can build evidence and foster local evaluation 

capacity simultaneously.

Learnings, Best Practices, and Key Resources

What are the learnings, best practices, and key resources that can inform the work of CNCS, its 

grantees, and other stakeholder groups?
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

28

National service members can be particularly effective in supporting clients’ access to service 

providers (such as health care providers). In particular, service members can do this by raising 

awareness about the availability of services and by helping clients navigate the system to gain 

access to services.

As illustrated by several sources examined here, one-on-one service models produce high levels 

of client satisfaction and tend to be most effective in serving the needs of vulnerable 

populations, including at-risk youth, frail and homebound seniors, and people with disabilities.

Research on interventions to promote healthy diet and reduce the risk of obesity (as noted in the 

Wisconsin Farm to School study) indicate that increased knowledge combined with increased 

access to healthy food choices can yield positive and potentially lasting results. Future research

in this area could delve deeper to identify additional factors that positively affect health.

Impact on National Service Participants (“Member Development”)

Skills Relevant to Professional Development

There were 170 documents that included a focus on aspects of member development related to 

job skills, life skills, and professional development. Twenty-six of these studies were impact, 73 

were outcome, and 71 were classified as other. The eight sources that were selected for in-depth 

review are listed in Table 8.

Although member development has been a routine part of CNCS performance measurement for 

many years, particularly for ASN programs, relatively few rigorous studies have been produced 

that examine the impact of service on participants’ professional development skills. Eight 

documents addressing these aspects of member development were selected for full review, 

including six impact evaluations and two outcome evaluations. Table 8 shows the member 

development sources reviewed in detail for this report.

Table 8: Member Development: Skills Relevant to Professional Development

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

National Evaluation of 

Youth Corps: Findings at 

Follow-up (2000)

Internal, nongovernment, 

report

RCT Strong Employment, school 

attendance, educational 

attainment, financial 

security 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

29

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Serving Country and 

Community: A 

Longitudinal Study of 

Service in AmeriCorps

(1418)

Government 

report (e.g., 

grey literature)

QED –

groups 

formed by 

matching

Moderate Education: acceptance of 

responsibility for 

educational success, 

confidence in ability to 

obtain an education , 

educational progress 

Employment: acceptance 

of responsibility for 

employment success, 

basic work skills , 

public service 

employment 

Teamwork and other life 

skills: appreciation of 

cultural and ethnic 

diversity , constructive 

group interaction , 

constructive personal 

behavior in groups

Examining the career 

choices and civic 

engagement activities of 

Teach for America alumni

(1704)

Thesis or 

dissertation

QED – nonequivalent 

comparison 

groups

Preliminary Making choices 

regarding graduate 

school study and careers 

that positively impact 

educational and social 

reform , employment 

in nonprofit or public 

sector 

Improving Lives and 

Communities: 

Perspectives on 40 Years 

of VISTA Service (1815)

Internal, nongovernment, 

report

QED – nonequivalent 

comparison 

groups

Preliminary Educational pursuits , 

labor force status 

The Effect of the City Year 

Experience Over Time: 

Findings from the 

Longitudinal Study of 

Alumni (2473)

Internal, nongovernment, 

report

QED – nonequivalent 

comparison 

groups

Preliminary Organizational 

leadership , overall 

social capital score
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

30

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Making a difference: 

Impact of AmeriCorps 

State and National Direct 

on Members and 

Communities 1994-95 and 

1995-96 (2511)

Internal, nongovernment, 

report

QED – nonequivalent 

comparison 

groups

Preliminary Increased educational 

attainment ; increased 

educational 

opportunities ; 

improved life skills: 

communication skills, 

interpersonal skills, 

analytical problem 

solving, understand 

organizational systems, 

information technology 

skills

Leveling the field is not 

enough: Promoting 

culturally responsive 

pedagogy in Teach for 

America (335)

Thesis or 

dissertation

Pre-Post Preliminary Intercultural competence

, colorblindness , 

White racial identity 

attitudes

The Effects of AmeriCorps 

on Participant Readiness 

for Education and 

Employment Success

(2384)

Internal, nongovernment 

report

Pre-Post Preliminary Basic work skills , 

confidence in ability to 

obtain an education , 

responsibility for 

educational success 

With the exception of one study that attained a strong level of evidence and another one that 

attained a moderate level of evidence—one utilizing an RCT design and another using a QED 

with statistical matching—the remaining six studies either used a QED design without 

statistical matching or relied on single-group pre/post outcomes design that attained a 

preliminary level of evidence. As noted below, the lead RCT study produced null findings, 

which may have been due to study implementation issues.

The studies examined in detail collectively represent the strongest evidence currently available 

addressing skills relevant to professional development. The studies covered a wide range of 

programs, including ASN, TFA, NCCC, VISTA, and LSA. Several studies followed cohorts of 

national service participants for up to one year, but one had a three-year follow-up period. Most 

of the studies were recent, with the oldest one being conducted in 2007.

The RCT study of The Corps Network examined post-service employment, school attendance, 

and educational attainment in The Corps Network. The study failed to find statistically significant 

differences between treatment and control group members on any of the key outcomes. The 

authors attributed the null findings to study implementation issues, including small sample 

size, measurement timing, and the possible (unmeasured) enrollment of control group members 

in other youth corps programs.

The quasi-experimental (QED) studies examined in detail covered a wide range of outcomes 

addressing relevant hard skills and soft-skill areas, as well as continued pursuit of education 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

31

and employment goals, for ASN, TFA, and VISTA programs. Only one study used statistical 

matching to equate treatment and comparison groups. One longitudinal study encompassing 

ASN and NCCC programs found statistically significant positive outcomes for acquisition of 

basic work skills and attitudes supporting employment success, but also found statistically 

significant negative effects for appreciation of cultural and ethnic diversity among NCCC 

alumni. Another QED study of TFA graduates found positive results regarding ongoing 

commitment to the education field, both in terms of employment and post-secondary education 

pursuits. A QED study of VISTA alumni found they had better post-service employment 

outcomes than non-participants, although this study suffered from low response rates that 

obviated significance testing. Another QED study of 60 ASN grantees nationally found the 

greatest gains in member hard skills and soft-skills occurred in members whose service fit a 

“human services” programmatic focus, suggesting a possible relationship between the types of 

service members perform and the types of skills they can expect to acquire.

In addition to studies covering large numbers of grantees, some studies were conducted on 

single, large grantees. The authors of one such study, on City Year alumni, found modest gains 

in social capital scores (a blend of life skills, job skills, and civic engagement measures) of 

alumni relative to non-alumni, but cautioned against attributing causality due to small sample 

size and lack of statistical power.

Among outcome studies, one looking at ASN grantees in Washington State, found increases in 

attitudinal measures of readiness for education and employment success along with gains in 

basic work skills. Another outcome study looking at TFA teachers found stagnation or decline 

in participants’ ability to engage in intercultural competence and provision of culturally 

responsive pedagogy. However, since the study did not employ a causal design, whether the 

observed effects were due to TFA participation cannot be determined.

Analysis

Effectiveness of the Interventions

What do the outcomes/impact evaluations of national service programs generally say about the 

effectiveness of those interventions?

The eight studies examined in detail for this report suggest—with some notable exceptions—

that participation in national service programs generally produces modest positive effects on a 

limited number of skill areas relevant to professional development. ASN and NCCC members 

showed gains in basic work skills compared to non-AmeriCorps members. TFA program 

graduates were relatively more likely to choose education and employment options in the field 

of education than their non- TFA counterparts. One outcome study found ASN members 

increased attitudinal outcomes—and, to a lesser extent, behavioral outcomes—related to 

education and employment. Taken collectively, the set of QED studies found modest 

statistically significant positive outcomes regarding attitudes and behaviors related to education 

and employment.

Nonetheless, the available body of evidence did not provide a basis for identifying consistent, 

positive outcomes for national service participants for specific skills relevant to professional 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

32

development. The RCT looking at ASN members only achieved a moderate level of evidence 

due to methodological shortcomings, and generated inconclusive results. Intriguing negative 

effects were detected in two instances related to cultural competence; a QED study of 

AmeriCorps found that NCCC alumni exhibited reduced appreciation of cultural and ethnic 

diversity, while an outcome study of TFA participants found declines in teachers’ capacity to 

engage in culturally responsive pedagogy. However, the weakness of the study designs in both 

cases cautions against drawing firm conclusions connecting program participation to the 

observed negative effects. Furthermore, these studies were limited to an examination of 

outcomes in the short-term and did not provide insight regarding the long-term effects of 

national service participation.

Strength of Evidence Base

The evidence base for national service with regard to development of professional skills is 

preliminary. This is surprising given the volume of studies in this area. Nonetheless, only two 

of the most promising studies selected for detailed review rated higher than preliminary. Most 

of the QED studies in the set relied on non-equivalent comparison groups. One RCT and one 

QED formed from statistically matched groups merit a rating of moderate. Both studies looked 

at educational attainment and employment, and the QED examined an extensive set of softskills. If more of the QED studies in the set had employed stronger counterfactual designs, then 

it might have been possible to characterize the evidence base as moderately strong. However, as 

it stands, these two studies cannot carry the weight of evidence needed for such a designation. 

The results reported in this literature also have limited internal validity and generalizability.

Learnings, Best Practices, and Key Resources

What are the learnings, best practices, and key resources that can inform the work of CNCS, its 

grantees, and other stakeholder groups?

As demonstrated by “The Effect of the City Year Experience Over Time” and the “National 

Evaluation of Youth Corps Findings at Follow Up”, national service participation tends to have 

the greatest impact on future education success for individuals who enter national service at 

either the low or high end of the educational spectrum. For individuals at the low end of the 

spectrum (i.e., with a weak attitudinal orientation to educational attainment and/or low 

educational achievement), national service can spark greater confidence to succeed, increase 

interest in educational pursuits, and foster greater educational attainment. For individuals at 

the upper end of the spectrum (i.e., with a strong desire to continue their education and/or high 

educational achievement), national service participation provides a valuable boost to existing 

educational plans and helps to clarify career goals. For individuals in this group, national 

service participation also offers the real-world experience needed to connect educational goals 

to specific career paths while simultaneously augmenting financial resources necessary to 

realize these goals (i.e., through access to an education award). Positive educational outcomes 

were more readily evident for the high-achieving group than for the low-achieving group.

Given that educational and employment outcomes can take years to manifest, future 

longitudinal studies should seek to examine outcomes over longer time frames. It is becoming 

increasingly common for individuals to pursue educational and employment goals in a cyclical 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

33

manner; some individuals may decide to pursue graduate degrees only after accumulating 

several years of workforce experience. Postponed graduate education can only be detected over 

longer periods of follow-up with study cohorts. The literature on employment stability of 

AmeriCorps alumni was limited mainly to the VISTA program, and showed no real differences 

between VISTA members and comparison group individuals in post-service employment 

stability.14 The pattern that emerged more generally was one of employment and career stability 

characterized by extended employment duration and limited number of job changes. The 

existing evidence suggests that there is a causal link particularly between participation in ASN 

and NCCC and the propensity to seek employment in the nonprofit and public sectors. 

Additional studies could help solidify this finding.

Two studies examined in this section may indicate a need for greater efforts to train and 

support national service participants who work in low-income and culturally diverse 

communities. These are “Leveling the Field is not Enough” (a study of TFA teachers) and 

“Serving Country and Community” (a longitudinal study of service in AmeriCorps). There are 

plausible grounds for accepting the negative findings of these studies. For example, participants 

in these programs often must deal with novel experiences and unfamiliar social environments,

and may not always react positively to the pressures and challenges they face. In this context, 

there is a real risk that—rather than increasing cultural sensitivity and open-mindedness—

exposure to low-income and diverse communities may reinforce or even deepen negative 

attitudes and behaviors in ways that not only undermine members’ personal development, but 

also undermine beneficiary outcomes. This lesson applies particularly to TFA (where teachers 

are placed in schools situated in low-income and inner city neighborhoods), NCCC (where 

teams of highly interdependent individuals from diverse backgrounds must work and live 

together for extended periods, and where assignments may place these teams in low-income or 

diverse communities), and VISTA (where members support the program’s mission to bring 

individuals and communities out of poverty). However, in the final analysis, the evidence 

supporting these conclusions is not strong enough for more than tentative findings. Therefore, 

more studies with strong designs are needed.

Civic Engagement

There were 248 documents that included a focus on aspects of member development related to 

civic engagement. Twenty-six of these sources were impact, 113 were outcome, and 109 were 

classified as other. Table 9 shows the member development sources reviewed in detail for this 

report.

 

14 “Improving Lives and Communities: Perspectives on 40 Years of VISTA Service”.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

34

Table 9: Member Development: Civic Engagement

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Civic Engagement 

Patterns and Transitions 

Over 8 Years: The 

AmeriCorps National 

Study (1468)

Peer-reviewed 

journal article

RCT (latent 

transition 

analysis)

Strong Volunteering , 

community participation

, civic organizational

involvement , local and 

national voting, civic 

consciousness, 

perceptions of civic 

knowledge

Predicting and cultivating 

public service motivation: 

A longitudinal study 

measuring the effect of 

participation in 

AmeriCorps programs

(1520)

Thesis or 

dissertation

QED –

groups 

formed by 

matching

Moderate Commitment to the 

public interest , 

openness to new ideas , 

knowledge of 

communities , 

attraction to public 

policy-making 

Assessing the Effects of 

Voluntary Youth Service: 

The Case of Teach for 

America (875)

Peer-reviewed 

journal article

QED – nonequivalent 

comparison 

groups

Moderate Civic activity, 

institutional politics, 

social movement, 

charitable giving

Evaluating the long-term 

impacts of AmeriCorps 

service on participants

(859)

Thesis or 

dissertation

QED –

groups 

formed by 

matching

Moderate Group interactions , 

behavior in teams , 

appreciation of diversity, 

community problem 

identification , 

connections to 

community , 

neighborhood 

obligations , civic 

obligations , local civic 

efficacy , grassroots 

efficacy , communitybased activism , 

engagement in the 

political process, 

volunteerism , political 

behavior, civic behavior

, job in the public or 

nonprofit sector

Examining the career 

choices and civic 

engagement activities of 

Teach for America alumni

(1704)

Thesis or 

dissertation

QED – nonequivalent 

comparison 

groups

Preliminary Civic engagement, 

employment in public 

and nonprofit sectors 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

35

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

The Effect of the City Year 

Experience Over Time: 

Findings from the 

Longitudinal Study of 

Alumni (2473)

Internal, nongovernment 

report

QED – nonequivalent 

comparison 

groups

Preliminary Political efficacy, 

egalitarianism , social 

trust , organizational 

membership , 

organizational 

leadership , media 

usage , voting, political 

expression , 

volunteerism , 

volunteer leadership 

Making a difference: 

Impact of AmeriCorps 

State and National Direct 

on Members and 

Communities 1994-95 and 

1995-96 (2511)

Internal, nongovernment, 

report

QED – nonequivalent 

comparison 

groups

Preliminary Civic engagement 

How Undergraduates Are 

Affected by Service 

Participation (601)

Peer-reviewed 

journal article

Pre-Post Preliminary Civic responsibility

Impact of AmeriCorps on 

Members’ Political and 

Social Efficacy, Social 

Trust, Institutional 

Confidence, and Values 

(1788)

Peer-reviewed 

journal article; 

Internal, nongovernment 

report

Pre-Post Preliminary Civic engagement, social 

and personal values 

Civic engagement has been central to the CNCS mission since the agency’s founding, and 

remains a key component of the service experience for all national service programs, but 

especially for ASN, VISTA, NCCC, and activities formerly funded through Learn and Serve 

America (LSA). Studies with varying degrees of methodological rigor have addressed the 

impact of service on participants. Ten documents addressing civic engagement were selected 

for full review, including seven impact evaluations (one RCT and six QEDs), and two outcome 

evaluations.

The majority of studies examined in this section have been published within the last eight years 

(2007 or later), although some studies predate 2000, pointing to the longstanding interest in 

civic engagement as a research topic.

There is only one randomized control trial design in the group. The national study applied

latent transition analysis to data originally collected by CNCS to examine changes in patterns of 

civic engagement among AmeriCorps members over an eight-year period, finding that 

AmeriCorps exposes members to developmental opportunities that nurture sustained civic 

engagement. Findings suggest that national service programs geared toward young people who 

are not in college may hold promise for addressing gaps in civic engagement.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

36

The other six impact studies are evenly split between moderate and preliminary levels of 

evidence. Two studies (“Evaluating the Long-term Impacts of AmeriCorps Service on 

Participants” and “Predicting and Cultivating Public Service Motivation”) attaining a moderate 

level of evidence looked at ASN alumni longitudinally to gauge the impact of participation on 

aspects of civic engagement. Both studies found long-term positive impacts on various 

measures of civic engagement, including commitment to volunteering, openness to new ideas, 

appreciation of diversity, and motivation to pursue careers in public service. Causal 

relationships were stronger in more supportive environments (e.g., focused training, mentoring 

of members, availability of leadership opportunities). The other study with a moderate level of 

evidence examined TFA program graduates’ civic attitudes and behaviors. This study found 

that alumni with continued TFA program involvement were more likely to show sustained 

positive attitudinal changes, but otherwise lagged behind the comparison group in sustaining 

behavioral changes.

All three impact studies with a preliminary level of evidence relied upon non-equivalent 

comparison groups. The first study found that TFA graduates were less likely than comparison 

group counterparts to volunteer, donate money, vote, and voice their opinions on public issues. 

The second study found that City Year had a strong positive impact on the attitudes and 

behavior of alumni, suggesting that City Year, particularly in terms of greater political selfefficacy, trust of others, volunteer participation, and voting participation. The third study 

examined exit survey data from ASN alumni at 60 randomly selected programs, and found that 

alumni were more likely to report they would definitely be involved in future community 

service. Approximately half of these alumni had either applied to or accepted another service 

position.

The two outcome studies examined in this section both found positive results for national 

service participants, including ASN members and LSA Higher Education program participants. 

The study looking at ASN members generally found positive post-service changes in members’ 

social and political awareness and their commitment to future community involvement, 

although the study failed to find changes in members’ beliefs about their role in politics and 

society. The study of LSA Higher Education participants found that LSA participation produced 

modest increases in students’ sense of civic responsibility.

Analysis

Effectiveness of the Interventions

What do the outcomes/impact evaluations of national service programs generally say about the 

effectiveness of those interventions?

While civic attitudes and behaviors have been extensively studied within national service, most 

of the studies provided only preliminary or moderate levels of evidence for the positive results 

typically found regarding civic engagement. Furthermore, these effects tended to be modest in 

size and varied by program. Results were generally most consistent within ASN and NCCC, 

although results for TFA graduates were mixed. The following observations can be offered 

regarding effectiveness of interventions addressing civic engagement:
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

37

 Studies have sought to disentangle the issue of positive predisposition from actual 

program effects on service participants (members). This has been done with reasonable 

success, particularly for the ASN program.

 Results vary across programs, with ASN typically showing more positive short-term 

results than NCCC, but with these differences tending to disappear over time (factors 

potentially deriving from demographic differences in the individuals these programs 

tend to attract and recruit).

 Findings suggest that national service programs geared toward young people who are 

not in college may hold promise for addressing gaps in civic engagement.15

Strength of Evidence Base

Overall, the level of evidence for the body of literature in this area can be characterized as 

moderate. More RCTs are needed to strengthen the evidence base and validate findings from 

less rigorous impact and outcome studies. While positive evidence is fairly consistent across 

national service programs, it is demonstrated most compellingly in large-scale impact studies, 

such as the one conducted for City Year. Furthermore, the existing longitudinal studies point to 

the value of conducting more extensive examinations of the long-term impacts of national 

service on alumni.

Learnings, Best Practices, and Key Resources

What are the learnings, best practices, and key resources that can inform the work of CNCS, its 

grantees, and other stakeholder groups?

The following learnings, best practices, and key resources emerged from the literature on civic 

engagement:

 Continued civic engagement is moderated by a number of factors that arise before, 

during, and after service, including participants’ demographic characteristics, presence 

or absence of supportive supervisors or mentors during service, and subsequent 

educational attainment and career choices.

 The effect of national service on some civic attitudes and behaviors is manifest in the 

short-term (e.g., personal growth through community service, community problem 

identification), while other effects may emerge later and/or demonstrate longevity (e.g., 

local civic efficacy, connection to community).

 Service learning has value for enhancing student development during the 

undergraduate years, while simultaneously fulfilling a basic institutional mission of 

providing service to the community.

 

15 “Civic Engagement Patterns and Transitions Over 8 Years: The AmeriCorps National Study”.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

38

Impact on Organizations and Communities (Organizational Capacity and “Strengthening 

Communities”)

Ninety-two documents included a focus on organizational capacity and sustainability. Eight

sources were impact, 48 were outcome, and 36 were classified as other. Table 10 shows the three 

organizational capacity and sustainability sources reviewed in detail for this report.

Table 10: Organizational Capacity and Sustainability

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Capacity as a 

Fundamental Objective:

Definition and 

Measurement in 

AmeriCorps VISTA -

Habitat for Humanity 

International (1512)

Government 

report (e.g., 

grey literature)

QEDComparison 

group – other

Moderate Organizational capacity 

AmeriCorps VIP 

Volunteer Capacity Study: 

November 2012 (2516)

Internal, nongovernment 

report

Groups 

formed by 

matching

Moderate Organizational capacity 

, volunteer recruitment 

, elements of a 

successful volunteer 

program 

The Sustainability of 

AmeriCorps VISTA 

Programs and Activities 

(2530)

Government 

report (e.g., 

grey literature)

Single group 

– outcome 

only

Preliminary Project survival

While organizational capacity and sustainability have less prominence as a CNCS focus area in 

comparison to other focus areas (e.g., education), CNCS has promoted organizational capacity 

and sustainability as key objectives for national service for many years. The best example is the 

VISTA program, which has a key objective of strengthening the capacity of the organizations in 

which members serve. Similarly, leveraging community volunteers has been a core service 

activity for ASN members, and grantees routinely report the number of volunteers recruited as 

part of their performance measures. At a broader level, CNCS has also sought to promote 

sustainability of local programs by requiring grantees to secure matching funds from 

community partners. VISTA and NCCC projects often focus on developing the infrastructure of 

host or sponsor organizations so that CNCS-funded efforts can have a lasting impact.

Three documents addressing aspects of organizational capacity and sustainability were selected 

for full review, including two impact studies and one outcome evaluation. The literature in this 

topic area gives greater attention to organizational capacity than to sustainability.

Arguably the most important study in the body of national service literature dealing with 

organizational capacity is the CNCS-sponsored study of “Capacity as a Fundamental 

Objective.” This QED study laid out a detailed conceptual framework for organizational 

capacity operationalized through the Capacity Index Scoring System (CISS). The CISS addresses 

organizational, financial, public advocacy, and programmatic facets of organizational capacity. 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

39

The author used the CISS to compare all Habitat for Humanity (HfH) affiliates known to have had 

recent VISTA projects (n=135) to a random sample of 400 additional HfH affiliates without 

recent VISTA projects. The QED study design attained a moderate level of evidence and found 

that affiliates with a VISTA component had total capacity scores nearly four times higher than 

organizations without a VISTA component, holding constant the effects of other factors in the 

model.

Another study looked at the AmeriCorps Volunteer Infrastructure Project (VIP), which places 

teams of ASN members in communities to provide infrastructure support for volunteer 

programs in service organizations. The QED study with groups formed by matching, which 

attained a moderate level of evidence, found that VIP sites experienced strong, positive changes 

over time on indicators measuring capacity relative to non-VIP sites.

The best example of a study with a primary focus on sustainability is a 1997 outcome study 

examining survival rates for a random sample of VISTA-supported projects two and five years 

after the VISTAs had departed. The study, which attained a preliminary level of evidence, 

found that nearly 68 percent of VISTA-supported projects continued to operate two and five

years after the VISTAs had completed their assignment.

Analysis

Effectiveness of the Interventions

What do the outcomes/impact evaluations of national service programs generally say about the 

effectiveness of those interventions?

The three studies provide evidence to support the notion that national service programs can 

have lasting effects on host organizations and, to some extent, the communities in which these 

organizations work. The best evidence currently available comes from the VISTA program, 

which has an explicit mission focusing on organizational capacity and sustainability. The 

carefully developed and articulated conceptual and measurement framework developed in the 

landmark 1997 study focusing on Habitat for Humanity affiliates could be extended to other 

organizational contexts to advance the theoretical conception and measurement of capacity, and 

to further understanding of the aspects of capacity that most influence accomplishment of 

organizational goals.

Strength of the Evidence

What is the strength of the evidence base for national service?

Overall, evidence in this area is moderate. The two impact studies examined in detail for this 

section exhibit a moderate level of evidence, while other studies identified through the search 

protocol attain a preliminary level of evidence. While the studies contained in the evidence base 

for national service remain largely preliminary in the area of organizational capacity, a solid 

conceptual and measurement framework is embodied in the Capacity Index Scoring System 

(CISS) developed for the 1997 VISTA study. Frameworks like the CISS can provide a rigorous 

empirical basis for assessing the impact of national service interventions designed to build 

organizational capacity and sustainability.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

40

Learnings, Best Practices, and Key Resources

What are the learnings, best practices and key resources that can inform the work of CNCS, its 

grantees, and other stakeholder groups?

Several learnings emerged from the body of literature on organizational capacity and 

sustainability, particularly in relation to the VISTA program. 

 Organizations with current or recent VISTA sponsorship can be expected to experience 

increases in organizational capacity in the areas directly addressed by the VISTA project 

(e.g., volunteer generation). Moreover, the impact of having a VISTA project appears to 

be sustainable over a period of two years or more.

 The impacts of VISTA can be difficult to measure, as they cannot be easily disentangled 

from the effects of other programs, events, and social processes; and because impacts 

may not be seen until months or years later. It is generally easier (and more appropriate) 

to measure the impact of VISTA on sponsoring organizations than on community 

beneficiaries.

Sustainable projects are most likely to be those in which organizations make a conscious effort 

through “continuation planning” to build internal systems and organizational infrastructure to 

sustain the changes accomplished by national service members.

Conclusions

After reviewed each of the focus and impact areas in detail, this section offers summary 

remarks on the strength of the evidence base for national service as a whole, including 

learnings, best practices, key resources, and areas for future research.

Strength of Evidence Base

As noted at the outset this report, there is substantial variation in the strength and depth of 

evidence across the eight focus and impact areas. The areas with well-established evidence are 

largely limited to intensive, subject-specific education interventions aimed at promoting student 

academic achievement. Economic opportunity and healthy futures represent two additional 

areas with a sufficient evidence base to allow tentative observations about the efficacy of 

selected program models. The two domains of member development reviewed contain 

extensive literature that includes studies employing well-implemented causal designs. While 

there is room for more rigorous studies in this area, there is considerable evidence to support 

the claim that national service participation promotes civic values and actions, and shapes the 

educational and career paths of national service alumni in positive ways.

Learnings, Best Practices, and Key Resources

The following learnings, best practices, and key resources emerged from the foregoing sections:

 Education. Highly structured, well-implemented, intensive, one-on-one tutoring models 

are effective in promoting academic achievement, especially at the early grades and for 

children from disadvantaged backgrounds. Within the TFA program, there is evidence 

demonstrating that students from disadvantaged backgrounds may experience small 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

41

improvements in math outcomes, although results are somewhat inconclusive with 

regard to literacy outcomes.

 Economic opportunity. Workforce development programs offer a promising model for 

bringing individuals out of poverty by building marketable skills, boosting selfconfidence, and connecting job seekers to opportunities. These programs can add further 

value to graduates’ training by encouraging them to pursue opportunities for ongoing 

education and certification to render themselves more marketable or to maintain 

flexibility in times of economic change.

 Environmental stewardship. While there are very few readily identifiable best practices 

for environmental stewardship, the example set by the Our City Forest provides a useful 

model for research employing an experimental design to determine causality.

 Disaster services. The body of literature in disaster services is extremely limited, and 

much of the information available in this area comes from the NCCC program. The 

literature suggests that NCCC teams may have a valuable role to play in helping 

communities mitigate and recover from disasters.

 Healthy futures. There is evidence indicating the potential value of national service 

participants in supporting access to and effective utilization of existing health and 

wellness resources, thereby freeing up administrative and professional staff of health 

and wellness organizations to focus on work that requires their specialized skills and 

training. However, more research employing counterfactual designs is needed to 

strengthen these tentative conclusions and to better understand the factors contributing 

to program success.

 Member development: Skills relevant to professional development. National service 

participation brings substantial benefits to members through the soft- and hard-skills 

acquired during service, the expansion of social networks, and the clarification of 

personal goals. Service programs that have a human services or hands-on work focus 

(e.g., NCCC) can provide members with concrete skills, build confidence, and provide a 

basis for further exploration of educational and career possibilities.

 Member development: Civic engagement. To the extent that civic engagement is an 

explicit and integral component of the experience for national service participants, these 

programs continue to successfully promote civic engagement as an ongoing part of 

alumni’s self-identities, as demonstrated, for example, by participation in the 

community, volunteering, and charitable giving. The research has shown that there are 

some service contexts known to carry risks of producing unintended or negative effects 

on civic mindedness, such as putting non-minority TFA teachers in poor, urban schools, 

or putting diverse NCCC teams in the field with limited support. These risks are easily 

countered by providing proper training and support to national service participants 

before and during their service assignments.

 Organizational capacity and sustainability. The VISTA program has demonstrated its 

ability to increase the organizational capacity of host organizations, particularly with 

regard to strengthening volunteer recruitment. There is some intriguing preliminary 

evidence to suggest these gains may be lasting. However, sustainability is an element 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

42

that must be consciously built into capacity-building efforts, either by host organizations 

or as a mandate from funders.

Areas for Future Research

There are fruitful avenues for future research within each of the noted areas.

 Education. Additional research can be conducted to help understand the ancillary 

benefits arising from tutoring and academic mentoring, which may include greater 

student retention, greater learner self-efficacy, increased affective commitment to 

learning, and school engagement. An improved understanding of the mechanisms 

driving these peripheral benefits could help to shape and refine these interventions to 

maximize the additional benefits. More attention can be given to education interventions 

that have a focus other than improved academic performance. For example, programs 

like College Possible that promote post-secondary educational participation among youth 

from disadvantaged backgrounds show great promise, and are worthy of further 

attention from researchers.

 Economic opportunity. Workforce development programs have demonstrated promise 

in pointing individuals in the right direction and helping them to find employment. 

Along with the recommendation emerging out of the literature for ongoing support to 

program graduates, comes the suggestion that researchers apply a longer lens to these 

programs. In other words, more emphasis on longitudinal studies would shed light on 

the value of these programs in promoting sustained self-sufficiency. Longitudinal 

studies could also provide data to inform practice by identifying the elements of 

programs that are successful in achieving sustainable gains for their beneficiaries.

 Environmental stewardship. Environmental programs with a focus on remediating 

natural settings can be made the subjects of studies employing a counterfactual design 

similar to that used by Our City Forest. There remains a great deal of room for 

application of QED designs to interventions in this focus area.

 Disaster services: Aside from the noteworthy efforts of NCCC, AmeriCorps members, 

and Senior Corps volunteers are known to be involved in disaster service activities, 

especially in training community members to respond to emergencies (CERT). These 

training efforts could be evaluated through performance assessment during drills and 

simulations to demonstrate fidelity of implementation and to empirically support 

assumed gains in community preparedness.

 Healthy futures. The value added by national service participants in the area of healthy 

futures is perhaps best understood for interventions addressing quality of life, 

particularly for elderly service recipients and individuals with disabilities. In this area, 

individualized care/service models have shown some success in improving or 

maintaining individuals’ self-rated quality of life, both in terms of physical and 

psychosocial indicators of well-being. However, there continues to be a challenge in 

making these positive results sustainable. A common theme running through all four 

studies was the importance of psychosocial factors for physical and mental well-being. 

In particular, reducing social isolation tended to promote positive outcomes. To date, 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

43

there are no known studies using a counterfactual design to demonstrate the impact of 

independent living services and similar interventions intended to affect quality of life.

 Member development: Skills relevant to professional development. Hypotheses in the 

field of skills relevant to professional development skills have been explored 

extensively, but studies have suffered from design and/or implementation challenges. 

Future research should seek to replicate findings from outcome and QED studies while 

employing more rigorous designs.

 Member development: Civic engagement: There is considerable evidence to support 

the notion that national service participation promotes life-long civic engagement. The 

evidence base is of moderate strength, with studies focusing mainly on former ASN 

members, and finding that service may have long-term effects on alumni attitudes and 

behaviors.

 Organizational capacity and sustainability. The studies that have been done on 

organizational capacity provide a guidepost to future studies that would seek to 

replicate and expand upon the positive findings yielded by existing studies. 

Organizational capacity and sustainability are particularly cogent programmatic 

emphases for CNCS at a time when resources are becoming increasingly scarce. The 

advantages of expanding research in these areas are twofold. First, such research has the 

potential to show the unique value of national service as a cost-effective way to 

strengthen communities. Second, this research could uncover new ways in which 

national service programs can build a basis for self-sufficiency at the community level.

What does the academic literature and research say about national service in general, 

national service interventions in particular, and other CNCS-funded programs?

There is a large and diverse body of literature regarding national service, with more plentiful 

and rigorous studies found in some focus areas (e.g., education) than others (e.g., 

environmental stewardship, disaster services). National service has established a presence in 

American social and cultural life, as hundreds of thousands of Americans have participated in 

service through CNCS-sponsored programs. Both as an agency and as an idea, national service 

continues to maintain its relevance.

CNCS, through its support for program evaluation, has the potential to become an important 

voice promoting promising interventions, program models, and practices. National service has 

contributed to building the evidence base for public sector programs. Building on the current 

evidence base, each focus area will require an ambitious research agenda coupled with efforts to 

promote the evaluation capacity of grantees. This can be done through consistent messaging to 

grantees about the importance of evaluation and by providing effective guidance and resources 

to foster an “evaluation culture” within national service. As grantees build evaluation capacity, 

they will be able to participate in more rigorous causal evaluations, whether in partnership with 

CNCS or on their own. Nurturing existing relationships and creating new ones with current 

and potential partners in state and local government, academic institutions, and the 

philanthropic community can accelerate this process.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

44

In recent years, CNCS has also begun to explore innovative ways to “raise the bar” for building 

evidence by working with the field to promote the use of more rigorous evaluation designs. The 

trend toward a growing proportion of impact studies is addressed in the introduction to this 

report. Therefore, it is simply worth reiterating that the growth in impact studies reflects a 

convergence of factors, both within the population of grantee organizations and within CNCS 

as an agency, that has given greater attention to evidence-driven programming over time.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

45

Appendix A: Search and Review Processes

The following describes the systematic search and review process used for the national service 

literature synthesis.

Systematic Search Process

The overall search process was conducted according to a Systematic Search Protocol developed 

for this systematic review and consisted of three steps:

1. Initial search,

2. Preliminary screening for eligibility, and

3. Coding for selection in the review.

The initial search screened out documents that were clearly not on topic (e.g., not concerning 

national service, outcomes not applicable to CNCS programs), not fact based, not in English, or 

not organized in an appropriate file type (e.g., an Excel spreadsheet). The preliminary screening 

examined each source for uniqueness and relevant timeframe (published after 1990), and 

ensured that reviewed documents were relevant to answer national service-related research 

questions. Coding for selection in the review process consisted of entering basic information 

about each document into a Microsoft Access database, including study type, funding source, 

and outcome focus area.

Documents were obtained from the following sources:

1. Hardcopy documents available at CNCS

2. Electronic sources on CNCS Servers

3. Hardcopies available outside CNCS

4. Open online sources

o Online databases, indexes, and search engines (e.g., EBSCO Host)

o Online theses and dissertation databases (e.g., ProQuest Dissertations and Theses)

o Online searches of clearinghouses (e.g., What Works Clearinghouse)

5. Documentation from CNCS-affiliated organizations (including documents retained, 

published on websites of, or recommended by, state service commissions and SIF 

grantee organizations)

The Systematic Search Protocol describes a series of search strategies appropriate to each 

medium (e.g., hardcopy, online). This includes the process for testing online sources using 

search terms, example search terms, and proposed strategies for creating and applying search 

term combinations to ensure comprehensive and timely exploration of online resources.

Systematic Review and Data Extraction

Reviewers selected a subset of documents for in-depth review, which was conducted according 

to a Systematic Review Protocol. In order to highlight the studies that provide the strongest 

evidence for the potential value of an intervention, items were selected using a tiered review 

process based on study rigor. In selecting documents for in-depth review, researchers looked 

for studies that used the most rigorous designs, prioritizing random control trials (RCT), 

followed by quasi-experimental (QED) designs that established baseline equivalence between 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

46

the treatment and comparison group, followed by QED studies that did not establish baseline 

equivalence, followed by outcome studies without a causal design (e.g., single group pre-post). 

An implementation study was chosen to represent one focus area (disaster services) because no 

impact or outcome studies were available in this area. There were several important modifiers 

to this selection process:

1. When multiple studies of comparable rigor were available, the most recent studies were 

selected.

2. Due to the unexpectedly large number of eligible documents that were found in the 

initial search process, document review included checks for eligibility, confirmation of 

study type, and the addition of supplementary coding information (e.g., Key Program, 

to allow for querying of information about associated programs with multiple name 

variants).

3. At the broadest level, the review seeks to represent the strategic focus areas of 

education, economic opportunity, environmental stewardship, disaster services, and 

healthy futures. In addition, the report seeks to include two key aspects of member 

development (civic engagement and skills related to professional development) as well 

as organizational capacity and sustainability. National service programs explicitly 

encouraged to promote the personal and professional development of national service 

participants through formal training and other opportunities. Organizational capacity 

and sustainability, an explicit goal of the VISTA program, is also important component 

of the larger mission of national service.

4. Studies were also selected based on a desire to include a representative array of program 

designs or outcome areas within a focus area. For example, the education area includes 

studies on TFA because this is a significant national service program. In addition, the 

education area includes two studies on college preparation and enrollment programs 

due to the importance of this programmatic focus within national service.16

In order to complete the in-depth review, reviewers thoroughly screened each document and 

extracted data using the Review Data Extraction form (RDE). The RDE form included five 

sections: Document Details, Program Description, Methodology Basics, Results, and 

Methodological Quality. After ascertaining the methodological quality of the study, the 

reviewer determined a level of evidence using the levels of evidence designations cited in the 

ASN 2015 NOFO: strong, moderate, preliminary, and pre-preliminary.

Information extracted during both the coding and review processes was entered into a

Microsoft Access database. Queries on the data were used to create summary tables and draft 

annotated bibliographies for this report. The literature search began on February 2, 2015 and 

was completed by March 31, 2015. Following this date, some additional documents were added 

from online clearinghouses and by special request.

 

16 If the selection of education studies had been guided strictly by rigor and recency, then this section of 

the report would have been limited to TFA and academic achievement (literacy and math tutoring).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

47

Appendix B: Annotated Bibliographies

Education

170 Antecol, H., Eren, O., & Ozbeklik, S. (2013). The effect of Teach for America on the distribution of student achievement in 

primary school: Evidence from a randomized experiment. Economics of Education Review, 37, 113–125. 

http://doi.org/10.1016/j.econedurev.2013.08.004

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Teach for America Teach for America AmeriCorps State 

and National

Heather Antecol, Ozkan Eren, Serkan 

Ozbeklik 

Baltimore, Chicago, Los 

Angeles, Houston, New 

Orleans and the 

Mississippi Delta 

This study used secondary data collected from 2001 to 2003 during Mathematica’s Policy Research Incorporated National Evaluation of Teach for 

America (2004). It used fixed effect quantile regression (FEQR) to examine the effects of having a TFA teacher on test scores across the entire 

achievement distribution of primary school students (N = 1710) in disadvantaged neighborhoods. The study generally found that TFA teachers 

neither help nor hurt students in terms of reading test scores. Positive and statistically significant effects of TFA across the math achievement 

distribution were found for the full sample and the effects were fairly uniform. A similar distributional effect of TFA was found within student 

gender, although the FEQR estimates for female students were two to three times larger than for male students. Evidence existed of 

heterogeneity in the effects of TFA for Hispanic and black students and for students taught by novice teachers. Finally, the effect of TFA is 

homogeneous across the math achievement distribution irrespective of certification type. Overall, this study provides further support for the 

strength of findings reported in Decker, Mayer, & Glazerman (2004).

Effect sizes: Reading (mean = 0.02, NS), Math (mean = 2.86, p<0.05).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

48

1357 Miller, S. (2008). 2007-2008 Evaluation of Jumpstart in California. Sacramento, CA: California Jumpstart.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Jumpstart Jumpstart AmeriCorps State 

and National

Shelby Miller California 

This quasi-experimental evaluation attempted to determine if Jumpstart participants show greater gains than comparison group children over 

the program year measuring language, literacy, initiative, and social relational skill development. The sample for this study consists of 891 preschool age children from 10 program sites in California, 58 percent of whom were Hispanic, of which 25 percent spoke only Spanish. Jumpstart 

provides an educational intervention for preschool-age children who are attending community-based, early education and child care programs,

including Head Start and pre-kindergarten classes. AmeriCorps members provided an educational intervention focusing on language, literacy, 

initiative, and social skill acquisition during two-hour long sessions twice a week during the school year in community-based, early education, 

and child care centers. Members spent six additional hours per week in the children’s classrooms assisting teachers or working with children.

This report, on the evaluation of Jumpstart in California, is based on the School Success Checklist obtained at the beginning and end of the school 

year. Results indicated that Jumpstart participant groups made gains on the total School Success Checklist as well as its two subscales in the 

areas of literacy and social-emotional development that were significantly greater than the gains of the non-matched comparison group 

children. Overall, the study provides preliminary evidence for these findings.

Effect sizes (t-tests): Language-literacy subscale (7.02, p<0.000); Initiative-social relations subscale (6.70, p<0.000).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

49

1370 Arcaira, E., Vile, J. D., Reisner, E. R. (2010). Citizen Schools: Achieving High School Graduation: Citizen School's Youth Outcomes in 

Boston. Washington, DC: Policy Studies, Associates.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Citizen School’s 8th

Grade Academy

Citizen Schools AmeriCorps State 

and National

Policy Studies Associates, Inc. Boston, Mass.

This quasi-experimental study assesses the impact of Citizen Schools’ 8

th Grade Academy on the long-term academic outcomes of its participants. 

The study followed 8

th Grade Academy participants from five different school years as they progressed through high school in the Boston Public 

School system. Citizen Schools works to prepare middle-school students for long-term academic, social, career, and civic success by extending 

the student learning day and offering hands-on learning opportunities in areas that include career exposure, high school and college 

preparation, and academic enrichment.

Researchers matched each of the 448 participants with three similar nonparticipants, ranked by similarity. If an alumnus’ most similar match 

left the BPS, they were replaced by the next most similar match in the next year’s analysis. Although demographics varied slightly by class, 

participants were roughly three-quarters African American, and between 18 and 21 percent Hispanic. About half of participants were female. 

Close to 20 percent of participants were enrolled in special education, and three percent enrolled in bilingual education. Primary outcome 

measures were based on data the BPS provided, and included enrollment in a top-tier high school, attendance, suspensions, course grades, 

standardized test scores, on-time promotion, and graduation rates.

Overall, former 8

th Grade Academy participants enrolled and completed all four years of high school in a top-tier school at more than three times 

the rate of matched comparison students. Former participants also had significantly higher attendance rates in high school than did matched 

nonparticipants, with differences ranging from an additional week of school attended in the tenth grade to an additional two and a half weeks 

attended in eleventh grade. There were no statistical differences in suspension rates between former participants and matched nonparticipants. 

Former participants were more likely than matched nonparticipants to pass math in ninth, tenth, and eleventh grade and were more likely to 

earn As and Bs than were their matches in their 9

th- and 10th-grade math courses. Former participants were also more likely to pass the 

mathematics MCAS than the average BPS student. Overall, the study provides moderate evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

50

1471 Melchior, A., & Orr, L. (1995). Evaluation of National and Community Service Programs Final Report: National Evaluation of ServeAmerica (Subtitle B1). Cambridge, MA: Abt Associates.

Intervention name: Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS:

Evaluator or evaluating organization name: Location(s) of the study:

Serve America ACTION AmeriCorps State 

and National

Larry Orr, Abt Associates Inc

Alan Melchior, Brandeis University

Colorado, 

Massachusetts, Ohio, 

and South Carolina

This quasi-experimental study assesses the impact of Serve America programs on young people in terms of increased civic responsibility and 

involvement in service; increased academic performance; and improved personal and social development. The report draws on data from a 

sample of thirteen local Serve America sites. Serve America was established to support the development of school-based service-learning 

strategies that incorporate community service into daily school activities. This is accomplished through the integration of service into academic 

classes and the establishment of service-learning courses and after-school service activities.

The study sites included a mix of school and community-based programs serving middle and high school aged youth in four states: Colorado, 

Massachusetts, Ohio, and South Carolina. The data collected includes participant and comparison group surveys, school record data, program 

observations, and interviews with program staff, students, school administrators, service site representatives, and others over an 18 month 

period (June 1993-December 1994). The report also draws on aggregate data collected from Serve America grantees nationally for the 1993-1994 

program year.

Findings from the study included a positive impact on participants in terms of civic and social attitudes, involvement in service, and schoolrelated behaviors. Among the high school programs, the evaluation found significant positive impacts on seven outcomes: school attendance, 

personal and social responsibility, involvement in volunteer service, hours of volunteer service, likelihood of future service, communications 

skills, and work orientation. Among the middle school programs, the evaluation found significant positive impacts on three measures: school 

attendance, hours of homework, and hours of volunteer service. Eighty-five percent of the high school participants and 66 percent of the middle 

school students indicated that they had learned a skill that would be useful in the future. A substantial percentage of the high school students 

(71 percent) also indicated that they had learned more through their service than through a typical class at school. Ninety-six percent of the high 

school participants and over 75 percent of the middle school youth indicated that they believed the services they performed were helpful to the 

community and the individuals served. Overall, the study provides preliminary evidence for these findings.

Effect sizes (percent difference in scale scores for high school [HS] and middle school [MS] students): Personal and social responsibility (5.3

percent, p<0.01); psychosocial maturity (3.5 percent, NS); engagement in formal helping activities (HS 13.0 percent, p<0.01; MS 3.1 percent, NS); 

number of formal helping activities (HS 59.4 percent, p<0.01; MS 5.2 percent, NS); hours volunteering in past six months (HS: 313.9 percent, 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

51

p<0.01; MS 89.8 percent, p<0.05); engagement in informal helping activities (HS 1.5 percent, NS; MS -4.0 percent, NS); number of informal helping 

activities (HS 3.3 percent, NS; MS -1.1, NS). Days absent (HS -20.3 percent, p<0.05; MS -35.0 percent, p<0.05); school attitudes and engagement (HS 

4.5 percent, NS; -3.0 percent, NS); education aspirations (HS expected direction, NS; MS expected direction, NS). Hours spent on homework (HS 

expected direction, NS; MS expected direction, p<0.01).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

52

1535 Hatchuel Tabernik and Associates (2009). Girls Incorporated of Alameda County AmeriCorps Program Evaluation 2008-09. Berkeley, 

CA.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

GIRLStart Girls Incorporated 

of Alameda County

AmeriCorps State 

and National

Hatchuel Tabernik and Associates Alameda County, CA

This quasi-experimental study assesses the impact of the GIRLStart program on reading ability and school day attendance. The study includes 

data from 668 students at six elementary schools in the Oakland and San Leandro school districts, collected during the 2008-2009 academic year. 

The 17 AmeriCorps members involved in GIRLStart provided direct service to kindergarten-third- graders at school sites in after-school 

programs, as well as academic support and assistance during the school day under the supervision of classroom teachers.

The study includes 131 kindergarten-third grade girls enrolled in GIRLStart and 537 of their female peers. GIRLStart participants were 48 

percent African-American, 29 percent Hispanic, and 18 percent Asian-American; demographic data from their peers was not reported. 

Researchers collected attendance records and standardized test scores from the students’ school districts, and conducted site visits to five of the 

six schools. Reading proficiency was measured with CST-ELA results, available only for the second and third graders, and from district 

benchmark tests, which varied by year and by school district.

CST-ELA results indicate that a greater proportion of participants than non-participants were proficient in reading in the second and third 

grade, although this difference was not statistically significant. Additionally, almost two-thirds of participants sustained reading proficiency 

from the second into the third grade. Moreover, a higher proportion of participants sustained reading proficiency when compared to 

nonparticipants – although again, this difference is not statistically significant. Participants did, however, have significantly higher rates of 

school attendance than nonparticipants. Overall, the study provides preliminary evidence for these findings.

Effect sizes: Percent change in ELA scores: 1st grade = -19.6 (sig); 2nd grade = 3.1 (NS); 3rd grade = 5.9 (sig). 

Reading proficiency: 3rd grade = -15 percent (sig). School attendance, grades 1-3: 89 percent (sig).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

53

1682 Decker, P.T., Mayer, D.P., & Glazerman, S. (2004). The effects of Teach for America on students: Findings from a national evaluation. 

Madison, WI: Institute for Research on Poverty.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Teach for America Teach for America AmeriCorps State 

and National

Paul T. Decker, Daniel P. Mayer, Steven 

Glazerman

Mathematica Policy Research, Inc. 

Nationwide

This randomized controlled trial assessed the effectiveness of TFA teachers on student achievement. The sample consisted of 17 schools with 37 

blocks, 100 classrooms, and nearly 1,800 students. Comparisons of outcomes of students taught by TFA teachers with outcomes of students 

taught by non- TFA, or control, teachers in the same schools and at the same grades occurred. For the analysis, control teachers included all 

teachers in the study who were not TFA corps members at the time of the study or at any time in the past. TFA teachers included all teachers 

who entered the profession through TFA —both current TFA corps members in their first two years of teaching, and alumni (former corps 

members) who were still teaching. The main source of data for this study was a set of achievement tests we administered in the fall (pre-test) 

and the spring (post-test) of the study year (2002-2003), including an abbreviated form of the mathematics and reading subtests of the Iowa Test 

of Basic Skills (ITBS).

TFA teachers had a positive impact on math achievement and no impact on reading achievement. The size of the impact on math scores was 

about 15 percent of a standard deviation, equivalent to about one month of instruction. No impacts on other student outcomes such as 

attendance, promotion, or disciplinary incidents were identified, but TFA teachers were more likely to report problems with student behavior 

than were their peers. Overall, the study provides strong evidence for these findings.

Effect sizes (ANOVA): Math (2.43, p<0.002); reading (0.56, NS); retention in grade (0.84, NS); summer school attendance (0.04, NS); absenteeism 

(0.52, NS); disciplinary incidents (0.04, NS).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

54

2270 Clark, M. A., Chiang, H. S., Silva, T., McConnell, S., Sonnenfeld, K., Erbe, A., & Puma, M. (2013). The Effectiveness of Secondary 

Math Teachers from Teach For America and the Teaching Fellows Programs. NCEE 2013-4015. National Center for Education 

Evaluation and Regional Assistance.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Teach for America Teach for America Some TFA members 

receive AmeriCorps 

education awards.

Melissa A. Clark , Hanley S. Chiang , Tim 

Silva , Sheena McConnell , Kathy Sonnenfeld , 

& Anastasia Erbe, Mathematica Policy 

Research

Michael Puma, Chesapeake Research 

Associates

Nationwide

This randomized controlled trial assesses the effectiveness of TFA secondary math teachers in improving student math achievement. The study 

examined the effectiveness of TFA teachers, comparing secondary math teachers from TFA with other secondary math teachers teaching the 

same math courses in the same schools. The TFA study sample consisted of 4,573 students, 111 classroom matches, 136 math teachers, 45 

schools, and 11 districts in 8 states.

In each participating school, evaluators identified “classroom matches”—two or more classes covering the same middle or high school math 

course at the same level, with at least one class taught by a TFA teacher and at least one class taught by another teacher, referred to as a 

comparison teacher, who did not enter teaching through a highly selective alternative route. In each classroom match, students were randomly 

assigned at the beginning of the school year to a class taught by a TFA teacher or a class taught by a comparison teacher. Student math 

achievement was measured by math assessments administered at the end of the school year in which the students were randomly assigned. For 

students in grades 6 to 8, scores were obtained on state-required assessments. Students in grades 9 to 12 were given end-of-course math 

assessments developed by the Northwest Evaluation Association (NWEA).

On average, students assigned to TFA teachers scored higher (by 0.07 standard deviations) on end-of-year math assessments than students 

assigned to comparison teachers. This difference in math scores was equivalent to an increase in student achievement from the 27th to the 30th 

percentile. This difference also translated into an additional 2.6 months of school for the average student nationwide. Overall, this study 

provides strong evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

55

2269 Clark, M. A., Isenberg, E., Liu, A. Y., Makowsky, L., & Zukiewicz, M. (2015). Impacts of the Teach For America Investing in 

Innovation Scale-Up. Princeton, NJ: Mathematica Policy Research.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Teach for America Teach for America Some TFA teachers 

receive AmeriCorps 

education awards.

Melissa A. Clark, Eric Isenberg, Albert Y. 

Liun, Libby Makaowsky, & Marykate 

Zukiewicz, Mathematica Policy Research

Nationwide

This randomized controlled trial assesses the effectiveness of TFA teachers in improving academic outcomes, following a major expansion effort 

of the TFA program. The final sample included 10 states, 13 school districts and other TFA placement partners, 36 schools, and 156 teachers (66 

TFA and 90 comparison teachers). The sample of TFA teachers was limited to those recruited in the first two years of the scale-up, who were in 

their first or second year of teaching at the time of the study, whereas the comparison teachers included both novice and experienced teachers 

teaching in the same schools and grades as the TFA teachers. In total, 3,724 students were randomly assigned to classes and outcome test score 

data were obtained for 2,153 students.

In the lower elementary grades (pre-kindergarten through grade 2), students were assessed using reading and math assessments from the 

Woodcock-Johnson III achievement test. In the upper elementary grades (3 to 5), in which annual reading and math assessments were required 

by the federal No Child Left Behind Act, evaluators collected state assessment data from district records. Evaluators also collected prior years’

test scores from state assessments when available, along with other student background characteristics. End-of-year test scores of students 

assigned to the TFA teachers and those assigned to the comparison teacher were compared.

On average, the TFA teachers in the sample were as effective as comparison teachers in both reading and math, including both novice and 

traditionally certified teachers. In both subjects, differences in test scores between students assigned to TFA teachers and those assigned to 

comparison teachers were not statistically significant. TFA teachers in lower elementary grades (pre-kindergarten through grade 2) had a 

positive, statistically significant effect on student reading achievement of 0.12 standard deviations, or about 1.3 additional months of learning 

for the average student in these grades nationwide. Overall, this study provides strong evidence for these findings.

Effect sizes: Letter sounds (0.71, p<0.001), rhyming (0.66, p<0.001), letter names (0.40, p<0.01); picture names (0.49, p<0.001); alliteration (0.72, 

p<0.001); rhyming fluency (0.43, p<0.001); letter name recognition (no effect); picture name fluency (0.42, p<0.01); alliteration fluency (no effect).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

56

2365 Jacob, R.T., Armstrong, C., Willard, J.A., (2015). Mobilizing volunteer tutors to improve student literacy: Implementation, impacts, and 

costs of the Reading Partners program. New York, NY: MDRC.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Reading Partners Reading Partners AmeriCorps State 

and National

Robin Tepper Jacob, Institute for Social 

Research, University of Michigan

Catherine Armstrong and Jacklyn Altuna 

Willard, MDRC

California, New York, 

Washington DC

This randomized controlled trial assesses the effectiveness of Reading Partners in improving literacy outcomes. The sample for this study 

consisted of Reading Partners programs located in 19 schools in California, New York, and Washington, DC, in which Reading Partners had been 

in operation for at least one year. AmeriCorps members serve as site coordinators who oversee instruction provided by volunteer tutors (not 

AmeriCorps) members. Tutors provide regular individualized one-to-one reading instruction utilizing a structured curriculum. AmeriCorps 

members also serve as Outreach Coordinators, tasked with recruiting volunteers to staff each program site.

Within each of the study schools and within grade groups, students in second through fifth grade who needed assistance in reading were 

randomly assigned to the Reading Partners program or to an “as-is” control condition. A total of 1,265 students were assigned, and the final 

sample included 1,166 students. The treatment and control group demonstrated baseline equivalence and did not differ in attrition. Participants 

in the study were in low-income communities and included a high percentage of minority (Hispanic) students. The study team administered 

three reading assessments measuring reading comprehension, fluency, and the ability to read sight-words efficiently.

Reading Partners had a positive and statistically significant impact on all three measures of student reading proficiency, with effect-size impacts 

of 0.10 on reading comprehension scores, 0.09 on reading fluency, and 0.11 on sight-word reading. An examination of growth between the 

beginning and end of the year on two of these three assessments shows that both groups scored higher in reading comprehension and fluency 

at the end of the year than they did at the beginning of the year, but that growth on these two assessments was greater for the program group 

than for the control group. On the sight-word reading test, there was no growth among the control group but positive growth for the Reading

Partners group. Overall, this study provides strong evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

57

2366 Markovitz, C., Hernandez, M., Hedberg, E., & Silberglitt, B. (2014). Impact evaluation of the Minnesota Reading Corps K-3 program. 

Chicago, IL: NORC at the University of Chicago.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Minnesota Reading 

Corps K-3 Program

ServeMinnesota 

Action Network

AmeriCorps State 

and National

Carrie E. Markovitz, Marc W. Hernandez, Eric 

C. Hedberg, Benjamin Silberglitt

Minnesota

This quasi-experimental study with groups formed by matching assesses the impact of the Minnesota Reading Corps program on student literacy 

outcomes. The study includes data from 23 participating elementary schools in Minnesota. Volunteers with the MRC program individually 

tutor approximately 15-18 kindergarten-third grade students daily for 20 minutes each, aiming to raise students’ literacy levels so that they are 

on track to meet the next program-specified benchmark.

In the fall, winter, and spring of each school year, AmeriCorps members collected general outcome measure data using AIMSweb literacy 

assessments. Eligible students in each grade within a school were matched based on their fall benchmark score. Students within pairs were then 

randomly assigned to either the program or control condition prior to the start of tutoring. After removing pairs that did not complete the 

treatment, the final sample included 1,341 students. Within each grade level, students were at least 26 percent Dual Language Learners, 60 

percent non-white, and 71 percent eligible for free or reduced price lunch.

Kindergarten, first, and third grade students who received MRC tutoring achieved significantly higher literacy assessment scores than students 

who did not. The magnitude of MRC tutoring effects differed by grade, with the largest effects found among kindergarteners and first graders 

(effect sizes of 1.06 and 0.37, respectively), and the smallest effects among third grade students (effect size of 0.10). Significant effects were not 

found for second grade students. In later grades (second and third), when students begin the more complex task of reading connected text, the 

MRC program appears to take longer than a single semester to produce significant improvements in student literacy. These results are 

significant across multiple racial groups and in multiple school settings, using AmeriCorps members with varied backgrounds. Overall, the 

study provides strong evidence for these findings.

Effect sizes: Literacy skills: kindergarten (1.06, p<0.001), 1st grade (0.37, p<0.001); 2nd grade (0.08, NS); 3rd grade (0.10, p<0.01).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

58

2369 Markovitz, C., Hernandez, M., Hedberg, E., & Silberglitt, B. (2014). Outcome evaluation of the Minnesota Reading Corps PreK 

program. Chicago, IL: NORC at the University of Chicago.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Minnesota Reading 

Corps Pre-K

ServeMinnesota 

Action Network

AmeriCorps State 

and National

NORC at the University of Chicago Minnesota

This quasi-experimental evaluation assessed the whether children in the Minnesota Reading Corps Pre-K program exhibited greater gains in 

literacy than children in a matched comparison group. The study was performed in 25 MRC Pre-K sites in Minnesota and 25 matched sites. The 

MRC Pre-K program recruits, trains, places, and monitors AmeriCorps members, who implement evidence-based literacy interventions for atrisk preschool children, using a Response-to-Intervention framework.

The final sample included 1534 students, approximately 52 percent of whom were white, 11 percent black, 7 percent Asian, 8 percent Hispanic, 

and 9 percent other. A small percentage was Dual Language Learners. Participants in the MRC Pre-K program had significantly higher scores 

than children in matched comparison sites on five IDGDI outcomes measures for 4- and 5-year old students: 1) letter sound fluency, 2) rhyming 

fluency, 3) letter name fluency, 4) picture name fluency, and 5) alliteration fluency. Effect sizes ranged from .40 to .72. Despite a small sample 

size, significant effects were also found for 3-year old participants in measures of rhyming fluency and picture name fluency. Overall, the study 

provides moderate evidence for these findings.

Effect sizes: Letter sounds (0.71, p<0.001), rhyming (0.66, p<0.001), letter names (0.40, p<0.01); picture names (0.49, p<0.001); alliteration (0.72, 

p<0.001); rhyming fluency (0.43, p<0.001); letter name recognition (no effect); picture name fluency (0.42, p<0.01); alliteration fluency (no effect).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

59

2489 ICF International (2010). Communities in Schools national evaluation: Five year summary report. Fairfax, VA.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Communities in 

Schools

Communities in 

Schools

AmeriCorps State 

and National

ICF International Texas, Florida, Georgia, 

Pennsylvania, Michigan, 

Washington, North 

Carolina17

This evaluation included a multi-level and multi-level approach that included multiple randomized control trials and a quasi-experimental 

study using propensity score matching. Though the three RCTs did not include sites with AmeriCorps staff, the quasi-experimental study, 

which included 602 CIS schools and 602 matched comparison schools, did incorporate AmeriCorps staff.

Results from the QED indicated that CIS students experienced small but consistent improvements in performance on state-mandated 

assessments, with stronger gains found in math than reading/English. Program effects were stronger when programs were implemented with a 

high level of fidelity. The QED study also showed small improvements in attendance in CIS schools relative to matched comparison schools.

The RCT studies found large positive improvements in retention (+ ES of .7 in one site), academics (+ ES of .26-.55) and attendance (+ES of .45-

.72.).

 

17 The locations reported were the sites for the QED study, which included programs with AmeriCorps staff. The RCTs, which did not includes 

AmeriCorps staff, were conducted in Jacksonville, FL, Austin, TX, and Wichita, KS.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

60

2442 Evaluation and Assessment Solutions for Education, LLC (2013). National College Advising Corps 2011-2012 Evaluation Report. 

Module 1: The NCAC Difference. Chapel Hill, NC: The University of North Carolina at Chapel Hill.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

National College 

Advising Corps

National College 

Advising Corps

Social Innovation 

Fund and 

AmeriCorps State 

and National

Bettinger, Antonio L. Antonio, Brent Evans, 

Jesse Foster, Brian Holzman, Hoori Satikian, 

Stanford University

Homg, EASE – Evaluation and Assessment 

Solutions for Education 

Nationwide

This quasi-experimental study examines the differences between post-secondary preparation in students who visited National College Advising 

Corps (NCAC) advisers and students who did not, according to a self-reported survey. Of 72,883 students nationwide were invited to participate 

in the survey these, 30,546 (42 percent) responded. These responses represent 168 schools across nine states and 13 partner institutions. Slightly 

more than half of the respondents were female, and 76 percent were first-generation college goers. Overall, 72 percent of the sample came from 

an underrepresented minority group (i.e., Black, Hispanic, Other, or Multicultural), including 38 percent identifying as Hispanic. The survey 

primarily targeted seniors who were making college decisions. Students were surveyed in April and May of 2012. The survey asked students 

about their college plans, to reflect on their academic preparation throughout high school, and about what college-going information they 

received and from whom they received it.

Results indicate seniors who have compared to those who have not met with the NCAC adviser at their school, students who have met with the 

NCAC adviser were: 22 percent more likely to aspire to attend college early in their education careers; 107 percent more likely to take three or 

more ACT/SAT prep courses; 198 percent more likely to attend financial aid workshops; 54 percent more likely to visit colleges three or more 

times; 40 percent more likely to take the ACT/SAT; 62 percent more likely to submit the FAFSA; 99 percent more likely to use a fee waiver for 

college applications; 42 percent more likely to apply to a college/university; 73 percent more likely to apply to a four-year institution of higher 

education; 67 percent more likely to be accepted to a college/university; 84 percent more likely to be accepted to a four-year institution of higher 

education; and 31 percent more likely to be committed to attending college in the fall (as indicated by having submitted a deposit to a 

college/university). There were greater effects for first-generation college students who met with an NCAC advisor than others. Since this study 

used non-equivalent comparisons, causality cannot be determined, as student characteristics likely contributed to differences in outcomes. 

Overall, this study provides preliminary evidence for these findings.

Effect sizes (percent more likely than comparison group to do the following): Aspire to attend college early in their education careers = 22 percent; 

take three or more ACT/SAT prep courses = 107 percent; attend financial aid workshops = 198 percent; visit colleges three or more times = 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

61

54percent;take the ACT/SAT = 40 percent; submit the FAFSA = 62 percent; use a fee waiver for college applications = 99 percent; apply to a 

college/university = 42 percent; apply to a four-year institution of higher education = 73 percent; be accepted to a college/university = 67 percent; 

be accepted to a four-year institution of higher education = 84 percent; be committed to attending college in the fall = 31 percent.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

62

2492 Center for Social Development (2009). Evaluation of Experience Corps: Student Reading Outcomes. St. Louis, MO: Washington 

University in St. Louis.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Experience Corps Experience Corps AmeriCorps State 

and National

Center for Social Development at Washington 

University Saint Louis’s Brown School of 

Social Work

Boston, New York City, 

Port Author Texas

This randomized controlled trial assesses the effectiveness of the Experience Corps (EC) program in improving student literacy outcomes. 

Twenty-three schools in Boston, New York City, and Port Arthur, Texas, participated in the study. Experience Corps members provide one-toone literacy tutoring using a structured curriculum to first through third grade students. Students referred by teachers as needing assistance in 

reading at the beginning of the school year were randomly assigned to the EC program or a control condition.

The total sample included 825 students. A slight majority of students were African American, and a large majority qualified for free or reduced 

lunch. Data for the study came from standardized reading tests: (a) the Woodcock Johnson word attack subscale (WJ-WA), the Woodcock 

Johnson passage comprehension subscale (WJ-PC), and (b) the Peabody Picture Vocabulary test (PPVT-III). At the beginning and end of the 

academic year, teachers completed assessments of grade-specific reading skills and classroom behavior. Analysis of pretest data showed that 

the EC students and control groups were equivalent on all measured characteristics. Referred students were poor readers based on the WJ-PC 

measure, which found that half of the students referred to EC perform as low as or lower than 84 percent of the students their age nationwide, 

and 12 percent score worse than 97 percent of the population.

About half of the EC students received 30 to 49 sessions, and the mean number of sessions was 45. Three-quarters of the students received over 

35 sessions, which represents about one session a week throughout the program period. Students in the EC group made over 60 percent more 

progress in word attack and passage comprehension and 40 percent more progress on grade-specific reading skills. Improvements compared to 

control group members were statistically significant, with ES.13 for passage comprehension, (p=.04) and ES.16 for grade specific reading 

(p=.004). The group difference for word attack was marginally significant, with ES.10 (p=.07). Overall, the study provides strong evidence for 

these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

63

2526 Avery, Christopher (2013). Evaluation of the College Possible Program: Results from a Randomized Controlled Trial. Cambridge, MA: 

National Bureau of Economic Research.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

College Possible College Possible AmeriCorps State 

and National

Christopher Avery

Harvard Kennedy School of Government

Minneapolis and St. Paul 

Minnesota

This randomized controlled trial assesses the effectiveness of the College Possible program in increasing students’ applications and enrollments 

to post-secondary institutions. The College Possible program provides two years of college preparatory work for high school juniors and seniors, 

including SAT and ACT test preparation services, college admission and financial aid consulting, and guidance in the transition to college. The 

study included 239 students from eight high schools.

Students applied to participate in the program in their sophomore year. Since the program had only 800 places and 900 qualified applicants, the 

last 101 were admitted to the program on a randomized group-by-group basis designed to admit a predetermined number from each high 

school. The remaining students were placed on a wait list, and 33 were admitted from this waitlist in a separate randomization procedure to fill 

new spots. There were slight differences between the first and second treatment groups in terms of household income and program 

participation. The majority of the overall sample was female (60 percent) and Hmong (60 percent). College Possible attendees were not 

significantly more likely to enroll in college, but were 15percent more likely to enroll in four-year colleges than control group members. Overall, 

this study provides strong evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

64

843 Wilson, J. (2003). An Investigation of the America-Reads Mississippi Program: What Works and What Doesn’t Work. Paper presented at 

the Mid-South Educational Research Association, Biloxi, MS. Retrieved from ERIC database.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

America Reads 

Mississippi Program

America Reads 

Challenge (ARC)

Utilized 

AmeriCorps 

Volunteers

Jennifer Wilson, Delta State University Mississippi

This outcome evaluation assesses the impact of America Reads-Mississippi on student reading achievement, and examines the attitudes of 

students, volunteer tutors, regional coordinators and assistant coordinators, and site supervisors (n=13). In 1999, 37 schools in 25 school districts 

participated in this program, sponsored by three regional centers in Mississippi. The ARC program, part of AmeriCorps, has the primary goal of 

ensuring that all children can read proficiently by the end of third grade. In Mississippi, the ARC program is referred to as the America ReadsMississippi (ARM) program.

Data were collected from ARM archival documents, surveys administered by ARM officials, and interviews with key informants such as the 

program’s visionaries, state director, regional coordinators, assistant regional coordinators, and site supervisors. Program officials used GatesMacGinitie Reading Tests, 4th edition, (GMRT) to assess students’ reading performance.

Findings show that principals did not have time to serve as site supervisors, and program implementation varied among program sites. 

Standardized achievement test scores improved pre to post for students involved in ARM, and reading performance improved. Students in the 

program enjoyed reading, and tutors appear to be beneficial in helping students’ interest and proficiency in reading. Tutors themselves had 

positive attitudes about the program, although they thought more training would be useful. The report recommends that schools with certified 

reading specialists on staff should designate these individuals to assume the role of site supervisor. In schools that do not have certified reading 

specialists on staff, certified teachers with training in reading should assume this role. For those schools that do not have structured reading 

programs and specific guidelines for tutors to follow, the report recommends that a paradigm or model of a research-based volunteer tutoring 

program be provided to site supervisors. Overall, the study provides preliminary evidence for these findings.

Effect sizes: During Year 2 of program implementation, reading comprehension as measured by Gates-MacGinitie Reading Tests improved 13 

points, total reading performance improved 27 points.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

65

Economic Opportunity

2144 Lattimore, J. D. (2014). Rebuilding social organization in low-income neighborhoods: The influence of Habitat for Humanity (Doctoral 

dissertation, Clemson University).

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Habitat for Humanity Habitat for 

Humanity

AmeriCorps State 

and National

John Daniel Lattimore Des Moines, 

Indianapolis, Louisville, 

Providence, San Antonio

This quasi-experimental study assesses the impact of the presence of Habitat for Humanity homes on neighborhood social organization in lowincome neighborhoods. This study focuses on five large U.S. cities spread across the country. This investigation hypothesizes that Habitat for

Humanity families are more motivated to better their lives than their neighbors because of Habitat’s selection criteria and because they have 

completed the process of becoming a Habitat homeowner. The theory also suggests that Habitat homeowners have a positive effect on their 

neighbors, and their neighborhood. This effect is measured through components of social organization. The dissertation takes advantage of the 

Making Connections survey sponsored by the Annie E. Casey Foundation as well as qualitative interviews, neighborhood observations and GIS 

analysis in order to determine the effect Habitat homeowners have on their neighborhoods.

The data collected were used to estimate the effect of the presence of Habitat for Humanity’s homes, either scattered or in clusters, on several 

variables that are indicators for sense of community, positive identification with the neighborhood, and explicit norms against aberrant 

behavior. Six variables related to social organization were analyzed with ANOVA procedures, and significant differences were found between 

people living in blocks with a Habitat for Humanity house and those living in blocks without a Habitat for Humanity house in measures of 

cohesion. Additional outcome measures determined that Habitat homeowners feel much better about raising their children in their 

neighborhoods and generally have a more positive outlook on the future of their neighborhoods. Overall, this study provides preliminary 

evidence for these findings.

Effect sizes (ANOVA): Neighbor activism (expected direction, NS), cohesion (not in expected direction, p<0.1), organizations/volunteerism 

(expected direction, NS), safety (expected direction, NS), physical/social disorder (expected direction, NS), services and amenities (expected 

direction, NS), police (not in expected direction, NS), informal social control (expected direction, NS).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

66

2293 Roder, A., & Elliott, M. (2014). Sustained gains: Year Up’s continued impact on young adults’ earnings. New York: Economic Mobility 

Corporation.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Year Up Year Up AmeriCorps State 

and National

Economic Mobility Corporation Program sites in 

Providence, Rhode 

Island; New York City; 

and Washington, DC.

Beginning in 2007 and running through September 2011, the Economic Mobility Corporation in partnership with the RAND Corporation’s 

Survey Research Group conducted an impact study with Year Up program participants three to four years after program participation. The 

evaluation sought to measure the impact of Year Up on program participants’ employment rates, average number of hours worked, types of 

jobs, and annual earnings using a randomized control trial (RCT) study design. The target population consisted of young adults ages 18-24; 54 

percent male, 46 percent female; 52 percent African American, 34 percent Latino; all with a high school diploma or GED. Researchers collected 

data from 143 program alumni (102 in the treatment group and 41 in the control group). The evaluation was successfully implemented with 

sufficient participation, randomization, and measurement implementation.

Over the three years after the program, Year Up participants earned about $13,000 more than members of the control group. Participants’

earnings were 32 percent greater than those of the control group. These earning gains were driven primarily by the higher wages paid to Year 

Up participants. Year Up participants who graduated and secured jobs in either of the program’s two target occupations, information 

technology and financial operations, earned the highest hourly wages and annual incomes.

Effect sizes: All estimates for outcome measures were in the expected direction, except college attendance at end of study period. The following 

results are statistically significant: had tuition assistance available through job (29.7 percent, p<0.1); worked in targeted occupations at some point 

during follow-up (54.1 percent, p<0.01); received financial assistance for college (42.9 percent, p<0.05); interested in attending college in future 

(27.3 percent, p<0.05); attending another training program (-32.2 percent, p<0.05).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

67

2384 VeraWorks. (2007). The Effects of AmeriCorps on Participant Readiness for Education and Employment Success. Waynesboro, PA.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Washington 

AmeriCorps

The Washington 

Commission for 

National and 

Community Service

AmeriCorps State, 

AmeriCorps VISTA

VeraWorks Washington

This outcome evaluations assesses the effect of Washington AmeriCorps programs on participants’ readiness for educational and employment 

success. The study targets Washington AmeriCorps members who began their first year of service in the fall of 2005. Researchers asked 

AmeriCorps program staff to administer pre- and post- service surveys during the first and last month of each member’s service. Of the 598 

members asked to participate in the study, 399 completed both surveys, a 67 percent response rate. Participants were largely white (86 percent), 

female (76 percent), and in their 20s (78 percent); eighty-five percent had completed at least some college, and 50 percent already had a 

Bachelor’s degree. Survey data were analyzed using the same methods described in a national study of AmeriCorps members conducted by 

Abt Associates Inc. (document 1418), allowing researchers to compare the effectiveness of Washington’s AmeriCorps program to AmeriCorps 

members as a whole.

Survey data indicate that participation in Washington AmeriCorps increases members’ confidence in their ability to obtain an education and 

personal responsibility for educational success (increases of 0.10 and 0.08 on 1-5 scale, respectively). These were greater increases than the 

national control group (which showed no significant increase in these constructs), indicating that Washington AmeriCorps was the likely causal 

effect for these changes. Washington AmeriCorps also increased the work skills measured in this study (an increase of 0.21 on a 1–3 scale) more 

than the national comparison group, although the national comparison group also showed a statistically significant positive increase in this 

construct. Although there was no significant improvement in participants’ responsibility for employment success across all respondents, there

was a significant effect among those without a four-year college degree. Overall, the study provides preliminary evidence for these findings.

Effect sizes: Confidence in ability to obtain an education (scale 1-5): pre-test mean = 1.54, post-test mean = 1.64, p<0.10. Responsibility for 

educational success (scale 1-5): pre-test mean = 3.30, post-test mean = 3.38, p<0.05. Basic work skills (scale 1-3): pre-test mean = 2.43, post-test mean 

= 2.64, p<0.05. Responsibility for employment success (scale 1-5): pre-test mean = 3.13, post-test mean = 3.18, NS.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

68

Environmental Stewardship

1489 Our City Forest. (2009). Our City Forest Evaluation Report. San Jose, CA.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Our City Forest 

AmeriCorps

Our City Forest AmeriCorps State 

and National

Project Staff San Jose, California

The Our City Forest AmeriCorps program is implemented in San Jose, California. The program aims to create a thriving “urban forest” using 

AmeriCorps to train community residents to plant and care for trees within the City of San Jose. The internal evaluation was conducted by

program staff during 2007-2009 to compare the survival and thriving rates for random samples of 200 OCF and 204 non-OCF trees. Program 

staff gave each tree a composite score by combining separate rating scales measuring foliage density, color, and size; physical damage; pest and 

disease; and proper structure of trunk and limbs.

The main evaluation question was whether the skills training and information provided to residents through Our City Forest increased tree 

health. The study used a comparison group design. Program staff for the comparison group randomly selected trees not planted and cared for 

by OCF -trained volunteers; the trees in the treatment group had been planted and cared for by OCF -trained volunteers. Trees in the 

comparison group had been planted two to four years earlier than OCF trees. Data were not available for non- OCF trees planted at the same 

time as OCF trees (2007-2008). The study detected statistically significant positive results for trees planted and cared for by OCF volunteers. The 

program used the results to identify benefits of program service that were not previously captured through data collection.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

69

1346 Arroyo Associates Inc. (2009). AmeriCorps Watershed Stewards Project Evaluation Study Final Report. Pasadena, CA.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

AmeriCorps 

Watershed Steward 

Project (WSP)

AmeriCorps 

Watershed Steward 

Project (WSP)

AmeriCorps State 

and National

Arroyo Associates, Inc Northern California

This outcome evaluation assesses the effectiveness of the AmeriCorps Watershed Steward Project (WSP) in terms of member development, 

watershed assessment, watershed outreach, and volunteer recruitment. The study includes data from all 20 of WSP’s Northern California sites. 

WSP AmeriCorps members are involved in a wide variety of watershed restoration, assessment and conservation projects, teaching watershed 

and fisheries-based curriculum to K-12 students, participating in myriad community outreach events, and receiving high quality training in 

natural resources and professional development topics.

The experiences of WSP participants and their mentors were assessed using a variety of techniques, including personal and small-group 

interviews, field observations, and three online surveys. More objective measures of program implementation/effect (e.g., the number of hours 

spend on watershed assessment) were tracked for the entire program each year. The evaluation of the AmeriCorps Watershed Stewards Project

found that the WSP has consistently met or exceeded its performance targets that were established to meet the organization’s mission to 

“conserve, restore, and sustain anadromous watersheds for future generations by linking education with high-quality scientific practices.”

Researchers also found that the WSP is a well-designed and efficiently managed organization. Overall, the study provides preliminary evidence 

for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

70

Disaster Services

1375 Tenenbaum, E., Markovitz, C., & Strang, W. (2003). AmeriCorps*NCCC’s Role in Homeland Security. Merced, CA: Westat.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

AmeriCorps NCCC AmeriCorps NCCC NCCC Westat National, along with 

case studies in MA and 

CA

This study, conducted by Westat in 2002, consisted of two parts. In the first part, researchers identified existing and potential relationships of 

key homeland security-related agencies and organizations to assess the potential for AmeriCorps NCCC to contribute to their missions. In the

second part, researchers also conducted two case studies of NCCC projects operating in Pittsfield, Massachusetts and Berkeley, California to 

gain information about their activities and effectiveness. The study sought to answer two questions: (1) What is the nature and effectiveness of 

NCCC teams’ activities in carrying out homeland security-related projects? (2) What roles can NCCC teams successful play in helping 

communities achieve homeland security objectives? Researchers gathered information from agency documents and conducted site visits and 

key informant interviews.

The study authors concluded that NCCC can be most effective at supplying well-trained, capable teams for concentrated periods of time to 

assist communities with homeland security-related efforts at the local level. This includes serving in administrative roles, helping form 

partnerships, training local leaders and other people, conducting community surveys, preparing and disseminating materials, teaching 

curriculum units in schools, and mitigating potential structural problems. The authors make the following recommendations: (1) set-up model 

NCCC projects and document their planning and implementation to serve as models for other communities; (2) create a homeland security / 

disaster preparedness track within NCCC; (3) recruit high-profile sponsors that have strong networks with organizations in their area to 

collaborate with NCCC teams; and (4) create and disseminate targeted marketing materials to potential sponsors publicizing NCCC’s emphasis 

on homeland security and disaster preparedness.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

71

Healthy Futures

1805 RTI International. (2003). Final Report of the Senior Companion Quality of Care Evaluation. Research Triangle Park, NC.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Senior Companions 

Program

Senior Corps Senior Corps RTI International Nationwide

This quasi-experimental study evaluates the effect of Senior Companion programs on the health and quality of life of their clients, and 

compares these results to the effects of other volunteer programs. The study includes data from over 1500 seniors in three categories: SCP 

Clients, individuals on the SCP wait list, and clients of other volunteer agencies similar to the SCP. Senior Companions is a national program 

that serves frail, isolated, or disabled adults. Volunteers serve between 15 and 40 hours a week, providing transportation, companionship, and 

basic household assistance.

Clients received a baseline phone interview when they were first matched with a Companion, with follow-ups conducted three and nine 

months later. Client outcomes included life satisfaction, depressive symptoms, unmet need for services, health status, functional status, social 

functioning, and satisfaction with services. Survey respondents were 86.5 percent female and 13.5 percent male, with a mean age of 80.7 years 

old. 84 percent were living alone, and many reported serious health problems: 24 percent were diabetic, 22 percent had suffered a stroke, and 50 

percent had heart disease, while only 11 percent self-reported excellent or very good health.

Significant findings included:

 No significant difference between SCP and waiting-list clients in reported current health status by nine-month follow-up.

 No differences between SCP and other agency clients on current health status at three- and nine-month follow-up.

 At three-month follow-up, there were no differences between client groups in the extent to which clients were limited in social activities 

during the past month due either to physical or emotional problems. However, at nine-month follow-up, other agency clients were 

somewhat less limited than SCP clients when examining the same two outcomes.

 Waiting list clients reported having a 7 percent lower functional status score relative to SCP clients at three-month follow-up; however, 

the difference was not significant by nine-month follow-up.

 No differences were reported in clients’ health status compared to that of one year ago at the time of the three-month follow-up survey. 

However, at nine-month follow-up, both waiting list and other agency clients reported that their health status had declined somewhat 

relative to SCP clients.

Overall, the study provides moderate evidence for these findings.

Effect sizes: Physical health status (no direction, NS); functional status-IADLs (no direction, NS); functional status-ADLs (no direction, NS); life 

satisfaction scale (no direction, NS); self-reported depression scale (no direction, NS); social well-being (no direction, NS).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

72

1373 Tierney, J. P., Grossman, J. B., & Resch, N. L. (1995). Making a difference: An impact study of Big Brothers Big Sisters. Philadelphia, 

PA: Public/Private Ventures.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Big Brothers, Big 

Sisters (BBBS)

Big Brothers, Big 

Sisters (BBBS)

AmeriCorps Joseph P. Tierny, Jean Baldwin Grossman, 

Nancy L. Resch, Public/Private Ventures

Philadelphia; Rochester, 

New York; Minneapolis; 

Columbus, Ohio; 

Wichita, Kansas; 

Houston; San Antonio; 

and Phoenix

This randomized control trial assesses the impact of Big Brothers, Big Sisters programs on the attitudes, perceptions and behaviors of at-risk 

young people. The sample for this study consists of 1,138 at-risk youth who applied to BBBS programs located in eight cities across the nation. 

BBBS operates mentoring programs that recruit and carefully screen volunteers for one-to-one matches with youth from single-parent 

households.

Students were interviewed at baseline and 18 months later. Students assigned to the treatment group met with their mentor three times a month 

for an average of 12 months, each meeting lasting around 4 hours. Out of the original 1,138 randomized youth, 959 youth completed both 

baseline and follow-up interviews. Participants were 62 percent male and 38 percent female, with a median age of 12 years old at baseline. 40 

percent were living in homes receiving public assistance. A substantial number of youth had experienced disruptive personal circumstances: 40 

percent lived in families with a history of substance abuse, 28 percent in families with a history of domestic violence, and 27 percent were 

themselves the victims of emotional, physical or sexual abuse.

Little Brother and Little Sisters were 46 percent less likely than controls to initiate drug use during the study period. An even stronger effect 

was found for minority Little Brothers and Little Sisters, who were 70 percent less likely to initiate drug use than other similar minority youth. 

Little Brothers and Little Sisters were also 27 percent less likely than controls to initiate alcohol use during the study period. The quality of 

relationships with parents was better for Little Brothers and Little Sisters than for controls at the end of the study period, due primarily to a 

higher level of trust in the parent. Likewise, there were improvements in Little Brothers’ and Little Sisters’ relationships with their peers relative 

to their control counterparts. No statistically significant improvements were found in self-concept, or in the number of social and cultural 

activities in which Little Brothers and Little Sisters participated. Overall, the study provides strong evidence for these findings.

Effect sizes: Initiating drug use (expected direction, p<0.05); initiating alcohol use (expected direction, p<0.01); number of times hit someone, 

(expected direction, p<0.05); number of times stole something (expected direction, NS); number of times damaged property (expected direction, 
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

73

NS); scholastic competence (expected direction, p<0.01; GPA (expected direction, p<0.10); number of times skipped class (expected direction, 

p<0.05); number of times skipped a day of school (expected direction, p<0.01); weekly hours of homework (expected direction, NS); weekly hours 

spent reading (expected direction, NS), school value scale (expected direction, NS); summary parental relationship measure (expected direction, 

p<0.05); trust (expected direction, p<0.05); communication (expected direction, NS); anger and alienation (expected direction, NS); number of 

times lied to parent (expected direction, p<0.05); intimacy in communication (expected direction, NS); instrumental support (expected direction, 

NS); emotional support (expected direction, p<0.10); conflict (expected direction, NS); global self-worth (expected direction, NS); social acceptance 

(expected direction, NS); self-confidence (expected direction, NS); total weekly hours spent in social and cultural activities (expected direction, 

NS); total attended social and cultural events (no direction, NS).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

74

838 Stipelman, C., Dinkins, C. P., Pruhs, A., Serr, R., & Young, P. (2015). AmeriCorps members increase enrollment in 

Medicaid/CHIP and preventive care utilization at a community health care center. Journal of Health Care for the Poor and 

Underserved, 25(2), 890–900. http://doi.org/10.1353/hpu.2014.0089

Intervention name Operating

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

AmeriCorps and the 

Medically 

Underserved in Utah

AmeriCorps and 

the Medically 

Underserved in 

Utah

AmeriCorps State 

and National

Faculty of the University of Utah department 

of pediatrics and staff members of the 

Association for Utah Community Health

Utah

Study participants consisted of samples of families with uninsured children presenting for health care services to two community health center 

clinics in urban Salt Lake City between July and October of 2009. The target population consisted of low-income families living in urban Salt 

Lake City. The intervention clinic utilized case management by AmeriCorps members while the control clinic did not. Data from administrative 

records were combined with data from follow-up phone interviews conducted with parents three months after presenting at clinics. The 

primary outcome measure was enrollment of the child in Medicaid or CHIP three months following initiation of the intervention. Secondary 

outcomes included the reasons for non-enrollment and the number of acute and primary care appointments completed in the six months 

following case determination by the state.

The main target outcome was increased enrollment in Medicaid/CHIP and preventive care utilization. The main evaluation questions sought to: 

(1) determine whether case management by AmeriCorps Members increased enrollment of children in Medicaid/CHIP at a federally qualified 

community health center; (2) identify factors associated with non-enrollment; and (3) compare health care utilization by enrolled and nonenrolled children. The evaluation design relied on convenience samples of treatment and comparison groups determined by site (i.e., program 

versus non-program sites).

There were 54 families with 107 children in the treatment group, and 43 families with 96 children in the control group. Treatment and control 

groups were statistically equivalent in income, number of children, and combined parental education. The treatment group contained a greater 

proportion of younger children than the control group. The authors note that younger children have less complicated eligibility criteria, which 

may account in part for the higher enrollment rates observed for the treatment group. Treatment group children were much more likely to 

successfully enroll in Medicaid/CHIP. There were no differences in health care utilization for enrolled children in the treatment and control 

groups.

Effect size: Enrollment of the child in Medicaid or CHIP three months following initiation of the intervention (expected, direction, p <0.001).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

75

1351 Camphill Association of North America (2010). Providing Quality of Life through Intentional Community Living: Outcomes 

Evaluation for the Camphill AmeriCorps Education Award Program of the Camphill Association of North America. Hardwick, VT.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Camphill 

AmeriCorps Program

Camphill 

Association of 

North America

AmeriCorps State 

and National

Jan Christopher Goeschel,

Diedra Heitzman

California, New York, 

Pennsylvania, Minnesota

This evaluation report assesses impact of Camphill Communities’ AmeriCorps programs on the quality of life and quality of service experienced by 

Camphill’s community members. The study was implemented at the five existing Camphill Communities in the U.S., spread across four states. 

Camphill communities provide an extended-family living model for adults with developmental disabilities, allowing them to experience 

genuine social inclusion while leading a life of meaningful and valued work.

The study included three main components: a quality of service (QOS) survey of the expected and actual contributions of AmeriCorps 

members, administered to the members and in-house caregivers (approximately 20-30 of each) before and after their term of service; an 

organizational capacities survey given to administrators during the same two time periods (approximately 20-30 administrators); and an 

evaluation of the quality of life (QOL) of community members, administered to approximately 60 participants (or their caregivers, if they were 

unable to complete it themselves). Around 30 community members who received the QOL evaluation were also asked to complete a shortened 

QOS survey assessing the contributions of each AmeriCorps member.

The results of the QOL study suggest that overall the quality of life individuals with disabilities experience in American Camphill communities 

is of an adequate to excellent standard. Responses from all participants in the evaluation indicate that AmeriCorps participants make their 

greatest contributions in the creation of a supportive social fabric. They also contribute to the creation of shared expanded-family homes that 

are experienced as safe, accessible and inviting, with safeguards for private space and shared ownership of common spaces. AmeriCorps 

participants provide healthy and nutritious meals and support healthy, active lifestyles. Through their personal relationships, they give 

informal support for social, emotional and physical wellbeing. AmeriCorps participants considered physical care a significant aspect of their 

work, though it was considered less important by other community members. AmeriCorps participants are seen as making less significant 

contributions in areas that require greater professional skill, roles of administrative responsibility, and roles requiring greater experience and 

commitment to the core principles of Camphill. Overall, the study provides preliminary evidence for these findings.

Effect sizes: Scales range from +10 to -10. Means are reported for verbal [v] and non-verbal [nv] respondents. Physical well-being (7.329 [v], 6.72 

[nv]); psychological well-being (7.24 [v], 0.93 [nv]); spiritual well-being (6.07 [v], 3.53 [nv]); physical belonging (7.30 [v], 6.88 [nv]); social belonging 

(6.27 [v], 5.21 [nv]); community belonging (6.20 [v], 5.16 [nv]); practical becoming (6.10 [v], 4.95 [nv]); leisure becoming (6.42 [v], 4.73 [nv]); growth 

becoming (4.60 [v], 3.65 [nv]).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

76

1360 LaRowe, T. L., Bontrager Yoder, A. B., Knitter, A., Liebhart, J. L., & Schoeller, D. (2011.). Wisconsin Farm to School: One Year 

Evaluation Report. Madison, WI: Department of Family Medicine, University of Wisconsin.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Wisconsin Farm to 

School

Wisconsin Farm to 

School

AmeriCorps University of Wisconsin, Department of 

Family Medicine

Wisconsin (statewide 

program)

The study focused on obesity prevention through promotion of knowledge and attitude changes towards food and nutrition, and through 

changes in behavior (i.e., greater consumption of fruits and vegetables) by increasing the availability of these foods in school lunch programs. 

Baseline data collection took place in Fall 2010 and follow-up data collection occurred in Spring 2011. Researchers used measures and tools 

adapted and modified from the Farm to School Evaluation Toolkit developed by the Center for Health Promotion and Disease Prevention at the 

University of North Carolina at Chapel Hill. Pre-post measures included the Body Mass Index (BMI), the Knowledge and Attitudes Survey 

(adapted and modified from the USDA), and lunch tray photo observation.

The target population consisted of 1,191 students enrolled at nine program sites (schools) around the state of Wisconsin. Students ranged in age 

from 6 to 12 (with an average age of 9.6 years). The sample was 53 percent male and 81 percent white. The sites included in the evaluation 

ranged from 0 (new) to 3 years of involvement in the program. The program sought to increase student knowledge about healthy food options, 

improve attitudes towards health food choices (fruits and vegetables), and increase consumption of fruits and vegetables through nutrition and 

agriculture activities such as school gardening, produce taste-testing, and farm field trips.

The aim of the evaluation was to examine the effectiveness of the Farm to School program on students’ knowledge, attitudes, and behavior with 

respect to nutrition, health, and food systems, while simultaneously increasing understanding of the dynamics surrounding Farm to School

program implementation. The study also aimed to describe: (1) baseline overweight and obesity prevalence; (2) changes in knowledge and 

attitudes pertaining to food, nutrition, agriculture, and fruit and vegetable consumption; and (3) fruit and vegetable availability and 

consumption during school lunch meals. Researchers employed a single group pre-post design supplemented with comparison data from a 

large national survey, the National Health and Nutrition Examination Survey (2007-2008).

Results show that the program increased knowledge and attitudes as well as consumption fruits and vegetables among students through 

improved access to these foods in school lunches. Improvements in student behavior tended to increase incrementally with years of school 

involvement in the program, implying that the program has gradual, yet sustainable positive impacts on study health behavior. Researchers 

recommend that future analysis be conducted to expand on the study conclusions and to delve deeper to identify additional factors that 

positively affect student health.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

77

Effect sizes: Means reported based on percentile scores; significance reported without confidence levels. Knowledge of food, nutrition and 

agriculture (baseline=77.5, follow-up=81.5, significant); attitudes toward liking and trying new FV (baseline=72.5, follow-up=74.5, significant); 

perception and self-efficacy for eating healthy (baseline=57.5, follow-up=56.3, significant); exposure to previously tasted FV (baseline=83.2, followup=85.8, significant); liking of the FV that they reported having tasted (null, NS); willingness to try the FV that they reported not having tasted 

(baseline=45.5, 47 follow-up=47).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

78

1376 Westat. (2005). Public Report on the Results of the 2005 Independent Living Survey of the Senior Companion Program. Rockville, MD.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Senior Companion 

Program

Senior Corps Senior Corps Westat Nationwide

This 2005 evaluation of the Senior Companion Program aims to determine the characteristics of service recipients, details on the format and 

frequency of the Senior Companion visit, and overall client satisfaction with the program. Senior Companions is a national program that 

provides services to frail and isolated adults. Volunteers serve between 15 and 40 hours a week in senior centers, hospitals, nursing or 

convalescence facilities, adult day care centers, and private homes. There they assist with simple chores, provide transportation, offer 

companionship and, overall, add richness to their clients’ lives.

Researchers used a telephone survey to collect information from 500 randomly selected seniors who were regularly visited by a Senior 

Companion. Participants were 18 percent male and 82 percent female, and ranged in age from 43 to 104 years old (the median age was 81). 

Seventy percent of the service recipients were White and 16 percent were Black or African American. Nine percent of the respondents identified 

themselves as being Hispanic or Latino. Roughly 40 percent reported suffering from one to two health conditions.

Respondents were asked to rate their satisfaction with respect to the capabilities of the Senior Companion Program to serve them. Virtually 

everyone provided positive ratings, with 98 percent indicating they were either extremely satisfied (82 percent) or somewhat satisfied (16

percent). Nine out of ten respondents reported that Senior Companions improved their quality of life. Companions gave them the sense of 

having a friend (73 percent said “considerably” and another 13 percent said “moderately”) and helped them have peace of mind (72 percent

“considerably” and 14 percent “moderately”). Close to half the respondents reported that Companions helped “considerably” or “moderately”

with eating regular meals, maintaining overall physical health, going shopping, and getting other errands done. Overall, the study provides 

preliminary evidence for these findings.

Effect sizes (percentile scores): Satisfaction with program (82 percent extremely satisfied); responsiveness to needs (94 percent considerably or 

moderately responsive); improved quality of life (67 percent considerably improved); helped with peace of mind (72 percent considerably helped); 

helped maintain overall physical health (34 percent considerably helped); companionship and sense of having a friend (73 percent considerably 

helped).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

79

1804 Rosenthal, E. L., Reyes, H., & Mendez, B. (2010). In Service to Community – Giving the Story a Name: The Community HealthCorps 

Impact Evaluation. El Paso, TX: University of Texas, El Paso.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Community 

HealthCorps

Community 

HealthCorps

AmeriCorps State 

and National

University of Texas, EI Paso Evaluation Team

E. Lee Rosenthal, Principal Investigator

Nationwide

This impact evaluation assesses the effect of Community HealthCorps Members on individuals’ and families’ access to and utilization of health 

care and health education. The study was implemented on four distinct levels, and includes data from Community HealthCorps programs 

operating nationwide. Community HealthCorps aims to improve access to necessary primary and preventative care services for the medically 

underserved, and to become a national pipeline for careers in community health centers.

Data collection for the individual/family level of the study took place between August 2008 and January 2010. Survey data were collected from 

46 administrators and 233 program alumni (up to ten years out of the program) via an online survey. Additional information was gathered 

from performance measures collected at sites, members’ journal entries known as Great Stories, and select member profile interviews.

Routinely documented performance measure data show 72,000 people per year received assistance in increasing access to care; 600,000 

individuals received assistance in improving utilization of care; and an average of 110,000 individuals per year attended workshops, health 

fairs, and health education sessions facilitated by HealthCorps Members and health center staff. According to administrators, HealthCorps

members have a “very significant” or “significant” positive impact on the individuals and families served. Alumni who were surveyed thought 

that they had a significant or very significant impact on individuals and families served (average 4.2 on a 5-point rating scale). Overall, the 

study provides pre-preliminary evidence for these findings.

Effect sizes: Percent answering “very significant” or “significant”: Health center staff (positive impact on individuals and families served=83

percent; positive impact on health centers=93 percent); Administrators/program coordinators (positive impact on communities=83 percent).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

80

1810 Griffith, J.D. (1994). Evaluation of the SCP/AoA Joint Initiative for the Vulnerable Elderly Program. Research Triangle Park, NC: 

Research Triangle Institute.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

SCP/Administration 

on Aging (AoA) Joint 

Initiative for the 

Vulnerable Elderly

Senior Companions, 

Administration on 

Aging

Senior Corps RTI International 11 unidentified states

This outcome evaluation examines SCP/Administration on Aging (AoA) Joint Initiative program outcomes and impacts, especially the ability of the 

Senior Companion to help homebound elderly clients maintain independence. The study encompasses 19 sites spread across 11 states, with an 

average of eight Senior Companions serving 29 clients at each site. Volunteers in the Joint Initiative program provide peer social support by 

listening and talking to their clients. They also prepare meals, do light housekeeping, provide assistance with walking, escort clients who 

require assistance when leaving their home, and provide information and access to services in the community.

Data collection took place at six month intervals between October 1990 and October 1993. Researchers gathered data from telephone interviews 

with SCP Project Directors, Volunteer Station Supervisors, members of the Advisory Council, Sponsor Agency staff, and the directors of local 

Area Agencies on Aging (AAA). They also collected and analyzed client and Companion rosters, grant applications, and other project materials. 

Additionally, site visits were conducted at nine Joint Initiatives locations.

The study found that Joint Participant clients were, on average, 80 years old, predominantly female (79 percent), and living alone (66 percent). 

Almost one-fifth of these clients were reported to be chair- or bed-fast, and 10 percent were diabetic. Functional impairments included difficulty 

in walking or balance, dependency in bathing, dressing or’ toileting, incontinence and confusion, or Alzheimer’s. Volunteer Station Supervisors 

and Companions reported positive impacts of SCP on clients and Companions. Practical benefits to clients came from Companions’ help with 

cooking or other household activities, grooming, and getting out for appointments, errands or visits. Other benefits included psychological 

ones—having someone to talk to, having the Companion’s visit to look forward to, and being less lonely or isolated. For Companions, SCP 

provided such direct benefits as stipends, physical examinations, and in-service training. Psychological and social benefits included a sense of 

having purpose in life, enhanced self-esteem, and a feeling of being helpful and needed. Overall, the study provides pre-preliminary evidence 

for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

81

Member Development

859 Epstein, D. (2009). Evaluating the Long-Term Impacts of AmeriCorps Service on Participants. PRGS Dissertation. RAND Corporation.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

AmeriCorps CNCS AmeriCorps State 

and National

Diana Epstein (Dissertation)

Data collected by Abt Associates, Inc.

Nationwide

This quasi-experimental study assesses the long-term impacts of AmeriCorps service on participants, particularly the effects of program 

characteristics on long-term member outcomes. The study draws on data collected from a previous evaluation conducted by Abt Associates 

Inc., which involved 2,192 AmeriCorps participants and a comparison group of 1925 similar individuals who had previously expressed interest 

in AmeriCorps.

Survey data were collected from AmeriCorps and comparison group individuals at four time points: pre-service (1999), post-service (2000), 3-

year follow-up (2003), and 7-year follow-up (2007). Fifty-one percent of the original sample completed all 4 surveys. This analysis focuses on 

members of the State and National treatment group. Most members began their service with some college experience or were college graduates; 

the majority fall into the 18-24 age range, although there is a large percentage of members who served when they were above age 30; and this 

sample includes a sizeable number of non-white participants (48 percent). Qualitative interviews were conducted to support the results of the 

qualitative analysis.

Results showed that members were more likely to show positive long-term effects after serving in programs that: (1) placed emphasis on 

mentoring, and encouraged members to develop relationships with other staff that persisted past their end of their service; (2) spent less time 

on general member development and more time on project-specific training and direct service; (3) focused on leadership, and provided 

members with the opportunity to organize and plan their own events; and (4) allowed members to feel useful and productive in their service, 

by providing opportunities for members to feel appreciated by service recipients and to see their contributions to the community. The study 

found mixed results for the effects of serving in teams: NCCC members reported an overwhelmingly positive team experience, while 

State/National members were often challenged or discouraged by their relationships with people they had not chosen to serve with. Similarly, 

the effects of serving in diverse communities or with diverse members were mixed; the only diversity variable with consistently positive effects 

was the extent to which a member worked with diverse service recipients. Overall, the study provides moderate evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

82

875 McAdam, D., & Brandt, C. (2009). Assessing the effects of voluntary youth service: The case of Teach For America. Social Forces, 

88(2), 945-969.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Teach for America Teach for America AmeriCorps Doug McAdam, Cynthia Brandt (Stanford 

University)

Nationwide

This quasi-experimental study seeks to assess the longer-term effect of TFA on participants’ current civic attitudes and behaviors. The study 

encompasses all individuals accepted into TFA between 1993 and 1998, including those who chose not to serve and those who dropped out 

midway through their service. TFA volunteers commit to two years of service at partner public schools, teaching low-income students with the 

aim of expanding their opportunities in school and in life.

Overall, 2,771 individuals were eligible for the study. Between May 2001 and April 2002, researchers were able to obtain addresses for and mail 

surveys to 2,541 of them: 1,583 graduates, 324 dropouts, and 634 non-matriculants. They received 1,124 responses, for a completed response rate 

of 44.2 percent. The demographics of the responders were found to be virtually identical to the larger population of TFA members. All 

participants were evaluated with paper survey that focused on civic behaviors, including participation in institutional politics, community 

service, social movements, voting, charitable giving, and pro-social employment.

While TFA graduates score consistently higher on a broad range of attitudinal items measuring civic commitment, these differences appear to 

be less a byproduct of the TFA experience than a reflection of the ongoing integration of a large subset of the alumni into the TFA corps. TFA

graduates lag significantly behind one or both comparison groups on a variety of measures of civic behavior – not only in regard to their 

current service commitments, but also on six other measures of civic participation. The great majority – roughly 85 percent – of graduates report 

having a very positive experience in TFA, and are indistinguishable from the comparison groups in their levels of current service/civic 

participation. It is the 15 percent of the graduates who have a retrospectively negative view of their TFA experience who account for the 

service/civic “gap” between graduates and the other two subject groups. Many of these graduates either came to doubt the effectiveness of the 

program or found the experience isolating, and exited TFA less motivated to remain involved in service or civic activities. Results also indicate 

that many TFA alumni prefer to become involved in advocacy for education reform through TFA’s offices, and consequently are less available 

for all other forms of civic/political participation. Overall, the study provides moderate evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

83

1418 Jastrzab, J., Giordono, L., Chase, A., Valente, J., Hazlett, A., LaRock, R., & James, D. (2007). Serving Country and Community: A 

Longitudinal Study of Service in AmeriCorps, Early Findings. Cambridge, Massachuchusetts: Abt Associates.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

AmeriCorps CNCS AmeriCorps State 

and National, 

AmeriCorps NCCC

Abt Associates Inc. Nationwide

This quasi-experimental study examines the impact of AmeriCorps on its members in terms of civic engagement, education, employment, and 

life skills. The study targeted over 2000 AmeriCorps participants and a comparison group of close to 2000 similar individuals who had 

previously expressed interest in AmeriCorps. Both AmeriCorps participants and the comparison group were surveyed at three time points: preservice (1999), post-service (2000), and 3-year follow-up (2003).

AmeriCorps participants were generally in their early twenties, with a median age of 23.8 for State/National programs and 22.1 for NCCC. 

More than two-thirds of participants were female. Race and ethnicity varied greatly between programs: NCCC members were 86 percent White, 

compared to only 46 percent of S/N members. AmeriCorps members were, on average, better educated than the nation’s population, with over 

95 percent of S/N members and 99 percent of NCCC members earning a GED or high school diploma by the 3-year follow-up.

The study found that ASN alumni demonstrated significant positive effects in all eight attitudinal outcomes related to civic engagement and 

both attitudinal employment outcomes. They also showed significant positive effects on one of the four behavioral civic outcomes and both

behavioral employment outcomes. Results from NCCC were more mixed, including statistically positive impacts on four of eight civic attitudes, 

two of four civic behaviors, and one of two employment behaviors. The study found no significant positive impacts on education outcomes or 

life skills, and a statistically significant negative effect on appreciation of cultural and ethnic diversity among NCCC alumni. Overall, the study 

provides moderate evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

84

1468 Finlay, A. K., Flanagan, C., & Wray-Lake, L. (2011). Civic engagement patterns and transitions over 8 years: the AmeriCorps 

national study. Developmental Psychology, 47(6), 1728-1743.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

AmeriCorps AmeriCorps/ CNCS AmeriCorps State 

and National

N/A Nationwide

This study applies latent transition analysis to data reported on previously in published CNCS reports (2001, 2004, 2008). It examines civic 

engagement patterns over eight years among 505 first-time, full-time AmeriCorps members who enrolled in September 1999 to January 2000, 

and a comparison group of 839 people who indicated interested immediately prior to this enrollment period, but did not become members. 

Only participants ages 19-29 at initial data collection were included, and others were excluded from this analysis due to factors including 

missing data on covariates.

Three latent statuses were identified, all of whom were likely to endorse attitudes of civic consciousness and perceptions of civic knowledge, 

but differed in behaviors: inactive people were not likely to engage in any civic behaviors, voting-involved people who were likely to volunteer 

and vote, and highly committed people who were likely to score high on all civic items. AmeriCorps and comparison group members were 

equally likely to be highly committed at baseline, but AmeriCorps members were less likely to be classified as voting-involved. At baseline, 

younger participants were less likely to be involved at than older participants, and Asian participants were less likely to be involved than white 

participants. Those who attended college were more likely to have higher levels of civic engagement at baseline and over time. AmeriCorps 

members who had been voting-involved at baseline were 1.12 times more likely (p<0.05) than comparison group members to have transitioned 

from voting-involved to highly committed.

Effect sizes: Chi-square: Volunteering = 0.27 (NS); community participation = 11.82 (p<0.01); civic organizational involvement = 4.70 (p<0.05); local 

voting = 11.31 (p<0.01); national voting = 20.56 (p<0.001); civic consciousness = 23.18 (p<0.001); perceptions of civic knowledge = 1.06 (NS).
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

85

1520 Ward, K. D. (2011). Predicting and cultivating public service motivation: A longitudinal study measuring the effect of participation in 

AmeriCorps programs. Denver, CO: University of Colorado at Denver.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

AmeriCorps CNCS AmeriCorps Abt Associates Inc. Nationwide

This quasi-experimental study assesses the effect of AmeriCorps programs on its members in terms of public service motivation (PSM) 

immediately after service and in the decade after service. The study draws on data collected from a previous evaluation conducted by Abt 

Associates Inc., which involved over 2000 AmeriCorps participants and a comparison group of close to 2000 similar individuals who had 

previously expressed interest in AmeriCorps.

Survey data were collected from AmeriCorps and comparison group individuals at four time points: pre-service (1999), post-service (2000), 3-

year follow-up (2003), and 7-year follow up (2007). Fifty-one percent of the original sample completed all 4 surveys. While the original in-depth 

survey contained over 400 variables, this study only draws on demographic information and survey items related to PSM. Participants were 74 

percent female and 63 percent white, with a mean age of 26.7 years old. Levels of income and education were relatively normally distributed: 95 

percent of participants had completed high school, and the median family income was $30,000 - $40,000.

At post-service, AmeriCorps members demonstrated a higher commitment to public interest and knowledge of their communities than the 

comparison group. These positive program effects persisted at the time of the 7-year follow-up survey. However, values on these constructs for 

those in both the treatment and comparison groups were significantly lower than at post-service. AmeriCorps members demonstrated a lower 

openness to new ideas than the control group at post-service; however the negative effect of program participation was no longer significant at 

the 7-year follow-up, when AmeriCorps members actually reported a greater openness to new ideas than the comparison group. For attraction 

to public policymaking, AmeriCorps members scored significantly lower than those in the comparison group at pre-service; however, by postservice the gap had narrowed by about half, suggesting AmeriCorps service may have had a positive impact. This trend became even more 

pronounced at the 7-year follow-up, with the difference between these two groups again decreased by nearly half. Overall, these findings 

suggest the effects of AmeriCorps service may be relatively persistent over time. The study provides moderate evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

86

1704 Snell, E. B. (2009). Examining the Career Choices and Civic Engagement Activities of Teach for America Alumni (Dissertation submitted 

to Indiana University).

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Teach for America Teach for America AmeriCorps Erin B. Snell (Dissertation) Nationwide

This quasi-experimental study assesses the impact of TFA on its alumni, in terms of civic engagement and continued professional involvement 

with education or social reform. The study targeted TFA alumni who completed their service between 1990 and 2004. TFA members commit to 

two years of service at partner public schools, teaching low-income students with the aim of expanding their opportunities in school and in life.

The quasi-experimental study component focused 3,644 TFA alumni who began their service between 2001 and 2004, and a similar group of 260 

individuals who were accepted into the TFA program but chose not to participate. Participants were largely female (83 percent), with a mean 

age of 28-29. All participants completed an online survey during winter 2006. Primary outcomes include civic engagement (e.g., participation in 

political meetings, online forums, religious organizations, etc.) and continued professional involvement in Education (e.g., current title/position 

in education nonprofit or policy making group, currently studying education, professor, etc.).

Results suggest that TFA alumni are less likely than the comparison group to volunteer for organizations, make monetary donations, contact the 

media or other organizations to voice their views on an issue, contact government officials, learn about candidates and ballots before voting, 

and vote in national elections. Alumni who are of minority ethnicity are more likely to attend religious organizations and meetings and 

participate in on-line discussions, but they are less likely to make monetary donations, contact government officials, learn before voting, and 

vote in local or national elections. TFA alumni report that their experience as a corps member inspired them to make career choices that 

positively impact educational and social reform. TFA alumni are more likely than their non-matriculant peers to work and pursue graduate 

study in the field of education or public service. The more recently a TFA alumnus completed the program, the more likely he/she is to choose 

employment or graduate school programs directly in education, generally in the field of education, or in the public or nonprofit sectors. Overall, 

the study provides preliminary evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

87

1815 Corporation for National and Community Service, Office of Research and Policy Development. (2008). Improving Lives and 

Communities: Perspectives on 40 Years of VISTA Service. Washington, DC.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Volunteers in Service 

to America (VISTA) 

VISTA (varying 

organizations)

VISTA Abt Associates, Inc. Nationwide

This study of the VISTA program includes phone surveys completed in the mid-2000s by 1,539 people that enrolled in VISTA from 1965 

through 1993, including 985 VISTA members that completed at least nine months of service, and 554 comparison group members who enrolled 

but did not serve or completed fewer than four months of service. Responses were drawn randomly from a VISTA database, and the response 

rate was 38 percent. Qualitative analysis of in-person interviews with a small subset of respondents supplemented the results. Respondents 

were divided into three groups: those enrolling between 1965 through 1972 (first generation), 1972 through 1980 (second generation), and 1981 

through 1993 (third generation). Demographics differed among the groups, with the first generation including more males (45 percent vs. 36 

percent vs. 21 percent), people aged 18-25 (93 percent vs. 76 percent vs. 43 percent), and unmarried people (83 percent vs. 72 percent vs. 42 

percent) than the second and third generations. Characteristics of service also changed over time, with the first generation receiving much more 

training and being more likely to be placed far from home than subsequent generations.

The majority of VISTA members in all three generations reported that they felt challenged by their VISTA service, they believed that they made 

a contribution, and they felt part of a community. Many reported positive-life changing experiences, three-quarters stated that their VISTA 

experiences made them re-examine their beliefs and attitudes about themselves, and a large majority (87 percent) said that they would join the 

program again.

Significance and effect sizes are not given for outcome comparisons between the treatment and comparison groups. However, results indicate 

that African American and Hispanic respondents may be more likely than white respondents, and women may be more likely than men, to say 

that VISTA influenced the amount and kind of volunteering they performed. Both treatment and comparison group members reported that 

they came into VISTA with high levels of civic engagement, and both reported generally high levels of civic engagement, education, and 

employment stability. A potential positive effect of VISTA service on parenting was apparent in measures of intergenerational transfer of 

values, particularly in the second and third generations. Among third generation respondents, 56 percent of VISTA members reported that their 

children regularly accompanied them on volunteering activities, while only 37 percent of comparison group members, and VISTA members 

were more likely to report that their children participated in AmeriCorps or similar programs than comparison group members.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

88

2000 Price, C., Williams, J., Simpson, L., Jastrzab, J., & Markovitz, C. (2011). National Evaluation of Youth Corps: Findings at Follow-Up. 

Prepared for the Corporation for National and Community Service. Cambridge, MA: Abt Associates Inc.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Youth Corps The Corp Network AmeriCorps Abt Associates, Inc Nationwide

This randomized control trial assesses the impacts of youth corps participation on corps members’ educational outcomes, employment-related 

outcomes, civic engagement, life skills, and risky behaviors. The study included 21 of the 59 Youth Corps programs that are members of the 

Corps Network and that, in 2005, had annual enrollment of 50 or more corps members. The Corps Network is a nationwide association of youth 

development programs that engage diverse and often disadvantaged young people in service projects designed to address important 

environmental and community needs.

A 12-month recruitment effort (mid 2006-mid 2007) resulted in 2,043 eligible study participants who complete baseline surveys, of which 1,357 

were randomized to the treatment group and 686 were randomized to the control group. Applicants assigned to the control group were 

embargoed for 18 months from enrolling in the Youth Corps program to which they had applied. Although researchers were only able to assess 

21 of the 34 randomly selected programs, the study team found no evidence that the study sample was unrepresentative of the population of 

Corps Network members. Participants in both groups were approximately 60 percent male and 40 percent female, with a median age of 18-21. 

Each participant received a telephone surveys before the random assignment, with follow-ups 18 and 30 months later.

The impacts of Youth Corps on the three key outcomes (educational attainment, employed or in school, and volunteering) were not statistically 

significant. Several possible explanations of these findings are discussed, including the sample size, timing of the follow-up surveys, attrition, 

the representativeness of the study sites, and the possibility that the true impact of Youth Corps is close to or equal to zero. The results did 

indicate that treatment group members experienced significant beneficial impacts on their financial well-being: for the year prior to the 18-

month survey, Youth Corps had a positive impact on annual income, hourly wages, and the ability to make ends meet. The researchers contend 

that these financial impacts are likely attributable at least in part to the stipends corps members received. This evaluation also included an 

extensive set of subgroup analyses, which found no evidence that Youth Corps was particularly beneficial for any of the selected subgroups. Of 

the programs studied, only a few had impact estimates that were significantly different than zero for any of the three outcomes. Overall, the 

study provides strong evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

89

2473 Anderson, L. M., Laguarda, K. G., Williams, I. J., & Munoz, A. (2007). The Effect of the City Year Experience Over Time: Findings 

from the Longitudinal Study of Alumni. Washington, DC: Policy Studies Associates.

Intervention name Operating 

program/ 

organization 

name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

City Year City Year AmeriCorps Policy Studies Associates Location(s) of the study: 13 

cities around the United 

States

Beginning in Fall 2004 and ending in Winter 2005, researchers with Policy Studies Associates collected data from City Year alumni 1 to 15 years 

following completion of their AmeriCorps service. Respondents were 54 percent female; 46 percent White, 29 percent Black, 13 percent 

Hispanic. The evaluation examined the long-term impact of City Year AmeriCorps participation on alumni civic engagement, and political and 

social attitudes and behavior.

The study employed a quasi-experimental design utilizing statistical matching to compare City Year alumni with similarly situated members of 

national cohorts drawn from the National Election Studies (2000 and 2002), and the Center for Research on Civic Learning and Engagement’s 

(CIRCLE) National Civic Engagement Survey I (2002). Primary data collection employed three measurement strategies: (1) the Alumni Cohort 

Study, a mail survey of the universe of City Year alumni; (2) the Study of City Year Parents, a telephone survey designed to obtain additional 

insight into the effects of City Year on younger alumni and their families; and (3) the Longitudinal Study of Alumni, which involves longer, 

open-ended telephone interviews with a sample of alumni from a recent cohort of City Year participants and a matched sample of nonparticipants. The alumni survey garnered responses from 2,189 of 5,668 former members, a response rate of 38 percent.

Researchers found that City Year had a strong, positive impact on the attitudes and behavior of alumni, suggesting that City Year experience 

leads alumni onto and/or alters their pathway to higher civic engagement and social capital. In addition, City Year had a strong, positive impact 

on the civic engagement of alumni, and substantially increased the percent of alumni who voted, make political contributions, and volunteer.

Effect sizes (expected direction): Political efficacy = yes; egalitarianism = no; social trust = yes; organizational membership = yes; organizational 

leadership = yes; media usage = yes; voting = yes; political expression = yes; volunteerism = yes; volunteer leadership = yes.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

90

2511 Aguirre International. (1999). Making a difference: Impact of AmeriCorps*State/National Direct on Members and Communities 1994-95 

and 1995-96. San Mateo, CA.

Intervention name Operating 

program/ 

organization 

name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

AmeriCorps State 

and National Direct

CNCS AmeriCorps State 

and National

Aguirre International Nationwide

This quasi-experimental study assesses the impact of ASN programs on the civic responsibility, educational attainment, and life skills of their 

AmeriCorps members. The study focuses on the 1994-95 and 1995-96 program years. Researchers employed three tiers of data collection: 

administrative data from all 310 AmeriCorps programs, including grant applications, reports to the CNCS, member enrollment and exit forms, 

and an annual review; a random sample of 60 programs involving 1800 AmeriCorps members and 700 controls, who submitted Entry, Exit, and 

Follow-Up Questionnaires and the Skills Inventory; and a subset of this sample including eight projects, 500 members, and 500 controls, who 

participated in Life Skills Assessments, focus groups, and journal reviews.

The Life Skills Assessments indicate that participation in AmeriCorps resulted in substantial gains in life skills for more than three-quarters of 

AmeriCorps members. Most AmeriCorps members who initially considered their life skills to be deficient achieved substantial or dramatic 

gains in every area except use of information technology. In terms of civic engagement, nearly two-thirds of AmeriCorps members reported 

that they would definitely be involved in future community service, and approximately half of these had already either applied to or accepted 

another service position. Only 1 percent of members reported that they would not be involved in future community service. Findings for 

educational attainment indicate that nearly 40 percent of AmeriCorps members were enrolled in educational programs while serving in 

AmeriCorps. A large majority of members intended to use their educational awards to pay for further education/training (approximately 74

percent) or to pay off student loans (approximately 8 percent). Concerns with internal validity – particularly the non-equivalence of the 

treatment and control groups and relatively high non-response rates – suggest results of this study should be interpreted with some caution. 

Overall, the study provides only preliminary evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

91

335 Kim, A. K., & Salazar, M. (2012). Leveling the field is not enough: Promoting culturally responsive pedagogy in Teach for America through 

the evaluation of intercultural competence, colorblindness, White racial identity attitudes, and the belief that culture matters in education

(Dissertation presented to University of Denver). 

Intervention name Operating 

program/ 

organization 

name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Colorado Teach for 

America

Teach for America AmeriCorps State 

and National

Amber K. Kim (Dissertation) Colorado

This outcome evaluation assesses the impact of Colorado TFA on its members in terms of intercultural competence, colorblindness, and white 

racial identity attitudes. The study involved 134 teachers who attended Colorado TFA’s summer orientation before beginning their first year of 

TFA service. TFA volunteers commit to two years of service at partner public schools, teaching low-income students with the aim of expanding 

their opportunities in school and in life.

Data were collected at CO TFA’s Summer Orientation in July of 2010 (n=134), and again via several online surveys during April 2011 (n=78). 

Although there was substantial attrition between the two evaluations, demographics for both groups were similar: participants were 63 percent 

female and 37 percent male, with a median age range of 22-30. Ninety-two percent of participants identified as White. At each point, 

participants were evaluated using the Intercultural Development Inventory, version 3 (IDI v. 3), the Colorblind Racial Attitudes Scale 

(CoBRAS), and (for those who indicated that they were white) the White Racial Identity Attitudes Scale (WRIAS).

At Time 2, the mean score for intercultural competence decreased for the sample of Colorado TFA corps members (CMs) examined in this study. 

Similarly, the mean score for colorblindness increased over time, and the gap between the sample’s actual level and perceived level of 

intercultural competence widened. These changes over time were statistically significant (p < .001), and indicate that the corps experience does 

not increase intercultural competence or lessen colorblindness; in fact, at the end of the first year of the corps experience, participants in this 

research study were less culturally competent and more colorblind. CoBRAS and WRIAS results suggest that white CMs tend to focus on 

cultural similarities and minimize cultural differences. Although many have a healthy, positive anti-racist white identity, they lack awareness of 

the socio-cultural significance of race, and struggle to see their own racial privilege. The conclusion that CMs regress in these essential 

constructs is important because this means that CMs are entering their second year of teaching with a greater tendency to minimize differences 

and ignore color/race that, in turn, means a subtractive schooling experience for students and resistance to culturally responsive pedagogy. 

Overall, the study provides preliminary evidence for these findings.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

92

601 Astin, A. W., & Sax, L. J. (1998). How undergraduates are affected by service participation. Journal of College Student Development, 

39(3), 251–263. Retrieved from http://cshe.berkeley.edu/events/seru21symposium2005/papers/sax2.pdf

Intervention name Operating 

program/ 

organization 

name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

LSA Higher 

Education 

LSA Higher 

Education

LSA UCLA and RAND Corporation National

Forty-two institutions of higher education conducting community service programs through the LSA Higher Education program participated in 

this outcome study, which was based upon data collected via a mailed paper survey collected annually from 1990 to 1994 (the College Student 

Survey). The study, conducted jointly by UCLA and the RAND Corporation, examined the impact of community service participation on 

undergraduate students’ propensity to engage in service as well as their academic development, acquisition of life skills, and sense of civic 

responsibility. The target population consisted of entering freshman attending these 42 institutions of higher learning. The group was 34 

percent female and 66 percent male.

The main evaluation question was “Does participating in service affect (a) civic responsibility (b) educational attainment, and (c) life skills?”

The outcome evaluation design relied on end-of-service surveys collected from five successive freshman classes. Additional data used as 

controls included: SAT and ACT scores, enrollment information from the U.S. Department of Education, and information on LSA Higher 

Education programs collected by the RAND Corporation. The sample consisted of 3,450 students.

The findings reported suggest participating in service activities during the undergraduate years enhances the student’s academic development, 

life skill development, and sense of civic responsibility. Every one of the 35 outcome measures was favorably influenced by engagement in 

some form of service work. These beneficial effects occur for all types of service, whether the activities are concerned with education, human 

needs, public safety, or the environment. Although most findings are highly significant statistically, the absolute effect sizes are generally quite 

small—especially in the case of academic outcomes. The study findings point to the value of service learning for enhancing student 

development during the undergraduate years while simultaneously fulfilling a basic institutional mission of providing service to the 

community.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

93

1788 Simon, C., & Wang, C. (1999). Impact of AmeriCorps on Members’ Political and Social Efficacy, Social Trust, Institutional Confidence 

and values in Idaho, Montana, Oregon and Washington. Portland, OR: Northwest Regional Education Laboratory.18

Intervention name Operating 

program/ 

organization 

name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

AmeriCorps CNCS AmeriCorps Northwest Regional Educational Laboratory Idaho, Montana, Oregon, 

Washington

This outcome evaluation assesses the effect of AmeriCorps programs on members’ community involvement, confidence in civic institutions, 

social trust, and public-oriented values The study included data from AmeriCorps programs operating in Idaho, Montana, Oregon, and 

Washington during the 1997-98 and 1998-99 program years.

Researchers selected 1,000 AmeriCorps members from each program year to receive pre-service and post-service surveys, with response rates of 

99 percent (pre-) and 56 percent (post-) for the 97-98 cohort and 86 percent (pre-) and 54 percent (post-) for the 98-99 cohort. Respondents to the 

post-service surveys did not differ significantly from the pre-service respondents in terms of political beliefs, social values, or basic 

demographic information. Survey items asked members to rate their confidence in various public institutions, to rate their agreement with 

statements about government and politics, and to rank a list of personal values based on their importance.

Participants showed no significant changes in their confidence in political and social institutions between the pre- and post-service evaluations. 

This is perhaps not entirely unexpected given that AmeriCorps does not actively attempt to build members’ confidence in public institutions. 

However, the researchers found that AmeriCorps members are politically and socially aware individuals: nearly 80 percent of the membership 

surveyed indicated that they pay some to a great deal of attention to national politics, while approximately 70 percent reported a similar level of 

attention paid to state and local politics. AmeriCorps members are active voters, with between 57 and 68 percent indicating that they nearly 

always vote in local, state, and national elections. No systematic changes over the pre- to post-service period were found in optimism, civic 

attitudes, or social trust. For the most part, AmeriCorps service appears to have had little impact on the participants’ beliefs about their role in 

politics and society. However, despite relatively stagnant public values, AmeriCorps members appear to experience a marked shift in personal 

or private values. Overall, the study provides preliminary evidence for these findings.

 

18 Results from this study were also reported in a peer-reviewed journal article: Simon, C. A., & Wang, C. (2002). The Impact of AmeriCorps 

Service on Volunteer Participants: Results from a 2-Year Study in Four Western States. Administration & Society, 34(5), 522–540. 

http://doi.org/10.1177/009539902237274 (document ID 0835)
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

94

Organizational Capacity

1512 Elliott, G. L. (2002). Capacity as a Fundamental Objective: Definition and Measurement in the AmeriCorps * VISTA- Habitat for 

Humanity International Affiliate Partnerships. Washington, DC: Corporation for National Service. 

Intervention name Operating 

program/ 

organization 

name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

Habitat for Humanity 

VISTA Program

Habitat for 

Humanity VISTA 

Program

VISTA CNCS Habitat for Humanity 

affiliates nationwide 

(excluding territories)

The target population for this CNCS-sponsored study consisted of all 535 Habitat for Humanity (HfH) affiliates in the United States, 135 of which 

(25 percent) had VISTA projects currently running or within the last three years (treatment group). All HfH affiliates were mailed a post-only 

survey in January 2002; 273 affiliates responded (51 percent).

The study employed a quasi-experimental design in which HfH affiliates with VISTA members were compared with HfH affiliates without 

VISTA members. Baseline equivalence was established for a number of key organizational characteristics. The study sought to assess the impact 

that having VISTA members as part an HfH affiliate made on affiliates’ organizational capacity, specifically their ability to complete work 

related to the organizational mission, including planning and production of affordable housing for low-income families. The study also 

includes a detailed discussion of the components of organizational capacity and operationalized this construct through a Capacity Index 

Scoring System (CISS).

Organizations that had VISTA members serving with them in the past three years had total capacity scores nearly four times higher than 

organizations that did not, holding constant the effects of other factors in the model. The study authors conclude that causation is not provable, 

but is supported. The authors advocate the application of the CISS to other organizational contexts to advance the theoretical conception and 

measurement of capacity, and to further understanding of the aspects of capacity that most influence accomplishment of organizational goals.
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

95

2516 CalSERVES-NCOE. (2012). AmeriCorps VIP Volunteer Capacity Study. Rohnert Park, CA.

Intervention name Operating

program/ 

organization 

name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

AmeriCorps VIP CalSERVES AmeriCorps State 

and National

JBS International, Inc. California

This quasi-experimental study assesses the impact of the CalSERVES Volunteer Infrastructure Project on nonprofit and educational organizations’ 

ability to create and sustain a successful volunteer program. The study targets 133 CalSERVES VIP partner sites from across California that 

received assistance during the 2011-2012 program year, as well as 59 similar sites that did not receive VIP services. Through the Volunteer 

Infrastructure Project, teams of AmeriCorps members are placed in California communities to provide infrastructure support for volunteer 

programs in non-profit and educational organizations. Each VIP Fellow helps build the systems necessary for organizations to structure, 

implement, increase and maintain volunteer programs, with the specific goal of recruiting 50 long-term and 50 short-term volunteers.

From the 133 partner sites and 59 comparison sites that were selected, 117 and 58 (respectively) provided responses. All programs were 

assessed using a Volunteer Capacity Assessment (VCA) that asked them to rate how well their organization fulfills statements related to 

organizational capacity, volunteer recruitment, and elements of a successful volunteer program. Treatment programs completed the VCA three 

times during the 2011-2012 program year: a pre-treatment, mid-year, and post-treatment. Comparison programs took the VCA twice, at midyear and end-of-year. Data used for propensity score matching was gathered once in Spring 2012 for all sites.

Overall, the partner sites that participated in the CalSERVES VIP program reported strong, positive changes over time on the items included in 

the VCA instrument. Organizations that did not participate in the CalSERVES VIP program reported much smaller (and sometimes negative) 

changes on the VCA items. The use of a propensity score matching procedure diminishes the likelihood that factors other than participation in 

the CalSERVES VIP program influenced the change in reported scores over time. Overall, the study provides moderate evidence for these 

findings.

Effect sizes: organizational capacity (expected direction, p<0.005), volunteer recruitment (expected direction, p<0.05), elements of successful 

volunteer program (expected direction, p<0.01)
NATIONAL SERVICE SYSTEMATIC REVIEW AND LITERATURE SYNTHESIS

August 19, 2015

96

2530 PeopleWorks, Inc. (1997). The Sustainability of AmeriCorps VISTA Programs and Activities. Washington, DC.

Intervention name Operating 

program/ 

organization 

name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

VISTA VISTA AmeriCorps 

VISTA

PeopleWorks, Inc. Nationwide

This outcome evaluation aims to determine the survival rates of VISTA supported projects after their VISTAs had departed, and examines the 

impact of the VISTA members and national VISTA policies on survival rates. The study encompasses two random samples of VISTA supported 

projects (240 total) that had ended two and five-years prior to April l996. These samples bracketed national policy changes: projects in the fiveyear sample had not been influenced directly by the new policies, while projects in the two-year sample had not started until the new policies 

were in place.

A primary or follow-up interview was conducted once with each participating project. Follow-up interviews (shorter interviews, collecting 

limited data) were conducted after three attempts had been made to reach an agency to complete the primary interview. Overall, 215 of the 240 

randomly selected projects were found to be eligible, and 201 of those 215 provided responses—119 with a primary interview, and 82 with the 

follow-up. A binary measure of project survival was employed on all interviews; on the primary interviews only, survival data were further 

broken down into five categories (expanded, maintained, transferred, reduced, and terminated).

Overall, nearly 68 percent of the VISTA supported programs continued to operate after the VISTAs had completed their assignment. The postproject sustainability rate was considerably higher for programs in the two-year sample (76.4 percent) than for programs in the five-year sample 

(60.7 percent). There were several differences between the two-year and five-year samples that may account for their higher survival rates, and 

which may be attributable to policy changes. The two-year sample was likely to begin continuation planning earlier in the project (95 percent

versus 66.9 percent began continuation planning in the first year of the project). VISTAs were far more likely to have played a central role in the 

VISTA supported project in the two-year sample (68.3 percent versus 29.8 percent), and were more likely to be credited with helping the project 

expand (63.3 percent versus 32.8 percent). The sponsoring agencies were more likely to use VISTAs to recruit and train local community 

volunteers in the two-year sample (90.0 percent versus 20.7 percent), and were far more likely to use community volunteers to support the 

program after the VISTAs left (88.2 percent versus 17.5 percent). The agencies in the two-year sample indicated that they faced fewer problems 

raising funds and locating resources to continue the project after formal VISTA support ended. Overall, the study provides preliminary 

evidence for these findings.

Effect sizes: project survival (expected direction, p<0.01).