Knowledge Management 

Education State of the Science 

Report

Submitted: August 19, 2015

Submitted to:

Adrienne DiTommaso

Corporation for National and Community Service

1201 New York Avenue, NW

Washington, DC 20525

Submitted by:

JBS International, Inc.

Aguirre Division

555 Airport Blvd, Suite 400

Burlingame, CA 94010-2002
Table of Contents 

Table of Contents .......................................................................................................................................2

Introduction and Overview of National Service Education Interventions........................................1

State of the Science.....................................................................................................................................3

School Readiness....................................................................................................................................3

Conclusion..........................................................................................................................................5

K-12 Education.......................................................................................................................................5

Teach for America .............................................................................................................................5

K-12 Literacy Programs ....................................................................................................................7

K-12 Education Studies: Behavior and General Academics........................................................8

Conclusion..........................................................................................................................................9

Post-secondary Preparation and Prospects for Success ...................................................................9

Conclusion........................................................................................................................................10

Strength of the Evidence Base ................................................................................................................11

Appendix A: Eligibility Criteria for National Service Review ..........................................................14

Appendix B: Selection Process for Education Full Reviews ..............................................................16

Appendix C: Levels of Evidence............................................................................................................17

Appendix D: Annotated Bibliography: Studies Selected for Full Review.......................................18

School Readiness..................................................................................................................................18

K-12 Education.....................................................................................................................................23

Teach For America ..........................................................................................................................23

K-12 Education Programs: Literacy..............................................................................................27

K-12 Education Programs: Behavior and Skills ..........................................................................36

Post-secondary Preparation and Prospects for Success .................................................................41
State of the Science

National Service Education 

Initiatives

August 19, 2015

1

Introduction and Overview of National Service Education Interventions

The Corporation for National and Community Service (CNCS) is dedicated to improving 

educational outcomes for students as a priority community need, and currently allocates over 

40 percent of funds toward education programs1. CNCS stands in a unique role as an entity that 

can assist in the implementation of programs that provide economically disadvantaged 

students personal attention from a caring and capable adult. The array of CNCS education 

programs engage early with students and maintain the engagement through the students’

academic careers, which can lead to improvements in: (a) attendance; (b) engagement in 

coursework; (c) rate of course completion; (d) percent of students at grade level; (e) performance 

on standardized tests; (f) rate of high school graduation; and (g) persistence in college and 

university settings.2 

This brief aggregates and synthesizes published and grey literature regarding the effectiveness 

of CNCS programs in the education focus area. Following a predetermined systematic search 

protocol, 2,508 documents were collected and reviewed for eligibility as part of a broad effort to 

assess the evidence base for CNCS programs (see Appendix A). Of the 689 documents deemed 

eligible for the overall review, 334 documents concerned programs dedicated to improving 

education outcomes in the areas of school readiness, K-12 education, and post-secondary 

preparation and success.3

This brief provides an overview of these 334 documents, as well as in-depth information 

regarding 26 documents that were selected for full review (see Appendix B). Researchers 

gathering information from these studies followed a detailed protocol for data extraction and 

determination of intervention effectiveness. To rate studies according to levels of evidence, 

researchers followed the 2015 AmeriCorps State and National level of evidence criteria (see 

Appendix C), which rates strong studies as those that target and attain very high internal 

validity, typically using randomized controlled trials (RCT), and moderate studies as those that 

increase internal validity by using quasi-experimental designs (QED) or correlational research 

with strong statistical controls. Studies that relate to outcomes but do not attempt to strengthen 

internal validity through a comparison group, or for which the comparison group is 

substantially non-equivalent to the treatment group, are considered preliminary; studies that do 

not address outcomes are pre-preliminary. An annotated bibliography with summarized 

information for these 26 documents is presented in Appendix D.

 

1 Corporation for National and Community Service (CNCS). (2011). The Corporation for National and 

Community Service Strategic Plan 2011-2015. Washington, DC.

2

Ibid.

3

Ibid. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

2

The strongest evidence of effectiveness is in the K-12 education focus area (strong), followed by 

school readiness (preliminary) and post-secondary preparation and success (preliminary). Table 

1 presents the proportion of studies selected for review by focus area and study type. 

Table 1: Documents Selected and Eligible for Full Review, by Focus Area and Study Type

Impact Outcome Other Total

Selected Eligible Selected Eligible Selected Eligible Selected Eligible

School 

readiness 2 7 2 13 1 19 5 39

K-12 

Education 11 54 7 80 0 95 18 229

Postsecondary 2 5 1 10 0 3 3 18

Multiple/ 

Other4

0 3 0 19 0 26 0 48

Total 15 69 10 122 1 143 26 334

 

4 Documents classified as Multiple/Other include studies primarily focused on teacher rather than student 

outcomes and overviews of one or more programs that included limited information about specific 

interventions and outcomes. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

3

State of the Science

Overall, the literature contained many completed studies with rigorous designs that assessed 

the effectiveness of CNCS education programs (see Figure 1). The most rigorous designs, such 

as randomized controlled trials or quasi-experimental studies, were found in 69 studies (see 

Figure 1).

5

Figure 1: Education Studies by Design

School Readiness 

To facilitate school readiness for economically disadvantaged young children, CNCS supports 

members, participants, and volunteers to develop or engage in practices that include teaching, 

tutoring, and/or providing education-related support services. Priority measures of impact for 

this strategy are: (1) the social and emotional development of youth, (2) youth literacy skills, 

and (3) youth numeracy (math) skills.6

There were 39 studies on the effectiveness of CNCS school readiness programs, including seven 

describing studies employing experimental or quasi-experimental designs. Five documents 

were selected for full review (Table 2). 

 

5

It should be noted that many eligible documents demonstrated rigorous research methods and were 

technically eligible for review, but were not primarily concerned with determining the effectiveness of 

programs or providing generalizable information. These included network analyses and qualitative 

studies. 

6 CNCS, Strategic Plan 2011-2015.

All Eligible 

Documents

N = 689

Education

N = 334 

Impact

N = 69 

Randomized 

Controlled Trial 

N = 16 

Groups Formed by 

Matching

N = 14 

Groups Formed by 

Cutoff Score

N = 1

Interrupted Time 

Series

N = 4

Comparison Groups 

– Non-equivalent

N = 34 

Outcome

N = 122 

Other

N = 143

Not concerning 

Education

N = 355
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

4

Table 2: School Readiness Studies7

Title (ID) Publication 

Type

Study Design 

Type

Strength Outcomes

Outcome Evaluation of 

the Minnesota Reading 

Corps PreK Program 

(2369)

Internal, nongovernment, 

report

QED- Groups 

formed by 

Matching 

Moderate IGDI literacy outcome 

measures 

2007-2008 Evaluation of 

Jumpstart in California 

(1357)

Internal, nongovernment, 

report

QED – Nonequivalent 

comparison 

group

Preliminary Literacy , Socialemotional 

development 

First 5 Service Corps 

Evaluation Report (1356)

Internal, nongovernment, 

report

Pre-post Preliminary Math skills ,

Literacy , 

Social competence 

Harlem GEMS Changes 

in Developmental 

Progress 2008-2009 

(1367)

Internal, nongovernment, 

report

Pre-post Preliminary Cognitive school 

readiness , Linguistic 

school readiness 

Effective Practices of 

Foster Grandparents in 

Head Start Centers 

(2034)

Internal, nongovernment, 

report

Implementation PrepreliminaryN/A 

The research base for AmeriCorps demonstrated positive results of the use of AmeriCorps 

members in programs designed to enhance school readiness. However, these evaluations 

typically did not include designs that could effectively speak to causal relationships. The 

exception to this was one rigorous quasi-experimental study of the Minnesota Reading Corps 

PreK program, which revealed statistically significant improvements on all five measured 

literacy outcomes. Three additional studies selected for full review found positive outcomes in 

literacy, math, and social-emotional development. This includes one quasi-experimental study 

on the Jumpstart program using a non-equivalent comparison group, which found positive 

impacts in literacy and social-emotional development, and two pre-post studies of the First 5 

Service Corps and Harlem GEMS programs which found positive results in multiple outcomes. 

Overall, literacy outcomes were the most likely to be measured, and frequently demonstrated 

positive results. Many studies examined multiple outcomes, including both literacy and social 

emotional development. The First 5 Service Corps study was one of few studies of school 

readiness that investigated and found positive effects in math skills. 

 

7

In Table 2 and the remaining summary tables, an upward pointing arrow indicates increases in an 

outcome, a downward pointing arrow indicates decreases in that outcome, and no arrow indicates that 

there were no detectable differences in that outcome. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

5

There were few studies of non-AmeriCorps programs. One implementation study of the Senior

Corps Foster Grandparent Program found that Foster Grandparents at Head Start Centers 

engaged in behaviors that were associated with positive effects on children’s social-emotional 

outcomes, including emotional well-being, self-esteem, and self-efficacy. 

Conclusion 

Overall, evidence for the effectiveness of CNCS programs to improve school readiness is 

preliminary, since there was only one study that could support causal conclusions. Several 

studies with less rigorous designs reported positive impacts on children’s school readiness 

skills. 

K-12 Education 

To improve educational and behavioral outcomes of students in low-achieving elementary, 

middle, and high schools, CNCS supports members, participants, and volunteers engaged in: 1) 

teaching, tutoring, and/or mentoring; 2) providing supportive services to assist school staff; 3) 

providing extended learning programs in cooperation with school systems; or 4) delivering 

other types of school improvement support. Additionally, CNCS members, participants, and 

volunteers engage in service-learning projects and provide assistance to higher education 

institutions and alternative teacher certification programs (e.g., Teach for America) to increase the 

number of teachers and teaching aids working in low-achieving schools. The priority measure 

of impact for this strategy is improved student academic performance.8

There were 229 studies examining the effectiveness of CNCS K-12 education programs in 

improving academic performance, including 54 describing studies employing experimental or 

quasi-experimental designs. Eighteen documents were selected for full review, including four 

documents focused on the Teach for America program (Table 3), nine documents focused on 

literacy programs (Table 4), and five documents describing programs aimed at improving 

student behavior and increasing academic and life skills (Table 5). There were many more 

rigorous studies in the K-12 Education focus area than in other focus areas. It appears that this is 

at least partially due to the availability of standardized test data from schools, which were often 

used to measure outcomes. 

Teach for America 

Of the 54 impact studies in the K-12 Education focus area, 37 (69%) were focused on Teach for 

America (TFA). Four documents describing three randomized controlled trials of the TFA

program were selected for full review (Table 3). 

 

8 CNCS, Strategic Plan 2011-2015.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

6

Table 3: K-12 Education Studies: Teach for America

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

The effect of Teach for 

America on the 

distribution of student 

achievement in primary 

school (170) 9

Peer-reviewed 

journal article

RCT Moderate Math achievement , 

Reading achievement

Effects of Teach For 

America on Students: 

Findings from a National 

Evaluation (1682)

Other RCT Strong Math achievement , 

Reading achievement 

Effectiveness of Secondary 

Math Teachers from Teach 

For America and the 

Teaching Fellows 

Programs. (2270)

Government 

report (e.g., 

grey literature)

RCT Strong Math test scores 

Impacts of Teach For 

America Investing in 

Innovation Scale-Up 

(2269)

Internal, nongovernment,

report

RCT Strong Reading skills (Pre-K to 

2

nd grade) , Reading 

skills overall, Math skills 

Two studies using the same data set from 2004 yielded similar results: students in the 1st

through 5th grade taught by TFA teachers performed as well as students taught by other 

teachers in reading, and outperformed students taught by other teachers in math. An increase in 

math achievement among students of TFA teachers was also found in a 2013 study of 6th

through 12th graders. However, the most recent randomized controlled trial of TFA, published 

in 2015, did not find significant differences between Pre-K through 5th grade students taught by 

TFA and non-TFA teachers in reading or math achievement, although positive results in reading 

skills were found in a subgroup analysis of Pre-K through 2nd grade students. Overall, the 

studies investigating the impact of TFA indicate that students in TFA classes generally perform 

as well or better in math than students in other classes, particularly when TFA teachers are 

compared to other novice teachers. However, TFA program impacts on reading are 

inconclusive, as some studies have found that students in TFA classes perform as well or better 

 

9 This study was performed using secondary data from a randomized controlled trial performed in 2004, 

and was published at a later date. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

7

on measures of reading10, while others have found that they perform worse than students in 

other classes.

11 TFA teachers’ high rate of attrition is also a concern for researchers.12

K-12 Literacy Programs 

The majority of non-TFA K-12 education programs were primarily designed to improve literacy 

outcomes. This review includes nine studies of the effects of national service programs on 

literacy, all of which demonstrated positive outcomes (Table 4). 

Table 4: K-12 Education Studies: Literacy

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Mobilizing Volunteer 

Tutors to Improve Student 

Literacy (2365)

Government 

report (e.g., 

grey literature)

RCT Strong Reading achievement 

Evaluation of Experience 

Corps Student Reading 

Outcomes (2492)

Internal, nongovernment,

report

RCT Strong Word attack , Passage 

comprehension , Grade 

specific reading 

Impact Evaluation of the 

Minnesota Reading Corps 

K-3 Program (2366)

Internal, nongovernment,

report

RCT Strong Literacy assessment 

scores 

Longitudinal 

Achievement Effects of 

Multiyear Summer School 

(2465)

Peer-reviewed 

journal article

RCT/QED Moderate Reading achievement , 

Learning rates, Summer 

drop-offs 

An Investigation of the 

America-Reads 

Mississippi Program: 

What Works and What 

Doesn’t Work (0843)

Internal, nongovernment,

report

Pre-Post Preliminary Reading achievement 

City Year Whole School 

Whole Child FY09 

Evaluation Report (1349)

Internal, nongovernment, 

report

Pre-Post Preliminary Oral reading fluency , 

Student attitude scores 

Evaluation of DC Reads 

(1505)

Internal, nongovernment, 

report

Pre-Post Preliminary Reading skills , SAT 9 

scores , Language arts 

grades 

 

10 Henry, G. T., Bastian, K. C., Fortner, C. K., Kershaw, D. C., Purtell, K. M., Thompson, C. L., & Zulli, R. 

A. (2014). Teacher preparation policies and their effects on student achievement. Education, 9(3), 264-303.

11 Darling-Hammond, L., Holtzman, D. J., Gatlin, S. J., & Heilig, J. V. (2005). Does teacher preparation 

matter? Evidence about teacher certification, Teach for America, and teacher effectiveness. Education 

Policy Analysis Archives, 13(42). Note: This study found decreases in both reading and math. 

12 Vasquez Heilig, J. , & Jez, S. J. (2014). Teach For America: A return to the evidence. Boulder, CO: National 

Education Policy Center.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

8

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

AmeriCorps External 

Evaluation 2008-2009 

AppleCorps (1623)

Internal, nongovernment, 

report

Pre-Post Preliminary Reading achievement 

Improving Literacy with 

VISTA Volunteers (1440)

Internal, nongovernment, 

report

Outcome 

only

PrepreliminaryEnjoyment of reading , 

Self-esteem , Family 

satisfaction , Life skills 

, Ability to perform 

tasks 

Several impact studies demonstrated improvements in literacy, including randomized 

controlled trials of the Reading Partners, Experience Corps, and Minnesota Reading Corps K-3

programs, and a quasi-experimental study of the Teach Baltimore program. Additional studies 

demonstrating positive impacts in literacy include outcome studies of America ReadsMississippi, City Year, AppleCorps and VISTA Literacy Corps. 

K-12 Education Studies: Behavior and General Academics 

A minority of the K-12 education studies examined impacts of national service programs on 

academic behaviors including attendance, and skills including academic and life skills. Five 

documents describing two quasi-experimental studies and three outcome studies were selected 

for full review (Table 5).

The five-year report of the Communities in Schools (CIS) program included a quasi-experimental 

study in which 602 CIS schools, including several that incorporated AmeriCorps members, 

were statistically matched to 602 non-CIS schools. In this study, CIS schools showed small but 

significant advantages in academic achievement and attendance. In addition to this study, the 

report included results from three randomized controlled trials, which were conducted in 

locations that did not incorporate AmeriCorps members. These showed positive results in 

academics, retention, and attendance.

A quasi-experimental evaluation of the Citizen Schools 8

th grade academy compared the longterm academic outcomes of alumni to those of matched non-participants. The Citizen Schools

program prepared middle school students for long-term success by extending the student 

learning day and offering hands-on learning opportunities. The study found that alumni were 

more than three times more likely to graduate from a top-tier high school than comparison 

students. Alumni also had improved outcomes in attendance and academic achievement, 

including higher grades in math courses and higher scores on standardized math tests.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

9

Table 5: K-12 Education Studies: Behavior and Skills 

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Communities in Schools 

National Evaluation Five 

Year Summary Report 

(2489)

Internal, nongovernment, 

report

RCT/QED Moderate Academics , 

Attendance 

Achieving High School 

Graduation: Citizen 

Schools' Youth Outcomes 

in Boston (1370)

Internal, nongovernment, 

report

QED- Groups 

formed by 

Matching

Moderate High School Selection 

and Persistence , HS 

Engagement , HS 

Academic Achievement 

Girls Incorporated of 

Alameda County 

AmeriCorps Program 

Evaluation 2008-09 (1535)

Internal, nongovernment, 

report

QED: Nonequivalent 

comparison 

group

Preliminary Attendance , Reading 

proficiency

Evaluation of New Jersey 

After 3 (1472)

Internal, nongovernment, 

report

Pre-post Preliminary Academic benefits scale 

scores 

Youth Investment Center 

AmeriCorps Evaluation 

Report (1526)

Internal, nongovernment, 

report

Pre-post Preliminary Life skills , School 

attendance , Behavioral 

outcomes 

Conclusion

Overall, evidence for the effectiveness of CNCS K-12 education programs is strong, due to the 

presence of numerous well-designed and rigorously conducted evaluations demonstrating 

positive outcomes in this area. Positive outcomes were most consistently seen in math 

achievement for Teach for America programs, and literacy outcomes for literacy tutoring 

programs. Improvements were also found in other outcomes such as attendance, but these were 

less frequently assessed and associated with more varied interventions. 

Post-secondary Preparation and Prospects for Success 

To support improved prospects and preparation for post-secondary education for economically 

disadvantaged students, CNCS members, participants, and volunteers engage in mentoring, 

counseling, and other academic, enrollment, and retention support activities. Priority measures 

of impact for this strategy are: (1) the provision of college counseling, (2) student application for 

and enrollment in a post-secondary institution, and (3) student continuation to a second year of 

post-secondary education.13

The search found 18 studies examining the effectiveness of CNCS programs designed to 

improve post-secondary preparation and prospects for success, five of which employed 

 

13 CNCS, Strategic Plan 2011-2015. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

10

experimental or quasi-experimental research designs. Three studies examining the effectiveness 

of CNCS programs in this focus area were selected for full review, including one randomized 

controlled trial, one quasi-experimental study, and one pre-post outcome study (Table 6).

Table 6: Post-Secondary Preparation and Prospects for Success Studies

Title (ID) Publication 

Type

Study 

Design Type

Strength Outcomes

Evaluation of the College 

Possible Program (2526)

Internal, nongovernment,

report

RCT Strong Enrollment in four-year 

colleges , Overall 

college enrollment 

The National College 

Advising Corps 

Evaluation Report (2442)

Government 

report (e.g., 

grey literature)

QED: Nonequivalent 

comparison 

group

Preliminary Attitudes and behaviors 

associated with postsecondary achievement 

Bay Area Youth Agency 

Consortium: Analysis of 

AmeriCorps Member and 

Youth Outcomes (1350)

Internal, nongovernment, 

report

Pre-post Preliminary Youth Development 

Asset Scale scores 

There was only one study that could support causal conclusions regarding the effectiveness of 

national service programs on post-secondary preparation and prospects for success. This was a 

randomized controlled trial of the College Possible program, which assessed the effect of the 

program on college enrollment outcomes. Although overall college enrollments did not differ 

between the treatment and control groups, students in the College Possible program were more 

likely to enroll in a four-year college than students in the control group. 

A quasi-experimental study of the National College Advising Corps employed non-equivalent 

comparison groups, and found increases in a wide variety of attitudes and behaviors associated 

with post-secondary achievement. An outcome study of the Bay Area Youth Agency Consortium

found some increase in a measure of youth assets14 that could support positive post-secondary 

outcomes.

Conclusion

Overall, evidence for the effectiveness of national service programs in post-secondary 

preparation and prospects for success is preliminary, since there was only one study that could 

support causal conclusions. Several studies were identified with less rigorous designs that 

reported positive impacts in this focus area. 

 

14 The BAYAC Youth Assets Development Scale includes measures of youth connection to community, 

ability to complete work, and other factors associated with youth success. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

11

Strength of the Evidence Base

Evidence for the effectiveness of CNCS economic opportunity programs can be classified as

follows: 

 School Readiness: Preliminary

 K-12 Education: Strong 

 Post-secondary Education and Prospects for Success: Preliminary

This brief provides a summary of the literature regarding the effectiveness of national service

education programs. In the area of school readiness, no one program was the subject of multiple 

evaluations, but the use of AmeriCorps volunteers as service providers demonstrated positive 

impacts upon literacy, math, social-emotional development, and cognitive and linguistic school 

readiness for economically disadvantaged students. In the literature describing the effect of 

national service programs on school readiness, there was one rigorous evaluation. This study of 

the Minnesota Reading Corps Pre-K program used a quasi-experimental design with matched 

comparisons and found improvements in literacy outcomes. Three additional studies 

employing less rigorous methods also demonstrated positive outcomes. The review of school 

readiness also included one implementation study of the Foster Grandparent Program, which 

noted that Foster Grandparents at Head Start Centers engaged in behaviors that were linked 

with positive outcomes in social-emotional development. 

Documents regarding the effectiveness of national service in improving K-12 education 

outcomes constituted the majority of the available literature. This focus area was by far the most 

rigorously studied. There was a great deal of research on the effectiveness of the Teach for 

America (TFA) program. More than half of the national service education impact studies were 

evaluations of the TFA program. This review described three randomized controlled trials of 

TFA. The results of these studies were published in 2004, 2013, and 2015; data from the 2004 

study also formed the basis for secondary analysis published in a later document. Studies of the

effectiveness found that students of TFA teachers generally performed as well or better in math 

than students of non-TFA teachers, while the impacts of TFA on reading achievement were 

mixed. Aside from TFA studies, the majority of remaining studies assessed literacy outcomes. 

This review included randomized controlled trials of the Reading Partners and Experience Corps

programs, and eight additional studies with varying degrees of rigor, all of which demonstrated 

a positive impact of national service programs on literacy outcomes. Finally, a minority of 

studies looked at outcomes other than reading and math achievement, such as attendance, life 

skills, and behavior. The research in this area was less conclusive, though there was one study 

of the Communities in Schools program that used a quasi-experimental design with matched 

comparisons and demonstrated improvements in attendance and academic achievement. 

The least investigated outcome area was the impact of national service programs on preparation 

and prospects for success in post-secondary education. One randomized controlled trial of the 

College Possible program found that participants were more likely than non-participants to enroll 

in a four-year college, while additional studies employing less rigorous designs found 

improvements in related areas such as attitudes and behaviors associated with post-secondary 

achievement. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

12

Overall, the literature suggests the important role of national service participants in providing 

supplementary support for disadvantaged students in schools and other education settings, 

particularly when that support is provided in the form of individualized or small group 

tutoring using a structured curriculum. Many programs in diverse settings employ national 

service members as tutors to improve literacy outcomes in K-12 students, either in after-school 

programs or other settings. Several rigorous studies have demonstrated moderate evidence for 

the success of these programs. Though the literature in other areas is less robust, elements such 

as the provision of individualized or small group support, the use of structured curricula, and 

the integration of activities into existing educational settings recur as common elements of 

several successful programs. 

The Teach for America program, which is the subject of a significant proportion of the overall 

literature, is unique in that it places national service members as teachers rather than in 

supportive roles as tutors or counselors. In this program the most robust positive results are 

found when members are serving as supplements in schools where there is a scarcity of 

qualified teachers. The strongest and most consistent evidence for the success of Teach for 

America has been found in the improvement of math outcomes among students of TFA teachers 

as compared to students of non-TFA teachers with similar levels of training and experience. 

Rigorous research in the Education focus area is aided by the strong emphasis on assessment in 

schools and other educational settings. Due to the long history of educational assessment, there 

are many validated instruments that can be used to assess a wide variety of student outcomes. 

Federal and state mandates requiring frequent assessment of math and literacy outcomes in 

public schools contribute to a large and ongoing body of data regarding student outcomes in 

these areas. Evaluators who are able to successfully access this data and collaborate with 

schools can use this data to assess the effectiveness of many interventions, as demonstrated by 

several experimental and quasi-experimental studies conducted on the Teach for America

program that made use of such data. Evaluators assessing the effectiveness of programs in

improving outcomes of K-12 students in math, literacy, and other areas that are subject to 

mandatory assessment by schools (e.g., attendance) can most easily take advantage of this large 

body of data. However, they still may face significant challenges in obtaining access to data and 

ensuring data quality, as schools may not readily share data with researchers or may have poor 

data-keeping systems that produce incomplete or flawed data sets. The existence of multiple 

assessment tools can also present challenges, particularly for multi-site evaluations that seek to 

reconcile data from different locations, as many states differ in terms of which instruments 

schools are required to use. 

Research in other focus areas, including school readiness and post-secondary preparation, as 

well as areas not subject to frequent assessment (e.g., social-emotional development, college 

readiness), typically must rely on independently collected data, or data that is collected in 

partnership with a local program or school for the specific purposes of the study. Data 

collection in settings other than K-12 schools (e.g., community-based school readiness 

programs), can be particularly difficult, as study participants may not be as consistently present 

in these settings. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

13

In addition to challenges in data collection, research in the education setting must address 

concerns related to participant diversity and homogeneity of groups (e.g., classes) during data 

analysis. Assessments that do not account for clustering of students within schools and 

classrooms, demographic and other differences at the individual, classroom, and/or community 

level have significant potential for providing incorrect or misleading results. On the other hand, 

advanced statistical techniques such as multilevel modeling to account for group-level effects or 

propensity score matching to enable valid comparisons between non-randomized groups

provide opportunities for using existing and original data to draw strong conclusions regarding 

student outcomes. Partnerships with schools and community-based organizations, which may 

be vital to program implementation, also provide significant opportunities for collaborative 

evaluation. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

14

Appendix A: Eligibility Criteria for National Service Review

1. The document is factual (versus opinion15).

o The document is referring first-hand to research findings, using original or 

secondary data collection, a synthesis of the literature, or a systematic 

review/meta-analysis.

o The document is not an Op-Ed, testimonial, or marketing material.

2. The study is relevant to CNCS-sponsored National Service or CNCS.

o The study is conducted on a program that, at the time of the study, engaged 

AmeriCorps State/National, NCCC or VISTA Members, Senior Corps Volunteers, 

or was funded by SIF or a SIF Grantee. A program refers to an ongoing service

and does not include short-term special initiative projects.

o The study is conducted on a program model that currently engages AmeriCorps 

members or Senior Corps volunteers, even if the study itself does not identify or 

include specific program sites with AmeriCorps members or Senior Corps 

volunteers (e.g., studies of Reading Partners, City Year).16

o The program’s outcomes are applicable to CNCS. The study investigates the 

effectiveness of volunteers or national service programs in the focus areas of

Education, Economic Opportunity, Healthy Futures, Environment, or Disaster 

Services and Response. 

3. The document is in English.

4. The document is a final report for the program, project or evaluation rather than an 

interim or progress report.17

 

15 Factual documents report data or other information in a manner that is subject to empirical verification. 

Opinion documents pass judgment on the value or merit of a set of facts or circumstances based on a set 

of evaluative criteria that are not strictly subject to empirical verification (e.g., based on moral or political 

considerations).

16 It was anticipated that the review would not include any study of a program that, although it could be 

adapted for national service, is not currently funded by CNCS, with the exception of evidence based 

practices that could be applied in a national service context found in online clearinghouses. 

17 Final reports summarize information about a program or intervention at the end of the program or 

intervention's life or at the end of a major cycle in the program or intervention's life (e.g., at the end of a 

funding period). By contrast, an interim report provides information on the status of a program or 

intervention at an intermediate juncture in its lifecycle. Information provided in an interim report does 

not contain complete information on results, as the program or intervention is still underway at the time 

when the report was written. Though not technically eligible for review, interim reports for SIF-funded 

programs may be discussed as emerging research, particularly in focus areas where forthcoming SIF 

studies address a gap in rigorous research. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

15

5. The document was published after 1990, but before the search kickoff date (February 

1, 2015), is a seminal document as indicated by citations in post-1990 literature, or was 

recommended for inclusion by a key stakeholder.

6. The document does not duplicate information from a previously screened source. 

Where duplicate information is identified, two sources will be listed as a single citation 

and only the latest source will be reviewed.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

16

Appendix B: Selection Process for Education Full Reviews

The following process was used to select articles for full review in this report. Reviewers:

1. Identified the subject matter of the study and confirmed its relation to the education

focus area.

2. Determined whether it was a final report or an interim report, with only final reports 

selected for inclusion in the report (although interim reports on emerging strategies are 

referred to in the appendix). 

3. Gave priority in selection to those studies that showed impact on beneficiaries with 

outcomes addressing one of CNCS’s three education objectives18: 

o Improve school readiness19 for economically disadvantaged young children;

o Improve educational and behavioral outcomes of students in low-achieving 

elementary, middle, and high schools; and 

o Improve preparation for and prospects of success in post-secondary education 

institutions for economically disadvantaged students.

4. Gave priority in selection to those studies that examined the program features or 

components that were related to successful outcomes for the beneficiaries.

5. Ascertained the level of evidence and, to the extent possible, selected those with the 

strongest level of evidence.

 

18 CNCS, Strategic Plan 2011-2015.

19 School readiness refers to preparation for kindergarten, which encompasses multiple indicators 

assessed across developmental and behavioral domains, including, but not limited to, physical wellbeing, 

health and motor development, social and emotional development, approaches to learning, language 

development, cognitive development, and age-appropriate skills and behaviors.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

17

Appendix C: Levels of Evidence

Levels of Evidence for Individual Studies

Strong evidence means the study incorporates a design that can support causal conclusions for 

the specific program with the highest level of confidence (i.e., very high internal validity). This 

would include well-designed and well-implemented experimental studies conducted on the 

proposed program.

Moderate evidence means the study can support causal conclusions for the program studied 

with moderate confidence (i.e., those that take steps to increase internal validity). This would 

include well-designed and conducted quasi-experimental studies or correlational research with 

strong statistical controls for selection bias and for discerning the influence of internal factors. 

An example of research that meets the standards would be a well-designed and wellimplemented quasi-experimental study that compares outcomes between the group receiving 

the intervention and a matched comparison group (i.e., a similar population that does not 

receive the intervention).

Preliminary evidence includes non-experimental studies such as those that seek to demonstrate 

improvement in program participants over time on one or more intended outcomes or an 

implementation (process evaluation) study used to learn and improve program operations. 

Examples of these include: 1) outcome studies that track program participants through a service 

pipeline and measure participants’ responses at the end of the program; and 2) pre- and posttest research that determines whether participants have improved on an intended outcome.

This level also includes RCTs and QEDs with significant threats to validity, such as evidence of 

selection bias. 

Pre-preliminary evidence would include studies reporting quantitative or qualitative data from 

program staff, program participants, or beneficiaries that have been used for program 

improvement, performance measurement reporting, and/or tracking. An example could be 

gathering feedback from program participants following their service year.

Overall Levels of Evidence Determinations for Focus Areas

In determining the overall level of evidence for specific focus areas (e.g., housing), we selected 

the highest level at which there were multiple studies supporting positive outcomes.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

18

Appendix D: Annotated Bibliography: Studies Selected for Full Review

School Readiness 

2369 Markovitz, C., Hernandez, M., Hedberg, E., & Silberglitt, B. (2014). Outcome evaluation of the Minnesota Reading Corps PreK 

program. Chicago, IL: NORC at the University of Chicago.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Minnesota Reading 

Corps PreK

ServeMinnesota 

Action Network

ASN NORC at the University of Chicago Minnesota

This quasi-experimental evaluation assessed whether children in the Minnesota Reading Corps PreK program exhibited greater gains in literacy 

than children in a matched comparison group. The study was performed in 25 MRC PreK sites in Minnesota and 25 matched sites. The MRC 

PreK program recruits, trains, places, and monitors AmeriCorps members, who implement evidence-based literacy interventions for at-risk 

preschool children, using a Response-to-Intervention framework.

The final sample included 1,534 students, approximately 52 percent of whom were White, 11 percent Black, seven percent Asian, eight percent

Hispanic, and nine percent Other. A small percentage of students were Dual Language Learners. Participants in the MRC PreK program had 

significantly higher scores than children in matched comparison sites on five IDGDI outcomes measures for four- and five-year-old students: 1) 

letter sound fluency, 2) rhyming fluency, 3) letter name fluency, 4) picture name fluency, and 5) alliteration fluency. Effect sizes ranged from .40 

to .72. Despite a small sample size, significant effects were also found for three-year-old participants in measures of rhyming fluency and 

picture name fluency. Overall, the study provides moderate evidence for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

19

1357 Miller, S. (2008). 2007-2008 Evaluation of Jumpstart in California. Sacramento, CA: California Jumpstart.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Jumpstart Jumpstart ASN Shelby Miller California

This quasi-experimental evaluation attempted to determine if Jumpstart participants showed greater gains than comparison group children over 

the program year on a measure of language, literacy, initiative, and social relational skill development. The sample for this study consisted of 

891 pre-school age children from 10 program sites in California, 58 percent of whom were Hispanic, and of which 25 percent were Spanish only 

speakers. Jumpstart provides an educational intervention for preschool-age children who are attending community-based, early education and

childcare programs including Head Start Centers and pre-kindergarten classes. AmeriCorps members provided an educational intervention 

focusing on language, literacy, initiative, and social skill acquisition during two-hour long sessions twice a week during the school year in 

community-based, early education, and child care centers. Members spent six additional hours a week in the children’s classrooms assisting 

teachers or working with the children.

This report on the evaluation of Jumpstart in California is based on the School Success Checklist obtained at the beginning and end of the school 

year. Results indicated Jumpstart participant groups made gains on the School Success Checklist total and its two subscales on literacy and 

social-emotional development, and the gains were significantly greater than the gains of the non-matched comparison group children. Overall, 

the study provides preliminary evidence for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

20

1356 Grant Management Associates. (2011). First 5 Service Corps evaluation report, 2009 - 2011. Durham, CA.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

First 5 Service Corps The Child Abuse 

Prevention Center

ASN Grant Management Associates California

This evaluation report provides information on operations, service deliverables, and outcomes of the First 5 Service Corps program from 

September 1, 2009 through August 31, 2011. First 5 Service Corps provide child development services for children ages two to five, including 

language and literacy, math concepts, social-emotional skills, and personal health and safety skills. Members serve in three settings: classrooms, 

family resource centers, and in homes. During the evaluation period, First 5 Service Corps served approximately 3,500 children. 

Results indicated that children who received at least 20 hours of service improved their school readiness scores from pre to post in every 

domain, including math, literacy and social competence. There were increases of 32-75 percent in those who participated for at least 20 hours 

versus decreases or increases of less than 18 percent in those who participated less than 20 hours. The report recommended enhanced data 

collection efforts and to streamline the data collection process in order to maintain uniform data sets among program years so that longitudinal 

data could be abstracted for further program enhancements. Additionally, it was suggested that future studies analyze the data more deeply to 

better inform subtle, but significant program changes. This was based on the data analysis performed by the evaluators - data definitions were 

inconsistent across program years and some data was missing or inaccurate. Overall, the study provides preliminary evidence for these 

findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

21

1367 Philliber Research Associates. (2010). Harlem GEMS changes in developmental progress 2008-2009. Accord, NY.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Harlem GEMS Harlem Children’s 

Zone 

ASN Philliber Research Associates

Accord, NY 

New York, NY

During the 2008-2009 school year, 191 children from Harlem GEMS were developmentally assessed in the fall and again in the summer. All 

children were screened using the Bracken Basic Concept Scale - Revised. Total test outcomes showed significant improvements in the children's 

post-test scores and percentile rankings overall and at all four locations (Columbia, Uptown, Head Start children aged 3 or younger and Head 

Start children aged 4), and in test classifications overall and at three of the four locations. 

School readiness outcomes show significant improvement in posttest scores and percentile rankings overall and at three of the four locations,

and readiness test classifications overall and at all four locations. The Bracken is a standardized test. As a result, GEMS results were compared 

to a national sample. Overall, the outcomes show that, at post-test, 66 percent of the children attending GEMS had total test scores ranked above 

the 50th percentile, and 96 percent had school readiness scores that were ranked above the 50th percentile. At the four locations, total test and 

school readiness scores remained high with over half of children's total test scores ranked above the 50th percentile and nine in ten or better in 

each location's school readiness scores ranked above the 50th percentile.

The outcomes in this report reflect positive achievements for these preschoolers and suggest a successful intervention by the GEMS program.

Overall, the study provides preliminary evidence for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

22

2034 Westat (1998). Effective Practices of Foster Grandparents in Head Start Centers: Benefits for children, classrooms, and centers. 

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Foster Grandparent 

Program

Corporation for 

National Service

Senior Corps Westat Northeast, South, and 

West 

The purpose of this implementation study was to learn more about what Foster Grandparents do in Head Start Centers and how their 

contributions benefit the children they serve. Evidence from the intensive observations of Foster Grandparents in six classrooms showed that 

the majority of Foster Grandparents engage in a wide variety of activities and interactions that are associated with positive developmental 

outcomes for children in five areas: (1) emotional well-being; (2) self-esteem; (3) social and behavioral skills development; (4) language 

development; and (5) cognitive development. In addition, the majority of Foster Grandparents engage in a wide range of activities and 

interactions that contribute positively at the classroom and station levels. These contributions include the following: (1) increased opportunities 

for all children in the classroom to receive individualized adult attention; (2) a calming influence on the classroom environment; (3) behind-thescenes support that facilitates smooth transitions from one activity to the next; and (4) continuity for children when teaching staff are absent or 

leave. The aforementioned contributions to children, classrooms, and stations were observed among Foster Grandparents who demonstrated a 

variety of positive caregiver behaviors and served in classrooms where certain practices were implemented. In the effective classrooms, Foster 

Grandparents demonstrated various positive caregiver behaviors. This report did not include information regarding outcomes, and is rated as 

pre-preliminary. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

23

K-12 Education

Teach For America

1682 Decker, P.T., Mayer, D.P., & Glazerman, S. (2004). The effects of Teach for America on students: Findings from a national evaluation. 

Madison, WI: Institute for Research on Poverty.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Teach for America Teach for America ASN Paul T. Decker, Daniel P. Mayer, Steven 

Glazerman

Mathematica Policy Research, Inc. 

Nationwide

This randomized controlled trial assessed the effectiveness of Teach for America (TFA) teachers on student achievement. The sample consisted of 

17 schools with 37 blocks, 100 classrooms, and nearly 1,800 students. Comparisons of outcomes of students taught by TFA teachers with 

outcomes of students taught by non-TFA, or control, teachers in the same schools and at the same grades occurred. For the analysis, “control 

teachers” included all teachers in the study who were not TFA corps members at the time of the study or at any time in the past. “TFA teachers” 

included all teachers who entered the profession through TFA—both current TFA corps members in their first two years of teaching, and 

alumni (former corps members) who were still teaching. The main source of data for this study was a set of achievement tests we administered 

in the fall (pre-test) and the spring (post-test) of the study year (2002-2003), including an abbreviated form of the mathematics and reading 

subtests of the Iowa Test of Basic Skills (ITBS).

TFA teachers had a positive impact on math achievement and no impact on reading achievement. The size of the impact on math scores was 

about 15 percent of a standard deviation, equivalent to about one month of instruction. No impacts on other student outcomes such as 

attendance, promotion, or disciplinary incidents were identified, but TFA teachers were more likely to report problems with student behavior 

than were their peers. Overall, the study provides strong evidence for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

24

170 Antecol, H., Eren, O., & Ozbeklik, S. (2013). The effect of Teach for America on the distribution of student achievement in 

primary school: Evidence from a randomized experiment. Economics of Education Review, 37, 113-125.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Teach for America 

(TFA)

Teach for America ASN Heather Antecol of The Robert Day School of 

Economics and Finance, Claremont McKenna 

College and IZA

Ozkan Eren of the Department of Economics, 

Louisiana State University

Serkan Ozbeklik of The Robert Day School of 

Economics and Finance, Claremont McKenna 

College

Baltimore, Chicago, Los 

Angeles, Houston, New 

Orleans and the 

Mississippi Delta 

This study used secondary data collected from 2001-2003 during Mathematica’s Policy Research Incorporated National Evaluation of Teach for 

America (2004). It used fixed effect quantile regression (FEQR) to examine the effects on tests scores of having a TFA teacher across the entire 

achievement distribution of primary school students (N = 1710) in disadvantaged neighborhoods. The study generally found that TFA teachers 

neither helped nor hurt students in terms of reading test scores. Positive and statistically significant effects of TFA across the math achievement 

distribution were found for the full sample and the effects were fairly uniform. A similar distributional effect of TFA was found within student 

gender, although the FEQR estimates for female students were two to three times larger than for male students. Evidence existed of 

heterogeneity in the effects of TFA for Hispanic and black students and for students taught by novice teachers. Finally, the effect of TFA was 

homogeneous across the math achievement distribution irrespective of certification type. Overall, this study provides further support for the 

strong evidence reported in Decker, Mayer, & Glazerman (2004).
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

25

2270 Clark, M.A., Hanley S.C., Silva, T., McConnell, S., Sonnenfeld, K., Erbe, A., & Puma, M. (2013). The effectiveness of secondary math 

teachers from Teach For America and the Teaching Fellows programs (NCEE 2013-4015). Washington, DC: National Center for 

Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Teach for America Teach for America ASN Melissa A. Clark , Hanley S. Chiang , Tim 

Silva , Sheena McConnell , Kathy Sonnenfeld , 

& Anastasia Erbe, Mathematica Policy 

Research

Michael Puma, Chesapeake Research 

Associates

Nationwide

This randomized controlled trial assessed the effectiveness of Teach for America secondary math teachers in improving student math 

achievement. The study examined the effectiveness of TFA teachers, comparing secondary math teachers from TFA with other secondary math 

teachers teaching the same math courses in the same schools. The TFA study sample consisted of 4,573 students, 111 classroom matches, 136 

math teachers, 45 schools, and 11 districts in eight states.

In each participating school, evaluators identified “classroom matches”—two or more classes covering the same middle or high school math 

course at the same level, with at least one class taught by a TFA teacher and at least one class taught by another teacher, referred to as a 

comparison teacher, who did not enter teaching through a highly selective alternative route. In each classroom match, students were randomly 

assigned at the beginning of the school year to a class taught by a TFA teacher or a class taught by a comparison teacher. Student math 

achievement was measured by math assessments administered at the end of the school year in which the students were randomly assigned. For 

students in grades six to eight, scores were obtained on state-required assessments. Students in grades nine to 12 took end-of-course math 

assessments developed by the Northwest Evaluation Association (NWEA).

On average, students assigned to TFA teachers scored higher (by 0.07 standard deviations) on end-of-year math assessments than students 

assigned to comparison teachers. This difference in math scores was equivalent to an increase in student achievement from the 27th to the 30th 

percentile. This difference also translated into an additional 2.6 months of school for the average student nationwide. Overall, this study 

provides strong evidence for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

26

2269 Clark, M.A., Isenberg, E., Liu, A.Y. (2015). Impacts of the Teach For America Investing in Innovation Scale-Up. Washington, DC: 

Teach for America.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Teach for America Teach for America ASN Melissa A. Clark, Eric Isenberg, Albert Y. 

Liun, Libby Makaowsky, & Marykate 

Zukiewicz

Mathematica Policy Research 

Nationwide

This randomized controlled trial assessed the effectiveness of Teach for America (TFA) teachers in improving academic outcomes, after a major 

expansion effort of the TFA program. The final sample included 10 states, 13 school districts and other TFA placement partners, 36 schools, and 

156 teachers (66 TFA and 90 comparison teachers). The sample of TFA teachers was limited to those recruited in the first two years of the scaleup, who were in their first or second year of teaching at the time of the study, whereas the comparison teachers included both novice and 

experienced teachers teaching in the same schools and grades as the TFA teachers. In total, 3,724 students were randomly assigned to classes 

and outcome test score data was obtained for 2,153 students.

In the lower elementary grades (pre-kindergarten through grade 2), students were assessed using reading and math assessments from the 

Woodcock-Johnson III achievement test. In the upper elementary grades (grades 3 to 5), in which annual reading and math assessments were 

required by the federal No Child Left Behind Act, evaluators collected state assessment data from district records. Evaluators also collected 

prior years’ test scores from state assessments when available, along with other student background characteristics. End-of-year test scores of 

students assigned to the TFA teachers and those assigned to the comparison teacher were compared.

On average, the TFA teachers in our sample were as effective as comparison teachers in both reading and math, including both novice and 

traditionally certified teachers. In both subjects, differences in test scores between students assigned to TFA teachers and those assigned to 

comparison teachers were not statistically significant. TFA teachers in lower elementary grades (pre-kindergarten through grade 2) had a 

positive, statistically significant effect on student reading achievement of 0.12 standard deviations, or about 1.3 additional months of learning,

from the average student in these grades nationwide. Overall, this study provides strong evidence for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

27

K-12 Education Programs: Literacy 

2365 Jacob, R.T., Armstrong, C., Willard, J.A., (2015). Mobilizing volunteer tutors to improve student literacy: Implementation, impacts, and 

costs of the Reading Partners program. New York, NY: MDRC. 

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Reading Partners Reading Partners ASN Robin Tepper Jacob, Institute for Social 

Research, University of Michigan

Catherine Armstrong and Jacklyn Altuna 

Willard, MDRC

California, New York, 

Washington DC

This randomized controlled trial assessed the effectiveness of Reading Partners in improving literacy outcomes. The sample for this study 

consisted of Reading Partners programs located in 19 schools in California, New York, and Washington, DC, in which Reading Partners had been 

in operation for at least one year. AmeriCorps members served as Site coordinators who oversaw instruction provided by volunteer tutors (not 

AmeriCorps) members. Tutors provided regular individualized one-to-one reading instruction utilizing a structured curriculum. AmeriCorps 

members also served as Outreach Coordinators, tasked with recruiting volunteers to staff each program site. Within each of the study schools 

and within grade groups, students in second through fifth grade who needed assistance in reading were randomly assigned to the Reading 

Partners program or to an “as-is” control condition. A total of 1,265 students were assigned, and the final sample included 1,166 students. The 

treatment and control group demonstrated baseline equivalence and did not differ in attrition. Participants in the study were in low-income 

communities and included a high percentage of minority (Hispanic) students. The study team administered three reading assessments 

measuring reading comprehension, fluency, and the ability to read sight-words efficiently.

Reading Partners had a positive and statistically significant impact on all three measures of student reading proficiency, with effect-size impacts 

of 0.10 on reading comprehension scores, 0.09 on reading fluency, and 0.11 on sight-word reading. An examination of growth between the 

beginning and end of the year on two of these three assessments shows that both groups scored higher in reading comprehension and fluency 

at the end of the year than they did at the beginning of the year, but that growth on these two assessments was greater for the program group 

than for the control group. On the sight-word reading test, there was no growth among the control group but positive growth for the Reading 

Partners group. Overall, this study provides strong evidence for these findings. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

28

2492 Center for Social Development (2009). Evaluation of Experience Corps: Student Reading Outcomes. St. Louis, MO: Washington 

University in St. Louis.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Experience Corps Experience Corps ASN Center for Social Development at Washington 

University in Saint Louis’s Brown School of 

Social Work

Boston, New York City, 

Port Author Texas

This randomized controlled trial assessed the effectiveness of the Experience Corps (EC) program in improving student literacy outcomes. 

Twenty-three schools in Boston, New York City, and Port Arthur, Texas, participated in the study. EC members provided one-to-one literacy 

tutoring using a structured curriculum to first through third grade students. Students referred by teachers as needing assistance in reading at 

the beginning of the school year were randomly assigned to the EC program or a control condition.

The total sample included 825 students. A slight majority of students were African American, and a large majority qualified for free or reduced 

lunch. Data for the study came from the following standardized reading tests: (a) the Woodcock Johnson word attack subscale (WJ-WA), (b) the 

Woodcock Johnson passage comprehension subscale (WJ-PC), and (c) the Peabody Picture Vocabulary test (PPVT-III). Referred students were 

poor readers based on the WJ-PC measure, which found that half of the students referred to EC performed as low as or lower than 84 percent of 

the students their age nationwide, and 12 percent scored worse than 97 percent of the population. At the beginning and end of the academic 

year, teachers completed assessments of grade-specific reading skills and classroom behavior. Analysis of pre-test data showed that the EC

students and control groups were equivalent on all measured characteristics.

About half of the EC students received 30 to 49 sessions, and the mean number of sessions was 45. Three-quarters of the EC students received 

over 35 sessions, which represented about one session a week throughout the program period. Students in the EC group made over 60 percent 

more progress in word attack and passage comprehension and 40 percent more progress on grade-specific reading skills. Improvements when 

compared to control group members were statistically significant, with ES scoring .13 for passage comprehension, (p=.04) and.16 for grade 

specific reading (p=.004). The group difference for word attack was marginally significant, with ES scoring .10 (p=.07). Overall, the study 

provides strong evidence for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

29

2366 Markovitz, C., Hernandez, M., Hedberg, E., & Silberglitt, B. (2014). Impact evaluation of the Minnesota Reading Corps K-3 program. 

Chicago, IL: NORC at the University of Chicago.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Minnesota Reading 

Corps (MRC) K-3

ServeMinnesota 

Action Network

ASN Carrie E. Markovitz, Ph.D., Principal Research 

Scientist, NORC at the University of Chicago

Marc W. Hernandez, Ph.D., Senior Research 

Scientist, NORC at the University of Chicago

Eric C. Hedberg, Ph.D., Senior Research 

Scientist, NORC at the University of Chicago

Benjamin Silberglitt, Ph.D., Director of 

Software Applications, TIES

Minnesota 

This randomized controlled trial assessed the effectiveness of the Minnesota Reading Corps (MRC) K-3 program in improving student literacy 

outcomes. Twenty-three schools that had fully implemented the MRC K-3 program for at least two consecutive years participated in the study. 

In this program, AmeriCorps members provide whole-class literacy enrichment for all students, and targeted small group and one-to-one 

emergent literacy interventions. Members individually tutored 15-18 students daily, providing a set of prescribed, research validated activities.

In total, 1,341 students participated, with approximately equal distribution between White, African American, and Hispanic; and approximately 

30 percent were Dual Language Learners. Each eligible student in each grade within a school was matched with another eligible student based 

upon their Fall benchmark score, each member of the pair was randomly assigned to a condition, and the matched pair was removed from the 

analytic sample if the treatment participant did not receive tutoring or if either member didn’t receive assessments. AmeriCorps members 

collected general outcome measure data using the AIMSweb literacy assessments. Kindergarten, first, and third grade students who received 

MRC tutoring achieved significantly higher literacy assessment scores than students who did not. The magnitude of MRC tutoring effects 

differed by grade, with the largest effects found among the youngest students (i.e., Kindergarten and first grade students), and the smallest 

effects among the oldest students (i.e., third grade students). Overall, the study provides strong evidence for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

30

2465 Borman, G. D., & Dowling, N. M. (2006). Longitudinal achievement effects of multiyear summer school: Evidence from the 

Teach Baltimore randomized field trial. Educational Evaluation and Policy Analysis, 28(1), 25-48.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Teach Baltimore Teach Baltimore ASN Geoffrey D. Borman & N. Maritza Dowling

University of Wisconsin-Madison 

Baltimore 

This randomized controlled trial assessed the effectiveness of the Teach Baltimore program in improving student reading outcomes. Program 

applications were distributed to all kindergarten and first-grade students in 10 high-poverty urban Baltimore schools during the spring of 1999. 

Applicants were assigned randomly to either a program or control group. In total, 293 youth were assigned to the program group, and 135 were 

assigned to the control group.

Two sets of analyses were conducted to look at first-year program effects. First, all treatment and comparison group members were compared 

according to the initial RCT design. No significant effects were found in this comparison. 

Second, program effects were then estimated for all youth in the program group with attendance rates of 75 percent or higher. The degree to 

which youth assigned to the treatment group actually attended the program varied. The evaluators used a statistical method called latent 

variable mixture modeling to determine a subset of the control group that could be considered “would-be compliers” if they had been offered 

the program treatment. Compared to statistically matched “would-be compliers”, actual treatment compliers scored 40 to 50 percent higher in 

measures of reading achievement. Overall, the study provides moderate evidence for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

31

843 Wilson, J. (2003, November 6). An investigation of the America-Reads Mississippi Program: What works and what doesn’t work. Paper 

presented at the Mid-South Educational Research Association, Biloxi, MS. Washington, D.C. Office of Educational Research and 

Improvement, U.S. Department of Education.

Intervention name: Operating 

program/

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

America Reads 

Mississippi Program

America Reads 

Challenge (ARC)

ASN Jennifer Wilson, PhD

Delta State University

Mississippi

The ARC program, part of AmeriCorps, has the primary goal of ensuring that all children could read proficiently by the end of third grade. In 

Mississippi, the ARC program is referred to as the America Reads-Mississippi (ARM) program. In 1999, 37 schools in 25 school districts 

participated in this program, sponsored by three regional centers in Mississippi. An outcome evaluation was conducted to study program 

history, examine impact on student reading achievement and student attitudes, and determine the attitudes of volunteer tutors, regional 

coordinators and assistant coordinators, and site supervisors (n=13). Findings showed that principals did not have time to serve as site 

supervisors, and program implementation varied among program sites. Standardized achievement test scores improved pre to post for 

students involved in ARM, and reading performance improved. Students in the program enjoyed reading, and tutors appeared to be beneficial 

in helping students' interest and proficiency in reading. Tutors themselves had positive attitudes about the program, although they thought 

more training would be useful. 

Recommendations put forth included:

 Schools that have certified reading specialists on staff should designate these individuals to assume the role of site supervisor. 

However, in schools that do not have certified reading specialists on staff, certified teachers with training in reading should assume this 

role.

 There is a need for continual research to determine ARM’s long-term impact on the reading achievement and attitudes of students 

toward reading. 

 For those schools that do not have structured reading programs and specific guidelines for tutors to follow, it is recommended that a 

paradigm or model of a research-based volunteer tutoring program be provided to site supervisors.

Overall, the study provides preliminary evidence for these findings. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

32

1349 Brett Consulting Group & City Year WSWC Evaluation Team (2009). City Year Whole School Whole Child FY09 Evaluation Report. 

Somerville, MA. 

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Whole School Whole 

Child 

City Year ASN Brett Consulting Group Nationwide 

This pre-post study assessed the effectiveness of the City Year program in improving academic outcomes. City Year teams provided literacy 

tutoring using a variety of curricula. The most common approach was to carry out work that teachers requested, followed by the use of a 

packaged curriculum. 

The average number of students enrolled per team in any school-based program was 143, with a mean of 106 in school-based programs and 60 

in afterschool programs. Data were collected using multiple assessment tools, including: DIBELS, DC-BAS, DRA, Dominie, Great Leaps, 

Measures of Academic Progress (MAP), Ohio State Benchmarks, and Open Court. Full data were collected from students in grades three

through five, while pilot data were collected from students in grade six through eight. This study also included an implementation component 

that assessed a variety of other outcomes at the student, teacher, and classroom level (e.g., teacher assessments of AmeriCorps members’ impact 

on students, student and teacher perceptions of City Year’s influence on school climate). 

Complete results were not included in this report. Sample results were provided at the elementary level as follows: Fifth grade students who 

received City Year tutoring improved in oral reading fluency on average by 22.4 points and positively increased their distance from the 

benchmark by end of year. Looking at the same data by student proficiency level (i.e., percentage of students at risk; some risk and low risk)

from start to end of year, the percentage of at risk students who remained at risk decreased. Overall, this study provides preliminary evidence 

for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

33

1505 Macro International Inc. (2000). Evaluation of DC Reads: Year 2 final report. Washington, DC: Corporation for National Service.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

DC Reads Corporation for 

National Service

VISTA Macro International Inc. Washington, DC

This Year 2 evaluation of DC Reads focused on the programs operated in 16 schools by Communities in Schools (CIS) during the 1998-1999 school 

year. Four of the schools served as primary study sites for the evaluation. Across its 16 sites, CIS DC Reads provided tutoring to approximately 

504 students during the 1998-1999 school year. In the four study schools, 136 students received tutoring from 74 tutors. DC Reads tutors, most of 

whom were Federal Work-Study students attending partner universities, provided tutoring to first- through third-grade students twice a week 

up to an hour each time. Tutors used a curriculum and instructional materials that the partners purchased and adapted. 

The Year 2 evaluation concentrated on assessing student outcomes and investigating program achievements. Significant gains were found on 

five of eight Tests in the Reading Performance Battery, with dramatic gains in phonemic awareness. The central element of the outcome study 

was the Reading Performance Battery, which consisted of eight individually" administered reading assessments. Most tests in the battery were 

administered three times. Between the fall and spring testing (Wave 1 and Wave 3), tutored students demonstrated statistically significant 

improvement on five of the eight measures in the Reading Performance Battery. Students improved in letter identification, word attack, and 

oral reading fluency and comprehension, and there were dramatic gains in phonemic awareness.

By using Hierarchical Linear Modeling (HLM), a statistical analysis technique, the evaluators determined there was significant variation among 

tutored students in their initial scores on measures of phonemic awareness, word identification, and passage comprehension. For each of these 

outcome measures, students who had lower scores tended to grow at a faster rate than those with higher scores. This indicates that the lowestperforming students were "catching up" in some areas. HLM also indicated that students who practiced reading aloud more frequently during 

tutoring improved at a higher rate than students who did so less frequently.

Tutored students made greater gains on the Stanford Achievement Test (SAT 9) reading tests than did non-tutored students. On Normal Curve 

Equivalent scores, tutored students gained nearly twice as many points, as non-tutored students. Also, 69 percent of tutored students improved 

on Total Reading, while just 60 percent of their non-tutored peers demonstrated improvement. Overall, this study provides preliminary 

evidence for these findings. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

34

1623 Scopatz, M. P. (2009). AmeriCorps external evaluation 2008-2009. Santa Barbara, CA: Santa Barbara County Education Office. 

Intervention name: Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

AppleCorps Santa Barbara 

County Education 

Office 

ASN Mary P. Schopatz Santa Barbara, CA 

The AppleCorps program was developed to provide one-on-one or small group reading tutoring for the target population of 845 low-level 

learners in 22 K-12 Title I schools and five educational enhancement after school programs. The external evaluator assisted program staff in the 

development of this Evaluation Plan as well as in the development of AppleCorps external evaluation plans for the previous six years. This 

external evaluation report focused on the outcomes of the Performance Measures for 2008-2009. 

Santa Barbara County Education Office determined 56 percent of K-12 students fell below proficiency on the STAR test in English Language 

Arts. The 2006- 2007 county statistics showed that 83 of 112 schools (74%) had Title l status. Further, 29 percent were English Language Learners 

and lacked the reading skills necessary to succeed in school. Additionally, 50 percent met the Federal poverty level for the Free or Reduced 

Lunch program, while 90 percent of the targeted schools sites were Title I. Stated goals for 2008-2009 were for 65 percent (549) of the targeted 

students be tutored for a minimum of 21 hours, aiming for an increase in their oral reading fluency by one proficiency level (one half grade 

level). 

All members used DIBELS (Dynamic Indicators of Basic Early Literacy Skills). Seven hundred and eighteen targeted students completed the 

program-85 percent of the 845 anticipated targeted students. Of the 718 targeted students who completed the program, 627 (87.3%) increased 

their reading ability by at least one half grade level. Of those 627 students, 272 (43%) improved by 1⁄2 grade level, 355 (57%) improved at least 

one grade level. Of the 355 students who improved at least one grade level, 181 (29%) improved more than one grade level. Through a survey, 

100 percent of site supervisors reported reading tutoring was much needed at their sites. Overall, this study provides preliminary evidence for 

these findings. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

35

1440 Development Associates, Inc. (1991). Improving literacy with VISTA volunteers: An evaluation report on the VISTA Literacy Corps. 

Arlington, VA.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS

Evaluator or evaluating organization name Location(s) of the study

VISTA Literacy 

Corps

ACTION AmeriCorps VISTA Development Associates Not reported

This report documents the results from the first evaluation of the VISTA Literacy Corps (March, 1999). The study included three major 

components: (1) a survey of 60 VISTA Literacy Corps Project Directors, 313 active VISTA Literacy Corps Volunteers, and 153 former VISTA Literacy 

Corps Volunteers; (2) site visits to 18 VISTA Literacy Corps Projects; and (3) an analysis of computer records concerning 10,184 active and inactive 

VISTA Volunteers both in the VISTA Literacy Corps and in the rest of VISTA. Effects reported by VISTA Literacy Corps Volunteer Project 

Directors on clients were increases in self-esteem (91 percent), personal enjoyment of reading (79 percent), and greater family satisfaction (72 

percent), as well as increased life skills (71 percent) and improved ability to perform tasks (70 percent). Overall, this study provides prepreliminary evidence for these findings. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

36

K-12 Education Programs: Behavior and Skills 

2489 ICF International (2010). Communities in Schools national evaluation: Five year summary report. Fairfax, VA. 

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Communities in 

Schools 

Communities in 

Schools (CIS)

ASN ICF International Texas, Florida, Georgia, 

Pennsylvania, Michigan, 

Washington , North

Carolina20

This evaluation adopted a multi-level approach that included multiple RCTs and a quasi-experimental study using propensity score matching

to assess the effectiveness of Communities in Schools (CIS). Though the three RCTs did not include sites with AmeriCorps staff, the quasiexperimental study, which included 602 CIS schools and 602 matched comparison schools, did incorporate AmeriCorps staff. 

The RCT studies found large positive improvements in retention (+ Effect Size [ES] of .7 in one site), academics (+ ES of .26-.55), and attendance 

(+ES of .45-.72.). Results from the QED, which included AmeriCorps participants, indicated that CIS students experienced small, but consistent 

improvements in performance on state-mandated assessments, with stronger gains found in math rather than reading/English. Program effects 

were stronger when programs were implemented with a high level of fidelity. The QED study also showed small improvements in attendance 

in CIS schools relative to matched comparison schools. Overall, this study provides moderate evidence for these findings. 

 

20 The locations reported were the sites for the QED study, which included programs with AmeriCorps staff. The RCTs, which did not include 

AmeriCorps staff, were conducted in Jacksonville, FL, Austin, TX, and Wichita, KS. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

37

1370 Policy Studies Associates, Inc. (2010). Citizen Schools: Achieving High School Graduation: Citizen Schools’ Youth Outcomes in Boston. 

Boston, MA. 

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Citizen Schools’ 8

th

Grade Academy

Citizen Schools AmeriCorps 

State/National

Policy Studies Associates, Inc. Boston, Mass.

This quasi-experimental study using matched comparisons assessed the impact of Citizen Schools’ 8th Grade Academy on the long-term academic 

outcomes of its participants. The study followed 8th Grade Academy participants from five different school years as they progressed through 

high school in the Boston Public School system. Citizen Schools works to prepare middle-school students for long-term academic, social, career, 

and civic success by extending the student learning day and offering hands-on learning opportunities in areas that include career exposure, 

high school and college preparation, and academic enrichment. 

Researchers matched each of the 448 participants with three similar nonparticipants, ranked by their similarity; if an alumni’s most similar 

match left the BPS, they were replaced by the next most similar match in the next year’s analysis. Although demographics varied slightly by 

class, participants were roughly three-quarters African American, and between 18 and 21 percent Hispanic; about half of participants were 

female. Close to 20 percent of participants were enrolled in special education, and three percent were enrolled in bilingual education. Primary 

outcome measures were based on data provided by the BPS, and included enrollment in a top-tier high school, attendance, suspensions, course 

grades, standardized test scores, on-time promotion, and graduation rates.

Overall, former 8th Grade Academy participants enrolled and completed all four years of high school in a top-tier school at more than three times 

the rate of matched comparison students. Former participants also had significantly higher attendance rates in high school than did matched 

non-participants, with differences ranging from an additional week of school attended in the tenth grade to an additional two and a half weeks 

attended in eleventh grade. There were no statistical differences in suspension rates between former participants and matched nonparticipants. 

Former participants were more likely than matched nonparticipants to pass math in ninth, tenth, and eleventh grade and were more likely to 

earn A’s and B’s than were their matches in their ninth- and tenth-grade math courses. Former participants were also more likely to pass the 

mathematics MCAS than the average BPS student. Overall, the study provides moderate evidence for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

38

1535 Hatchuel Tabernik and Associates (2009). Girls Incorporated of Alameda County AmeriCorps Program Evaluation 2008-09. Berkley, 

CA.

Intervention name Operating 

program/ 

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

GIRLStart Girls Incorporated 

of Alameda County

AmeriCorps 

State/National

Hatchuel Tabernik and Associates Alameda County, CA

This quasi-experimental study assessed the impact of the GIRLStart program on reading ability and school day attendance. The study included

data from 668 students at six elementary schools in the Oakland and San Leandro school districts, collected during the 2008-09 academic year. 

The 17 AmeriCorps members involved in GIRLStart provided direct service to K-3

rd graders at school sites in after-school programs, as well as 

academic support and assistance during the school day under the supervision of classroom teachers.

The study included 131 K-3

rd grade girls enrolled in GIRLStart and 537 of their female peers. GIRLStart participants were 48 percent AfricanAmerican, 29 percent Hispanic, and 18 percent Asian-American; demographic data from their peers was not reported. Researchers collected 

attendance records and standardized test scores from the students’ school districts, and conducted site visits to five of the six schools. Reading 

proficiency was measured with California Standard Test-English Language Arts (CST-ELA) results, available only for the 2nd and 3rd graders, 

and from district benchmark tests, which varied by year and by school district.

CST-ELA results indicated that a greater proportion of participants than non-participants were reading proficient in the second and third grade, 

although this difference was not statistically significant. Additionally, almost two-thirds of participants sustained reading proficiency from the 

second into the third grade. Moreover, a higher proportion of participants sustained reading proficiency when compared to non-participants –

although again, this difference is not statistically significant. Participants did, however, have significantly higher rates of school attendance than 

non-participants. Overall, the study provides preliminary evidence for these findings.
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

39

1472 Policy Studies Associates, Inc. (2009). Evaluation of New Jersey After 3: Reaching and Engaging New Jersey's Youth Through After 

School Programs, 2005-2008. New Brunswick, NJ. 

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

New Jersey After 3 New Jersey After 3 ASN Karen P. Walking Eagle

Tiffany D. Miller

North Cooc

Jennifer LaFleur

Elizabeth R. Reisner

Policy Studies Associates, Inc

New Jersey 

Policy Studies Associates, Inc., conducted a comprehensive evaluation of New Jersey After 3, a statewide afterschool initiative. The report, based 

on data collected in program year 2007-08, assessed the extent to which the initiative was making progress toward achievement of the following 

goals: (1) Expanding the availability of afterschool services in New Jersey; (2) Enhancing the quality of afterschool services; (3) Creating 

sustainable local programs that are financially stable and of high quality; and (4) Promoting the health and the emotional, social, and 

intellectual development of New Jersey’s children. In 2007-08, data was collected through the following activities, all of which had also been 

employed in prior years of the study: (1) Data from NJ After 3’s management information system; (2) Surveys of executive directors; (3) Surveys 

of site coordinators; and (4) Surveys of student participants. From a sample of programs that began operations in 2004-05 (known as the indepth study sample), additional data was collected in the form of surveys of teachers, as well as interview and observation data. 

Survey data from 955 youth participants (72% of 1,326 total), who were majority African American and Hispanic and from low socioeconomic 

homes, indicated students who participated in the program for three years versus one year demonstrated higher gains on an academic benefits 

scale. Recommendations for the future included: (a) encouraging sites to share resources, (b) continuing to help sites tap community resources, 

(c) continuing to build systems for nurturing relationships between programs and school, and (d) taking additional steps to increase interest in 

and access to professional development among staff who work directly with youth. Overall, this study provides preliminary evidence of these 

findings. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

40

1526 LPC Consulting Associates, Inc. (2009). Youth Investment Center AmeriCorps evaluation report: Program years 2007-2009. Report 

prepared by Sacramento, CA: Child Abuse Prevention Council of Sacramento, Inc.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Youth Investment 

Center (YIC)

Child Abuse 

Prevention Council 

of Sacramento, Inc.

ASN LPC Consulting Associated, Inc.

Sacramento, CA 

Sacramento, CA

The evaluation of the Youth Investment Center (YIC) Program addressed two measures of change: (1) improved life skills, and (2) promotion 

and/or graduation. Since FY 07/08, YIC Program has been using the Ansell Casey Life Skills Assessment (ACLSA), with its online data entry 

feature and reporting capabilities. The participants in the YIC Program complete the ACLSA at baseline as an integral component of the intake 

process shortly after enrollment and at intervals after they meet the criteria for being a core participant. For foster youth, the definition of "core 

participant" was 30 days of participation with at least three hours of life skills activities per day. For non-foster youth, the core participant had

30 days of participation with at least two hours of life skills activities each day.

The evaluator obtained school promotion and graduation data from the participating schools (via the Districts) at the end of each school year, 

most often in June or July. At the end of the school year, the YIC Project Manager provided a roster of all students who have participated in the 

YIC Program to the evaluator, who in turn submitted a request to the school districts for attendance, behavior incidents reporting, and grade 

promotion and graduation outcomes for the core participants in the YIC. The number of students included in recorded data varied, but overall,

among the sample participating students, there were demonstrated increases in measured life skills, school attendance, decreases in behavioral 

outcomes, and 96 percent of participants were promoted to the next grade level. Overall, this study provides preliminary evidence for these 

findings. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

41

Post-secondary Preparation and Prospects for Success

2526 Avery, Christopher (2013). Evaluation of the College Possible Program: Results from a Randomized Controlled Trial Cambridge, MA: 

National Bureau of Economic Research.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

College Possible College Possible ASN Christopher Avery

Harvard Kennedy School of Government

Minneapolis and St. Paul 

Minnesota

This randomized controlled trial assessed the effectiveness of the College Possible program in increasing students’ applications and enrollments 

to post-secondary institutions. The College Possible program provided two years of college preparatory work for high school juniors and seniors, 

including SAT and ACT test preparation services, college admission and financial aid consulting, and guidance in the transition to college. The 

study included 239 students from eight high schools

Students applied to participate in the program in their sophomore year. Since the program had only 800 places and 900 qualified applicants, the 

last 101 were admitted to the program on a randomized group-by-group basis designed to admit a predetermined number from each high 

school. The remaining students were placed on a wait list, and 33 were admitted from this waitlist in a separate randomization procedure to fill 

new spots. There were slight differences between the first and second treatment groups in terms of household income and program 

participation. The majority of the overall sample was female (60%) and Hmong (60%). College Possible attendees were not significantly more 

likely to enroll in college, but were 15 percent more likely to enroll in four-year colleges than control group members. Overall, this study 

provides strong evidence for these findings. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

42

2442 Evaluation and Assessment Solutions for Education, LLC (2013). National College Advising Corps 2011-2012 Evaluation Report. 

Module 1: The NCAC Difference. Chapel Hill, NC: The University of North Carolina at Chapel Hill.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

National College 

Advising Corps 

National College 

Advising Corps

SIF & ASN Eric Bettinger, Antonio L. Antonio, Brent 

Evans, Jesse Foster, Brian Holzman, Hoori 

Satikian, Stanford University

Eileen Homg, EASE – Evaluation and 

Assessment Solutions for Education 

Nationwide

This quasi-experimental study examined the differences between post-secondary preparation in students who visited National College Advising 

Corps (NCAC) advisers and a non-equivalent comparison group of students who did not, according to a self-reported survey. Of the 72,883 

students nationwide who were invited to participate in the survey, 30,546 (42%) responded. These responses represented 168 schools across 

nine states and 13 partner institutions. Slightly more than half of the respondents were female, and 76 percent were first-generation college 

goers. Overall, 72 percent of the sample came from an underrepresented minority group (i.e., Black, Hispanic, Other, or Multicultural), 

including 38 percent identifying as Hispanic. The survey primarily targeted seniors who were making college decisions. Students were 

surveyed in April and May of 2012, and were asked about their college plans, their academic preparation, and sources of information they 

received about college. 

Results indicated that when compared to those who have not met with the NCAC adviser at their school, seniors who had met with the NCAC

adviser were: 42 percent more likely to apply to a college/university; 73 percent more likely to apply to a four-year institution of higher 

education; 67 percent more likely to be accepted to a college/university; and 84 percent more likely to be accepted to a four-year institution of 

higher education. Additional positive outcomes were found in attitudes and behaviors related to post-secondary preparation and success. There 

were greater effects for first-generation college students who met with an NCAC advisor than others. Since this study used non-equivalent 

comparisons, causality cannot be determined, as student characteristics likely contributed to differences in outcomes. Overall, this study 

provides preliminary evidence for these findings. 
STATE OF THE SCIENCE NATIONAL SERVICE EDUCATION INITIATIVES

August 19, 2015

43

1350 Bay Area Youth Agency Consortium. (2010). Analysis of AmeriCorps member and youth outcomes. Richmond, CA.

Intervention name Operating 

program/

organization name

Relationship to 

national 

service/CNCS 

Evaluator or evaluating organization name Location(s) of the study

Bay Area Youth 

Agency Consortium 

(BAYAC)

CNCS and 

California 

Volunteers 

ASN Public Profit California, Bay Area

The Bay Area Youth Agency Consortium (BAYAC) is an intermediary organization that collaborates with over 40 community service organizations 

to serve youth and families in all six counties of the San Francisco Bay Area. Programs included in this report are funded by two grant sources. 

The study included AmeriCorps Member and participant data from both grants during 2010. Participant characteristics were gathered through 

the Youth Development Asset Scale, a 20-item scale comprised of Asset and Support scales completed three times per program year by Members 

for 4,710 participants. The major evaluation question was: “How do youth change while participating in BAYAC programs?”

Findings revealed youth participants who were initially rated as "Proficient" or "Advanced" on the Youth Development Asset Scale had 

comparable scores in the mid-year and end-of year ratings, suggesting that these youth sustained high levels of assets and supports while in the 

program. Participants who initially scored "Developing" or lower on the Youth Development Asset Scale made consistent improvements 

throughout the program year. The average Round 1 score for this group was 2.51. By Round 2, the average for this group was 3.09, rising to 3-49 

by Round 3. When inter-Round scores were disaggregated based on participants' initial score, it was found that improvements in Scales among 

youth initially rated low were driving the overall results. Overall, this study provides preliminary evidence for these findings.