Evaluation Report Brief

Children’s Aid Society: Carrera InSchool Adolescent Pregnancy 

Prevention Program 

Office of Research and Evaluation, Corporation for National and Community Service January 2017 

nationalservice.gov/research 1 

What are the goals of the program? 

Although teen birth rates have declined in the United States over the 

last 20 years, teen pregnancy remains a challenge for youth in 

underserved communities. When young people become parents 

before they are ready, the impact can be devastating for the teens 

themselves, their families, and larger society given the interconnected 

challenges of poverty, limited educational attainment, low 

employment rates, substance use and abuse, and overall poor health. 

More difficult to quantify are the lost educational and vocational 

opportunities that teen parents and their children will experience. 

The CAS-Carrera program is an evidence-based, primary adolescent 

pregnancy prevention program. The goal is to help young girls and 

boys avoid becoming mothers and fathers during the second decade 

of their lives. The program is guided by a philosophy that sees youth as "at promise" and not "at risk”. Its holistic, “above 

the waist” approach seeks to develop a participant’s capacity and desire to not only avoid early pregnancy, but to break the 

cycle of poverty and despair. The program model works with boys and girls beginning in middle school, through high school 

and college completion; it is implemented in one of two ways: 1. as a traditional after-school, youth development approach; 

or, 2. integrated within a public or charter school day working with succeeding grades each year until the entire school is 

receiving the model. CAS-Carrera’s program provides participants with seven integrated, scientifically accurate, and ageappropriate components including: daily academic enrichment; weekly exposure to the “world of work”/Job Club; weekly 

Power Group/mental health services; weekly family life and sexuality education (FLSE); comprehensive medical and dental 

services as needed; as well as regularly scheduled lifetime individual sports and self-expression. 

What was the purpose of evaluation?

CAS-Carrera engaged an independent evaluator, Philliber Research 

Associates, to evaluate the efficacy of the integrated school model 

version of the program in both New York City and Tulsa, Ok. The afterschool version of CAS-Carrera as demonstrated through random 

assignment, was shown to have a range of positive youth development 

outcomes; this study was designed to build on both the original RCT 

and the qualitative study published on the integrated school model. 

What did we learn from the evaluation? 

The implementation evaluation found: 

 The program was implemented as intended, although

implementation varied slightly between Tulsa and New York

City as well as across schools within each city.

 The program participants were youth in middle and high school grades from underserved backgrounds.

Program At-a-Glance 

CNCS Program: Social Innovation Fund

Intervention: Carrera Program

Subgrantee: Children’s Aid Society (CAS) 

Intermediary: Edna McConnell Clark Foundation 

Focus Area: Healthy Futures

Focus Population: Youth in grades 6th through 12th

in underserved communities 

Communities Served: Tulsa, OK, and New York 

City, NY 

Evaluation At-a-Glance 

Evaluation Design: Implementation and propensity 

score matched between groups impact evaluation 

Study Population: Students in grades 6 through 8 in 

Tulsa and grades 9 through 12 in New York 

Independent Evaluator: Philliber Research & 

Evaluation 

This Evaluation’s Level of Evidence*: Preliminary 

*SIF and AmeriCorps currently use different definitions of levels of evidence.
Evaluation Report Brief

Children’s Aid Society: Carrera InSchool Adolescent Pregnancy 

Prevention Program

Office of Research and Evaluation, Corporation for National and Community Service J

nationalservice.gov/research 2 

 Program staff were adequately trained and were qualified, but they experienced challenges in running the

program in-schools (compared to after school). While school administrators expressed support for the program,

program staff faced logistical challenges related to implementing school programs within the context of school

operations.

The impact evaluation found: 

 In Tulsa, 9th grade youth in the program were significantly less likely to report ever having sexual intercourse,

scored 27% higher on a scale measuring family life and sexuality knowledge, more likely to report having had an

eye exam, having a bank account, and higher financial literacy compared to a comparison group of 9th graders

who did not participate. There were no reported pregnancies during the course of this study.

 In New York City, program participating youth reported lower pregnancy rates, greater condom use, less physical

fighting, and lower levels of smoking, marijuana, and alcohol use. This finding is based on a comparison to a

matched sample of Youth Risk Behavioral Survey data and should be considered exploratory.

 Other findings suggest that the CAS-Carrera in-school model is strong enough to saturate a school. Interviews in

NYC show consistent reports of schoolwide changes as a result of the program such as adoption of the FLSE

curriculum at one site throughout the school; high awareness among non-program youth of program’s services and

how to get them; and reports that non-program youth received services from CAS-Carrera staff. While this

confounded the study, this is a positive indicator of the potential of this integrated model. The in-school version of

the program can be delivered at lower cost than the after-school version given the larger number of youth served

in the in-school model, and can reach more students/be scaled in a manner not possible in the after school version.

The in-school model reaches a larger number of youth than a typical after-school program and serves students who

might not normally join an after-school program.

Notes on the evaluation 

While the evaluation showed early evidence that implementing the Carrera Program during the school day may be 

effective, there were some limitations to the study design. This included a small sample size in Tulsa, and in NYC the 

comparison group received services due to the program saturating the entire school context, which confounded the 

lagged cohort design. For these reasons, the results of this research are exploratory and suggestive of the model’s 

potential. 

How is CAS using the evaluation experience and findings to improve? 

The in-school version of the program can be delivered at lower cost given the larger number of youth served in the inschool model. However, there are challenges to this delivery strategy in the student-to-staff ratio and adaptability to the 

school context (i.e., scheduling, logistical, and administrative demands of the school). Since the in-school model allows for 

greater access to youth, Carrera has added program resources to meet these challenges and to facilitate better linkages 

with students’ teachers.

The content of this brief was drawn from the full evaluation report submitted to CNCS by the grantee/subgrantee. The section of the brief that 

discusses evaluation use includes contribution of the grantee/subgrantee. All original content from the report is attributable to its authors.

To access the full evaluation report and learn more about CNCS, please visit nationalservice.gov/research.

The Social Innovation Fund (SIF), a program of the Corporation for National and Community Service (CNCS), combines public and private 

resources to grow the impact of innovative, community-based solutions that have compelling evidence of improving the lives of people in lowincome communities throughout the U.S. The SIF invests in three priority areas: economic opportunity, healthy futures, and youth development.