Page 1

Updated findings from the HHS Teen Pregnancy Prevention Evidence 

Review: July 2014 through August 2015

April 2016 

By Julieta Lugo-Gil, Amanda Lee, Divya Vohra, Katie Adamek, 

Johanna Lacoe, and Brian Goesling 

Mathematica Policy Research

Since 2009, the U.S. Department of Health and Human Services (HHS) has sponsored an 

ongoing systematic review of teen pregnancy prevention research to identify programs with 

evidence of effectiveness in reducing teen pregnancy, sexually transmitted infections (STIs), and 

associated sexual risk behaviors. The HHS Teen Pregnancy Prevention (TPP) Evidence Review 

was created in response to the 2010 Consolidated Appropriations Act, which indicates that teen 

pregnancy prevention programs must be “proven effective through rigorous evaluation to reduce 

teenage pregnancy, behavioral risk factors underlying teenage pregnancy, or other associated risk 

factors.” Mathematica Policy Research conducts the TPP Evidence Review, which is sponsored 

by the Office of the Assistant Secretary for Planning and Evaluation (ASPE), the Office of 

Adolescent Health (OAH) within the Office of the Assistant Secretary for Health, and the Family 

and Youth Services Bureau (FYSB) within the Administration for Children and Families (ACF).

As of February 2015, Mathematica had identified 37 programs meeting the review criteria for 

evidence of effectiveness. These criteria require programs to show evidence of at least one 

favorable, statistically significant impact on at least one sexual risk behavior or reproductive 

health outcome of interest (sexual activity, number of sexual partners, contraceptive use, STIs, or 

pregnancy). In addition, the supporting research studies must meet established criteria for the 

quality and execution of their research designs. The review team follows pre-specified criteria to 

assess study design, sample attrition, baseline equivalence, reassignment of sample members, 

and confounding factors. We detail the review procedures later in this brief.

Mathematica has recently updated the review findings to cover research released from July 2014 

to August 2015. As part of this update, the review team identified and assessed evidence for 16 

new programs that prior rounds of the review did not include. Seven of these 16 programs met 

the review criteria for evidence of effectiveness, bringing the total number of programs meeting 

this criteria to 44 (37 programs from earlier rounds of the review plus the 7 newly identified 

programs). The review team also identified and assessed newly available evidence for 7 

programs highlighted in previous rounds of the review. We discuss this evidence later in this 

brief.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 
Page 2

Newly identified programs

The 16 newly assessed programs (Table 1) ranged from broad classroom-based curricula for 

middle school students to specialized initiatives to prevent rapid repeating pregnancies among 

teen mothers. Seven of these 16 programs met the review criteria for evidence of effectiveness, 

showing evidence of a favorable, statistically significant program impact on at least one sexual 

risk behavior or reproductive health outcome of interest (as designated by + in Table 1). For 6 

other programs, the supporting impact studies met the review criteria for a study quality rating of 

High or Moderate, but the study findings did not meet the review criteria for evidence of 

effectiveness (as designated by 0 and ─ in Table 1). For the remaining 3 programs, the 

supporting impact studies did not meet the review criteria for a High or Moderate study quality 

rating.

Table 1. Newly identified programs: evidence by outcome domain and study rating

Outcome domains

Program

Sexual 

activity

Number of 

sexual partners

Contraceptive 

use STIs Pregnancy

Study 

ratinga

AIM 4 Teen Moms 0 0 + Moderate

Generations + Moderate

Healthy Futures + Moderate

Love Notes + 0 + + Moderate

Positive Potential Be The Exception 

Grade 6 + Moderate

Positive Prevention PLUS + + 0 Moderate

Teen Options to Prevent Pregnancy 

(T.O.P.P.) 0 0 + High

Gender Matters (GEN.M) 0 0 High

HealthTeacher 0 Moderate

It’s Your Game-Tech 0 Moderate

mCircle of Life 0 Moderate

Teen Prevention Education Program 

(TeenPEP) 0 0 0 Moderate

Sexuality Education Initiative 

(Planned Parenthood of LA) 0 0 ─ 0 High

Integrated Prevention Intervention 

for Homeless Youth n.a. n.a. n.a. n.a. n.a. Low

PHAT Life n.a. n.a. n.a. n.a. n.a. Low

South Carolina Parents Involved in 

Education Abstinence Education 

Program

n.a. n.a. n.a. n.a. n.a. Low

Note: 0, +, and – indicate evidence of null, favorable, and adverse effects, respectively. Empty cells indicate that the study did not 

examine program impacts on measures within that outcome domain.

a The review team established ratings for the supporting impact studies following pre-specified criteria to assess study design, 

attrition, baseline equivalence, reassignment of sample members, and confounding factors. See Table 3 below for a more 

detailed description.

n.a. = not applicable; following the procedures specified in the review protocol, the review team did not assess evidence of 

effectiveness for programs with a supporting impact study that received a Low quality rating.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 
Page 3

The seven new programs meeting the review criteria for evidence of effectiveness are as follows: 

AIM 4 Teen Moms. AIM 4 Teen Moms is a positive youth development program for new teen 

mothers. The program was adapted from an existing evidence-based teen pregnancy prevention 

program that features a classroom-based youth development curriculum for middle school 

students. The adapted AIM 4 Teen Moms program comprises nine sessions administered over 12 

weeks (seven one-hour home visits and two 90-minute group sessions at a community-based 

location). AIM 4 Teen Moms was evaluated in a randomized controlled trial involving 800 lowincome, adolescent new mothers in Los Angeles County.1 The study found that about nine 

months after the end of the program, teen mothers participating in the intervention were 

significantly less likely than those in the study control group to report having had sex without 

using an effective contraceptive method in the past three months. The study met the review 

criteria for a Moderate quality rating. 

Generations. Generations is a clinic-based, comprehensive health care program for adolescent 

parents and their families. The program provides teen mothers and their children with (1) 

medical care, including access to reproductive health services and individualized consultations

on sexual risk behavior, decision making, and contraception; (2) comprehensive social work 

services, including discussions with a social worker and reminder phone calls or text messages; 

and (3) mental health screening and treatment. The evaluation of the Generations program2 used

a quasi-experimental comparison group study with a sample of 124 teen mothers and their 

children in Washington, DC. Teen mothers and children in the comparison group received 

standard community-based pediatric primary care. The study authors found that teen mothers 

who participated in the Generations program were significantly more likely to report having used 

an effective method of birth control and having used a condom the last time they engaged in 

sexual activity. 

Healthy Futures. Healthy Futures is a school-based, comprehensive sex education program for 

middle school students. The three-year program features a relationship education curriculum, NuCULTURE, which is delivered in 24 lessons (8 per year in 6th, 7th, and 8th grade). At each 

grade level, the program also provides access to virtual classrooms, after-school and summer 

programs, and a website and workshops designed for parents. In a cluster randomized controlled 

trial involving 2,346 students from 15 middle schools in three cities in northeastern 

Massachusetts, 3 researchers found that at the end of 8th grade, female adolescents in the schools 

1 Covington, R. et al. “Interim Impacts of the AIM 4 Teen Moms Program.” Princeton, NJ: Mathematica Policy 

Research, 2015.

2 Lewin, A., S. Mitchell, and M. Boudreaux. “Improved Contraceptive Use Among Teen Mothers in a FamilyCentered Medical Home.” Unpublished manuscript. University of Maryland, 2015.

3 Calise T.V., W. Chow, and K.F. Doré. JSI Research & Training Institute, Inc. “Evaluation of Healthy Futures in 

Three Northeastern Massachusetts Cities: Findings from an Innovative Teen Pregnancy Prevention Program.” Final 

Impact Report for The Black Ministerial Alliance of Greater Boston, Inc. Prepared for the Office of Adolescent 

Health, U.S. Department of Health and Human Services, 2015.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 

 
Page 4

that delivered the intervention were significantly less likely to report ever having vaginal sex. The 

study met the review criteria for a Moderate quality rating.

Love Notes. Love Notes is a healthy relationship education curriculum consisting of 13 one-hour 

sessions. The program educates youth about healthy relationships and preventing dating violence 

and unprotected sex. The Love Notes program was evaluated in a cluster randomized trial 

involving 933 adolescents recruited from community-based organizations serving youth in lowincome areas of Louisville, Kentucky.

4

 The study found that six months after the program, 

adolescent participants were significantly less likely than youth in the study control group to 

report ever having had sex, ever having been pregnant, having had sex in the last three months, 

and having had sex without a condom or without birth control in the last three months. The study 

found no evidence of statistically significant program impacts on these outcomes for the study 

follow-ups conducted 3 and 12 months following the program. The study met the review criteria 

for a Moderate quality rating. 

Positive Potential Be The Exception Grade 6. Positive Potential Be The Exception (Positive 

Potential) is a school-based, youth development program developed primarily for adolescents 

attending middle school in rural communities. The Positive Potential program is provided as a 

supplemental program to the health and physical education curricula adolescents receive as part 

of their middle school education. The program offers five 45- to 50-minute classroom sessions 

on consecutive days during the 6th grade and one class assembly at the end of 6th grade. The 

program was evaluated in a randomized controlled trial involving 1,438 6th grade students in 14 

public middle and elementary schools in northwestern Indiana.5

 The study found that in schools 

that delivered the program both the full sample of students and the subgroup of males were 

significantly less likely to have had sexual intercourse (ever and in the last three months) at the 

beginning of the 7th grade. The study met the review criteria for a Moderate quality rating. 

Positive Prevention PLUS. Positive Prevention PLUS is a school-based, sex education program 

for high school students. The program consists of 11 lessons of 45 minutes each, provided during 

the school day in science, health, or physical education classes. Positive Prevention Plus was 

evaluated in a randomized controlled trial involving 3,490 9th grade students in 21 public high 

schools in southern California.6

 The study found that six months after the end of the program, 

students in the schools that offered the program were significantly less likely to initiate sexual 

activity and to have had sex in the last three months without using birth control. The study met 

the review criteria for a Moderate quality rating. 

4 Cunningham, M.R., M.A. van Zyl, and K.W. Borders. “Evaluation of Love Notes and Reducing the Risk in 

Louisville, Kentucky.” Final Evaluation Report to the University of Louisville Research Foundation. Louisville, 

KY, 2016. 5 Piotrowski, Z.H., and D. Hedeker. “Evaluation of the Positive Potential Be The Exception Grade 6 Program in 

Predominantly Rural Communities: Findings from an Innovative Teen Pregnancy Prevention Program.” Report to 

the Office of Adolescent Health, U.S. Department of Health & Human Services, August 2015.

6 LaChausse, R. “Evaluation of the Positive Prevention PLUS Teen Pregnancy Prevention Program.” Washington, 

DC: U.S. Department of Health and Human Services, Office of Adolescent Health, 2015. TP2AH000007.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 

 
Page 5

Teen Options to Prevent Pregnancy (T.O.P.P.) T.O.P.P. is a clinic-based intervention for 

pregnant and parenting adolescent females. The T.O.P.P. program is delivered individually to 

program participants over 18 months through telephone calls from trained nurse educators. The 

program promotes healthy birth spacing and use of effective contraception, and it provides direct 

access to contraceptive services through a program clinic and access to a program social worker. 

The program was evaluated in a randomized controlled trial involving 493 low-income expectant 

or adolescent new mothers in the Columbus, Ohio area. 7 The evaluation found that six months 

after study enrollment, adolescents participating in the intervention were significantly less likely 

to report having had sex without using birth control in the past three months. The study met the 

review criteria for a High quality rating. 

New evidence for previously reviewed programs

The recent update to the review findings also sought to identify and assess any new evidence for 

programs highlighted in previous rounds of the review. To date, most teen pregnancy prevention 

programs have been evaluated only once. However, a growing number of studies have sought to 

test how these programs perform when implemented on a broader scale, in different settings, or 

with different populations. 8 As part of the update to the review findings, the review team 

identified and assessed newly available evidence for seven programs highlighted in previous 

rounds of the review: (1) Children’s Aid Society (CAS)–Carrera program, (2) ¡Cuídate!, (3) 

Horizons, (4) It’s Your Game, (5) Reducing the Risk, (6) Safer Sex, and (7) Teen Outreach 

Program (TOP). Table 2 summarizes the evidence for these programs, which we further detail in 

the remainder of this section. 

7 Smith, K., et al. “Interim Impacts of the Teen Options to Prevent Pregnancy Program.” Princeton, NJ: Mathematica 

Policy Research, 2015.

8

 Goesling, Brian. “Making Sense of Replication Studies: Guidance for the Field of Teen Pregnancy Prevention 

Research.” Washington, DC: U.S. Department of Health and Human Services, Office of the Assistant Secretary for 

Planning and Evaluation, 2015. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 

 
Page 6

Table 2. Previously reviewed programs: evidence by outcome domain and study 

rating

Outcomes

Program/study 

Sexual 

activity

Number of 

sexual partners

Contraceptive 

use STIs Pregnancy

Study 

ratinga

Children’s Aid Society 

(CAS)–Carrera

Philliber et al. (2002) + + High

Tucker (2015) + 0 Moderate

¡Cuídate! 

Villarruel et al. (2006) + + + High

Abt Associates (2015c) 0 -- High

Horizons 

DiClemente et al. (2009) + + High

DiClemente et al. (2014) + + + High

It’s Your Game 

Tortolero et al. (2010) + Moderate

Markham et al. (2012); Markham et 

al. (2014) + -- + Moderate

Coyle et al. (2015a) -- 0 Moderate

Coyle et al. (2015b) 0 Moderate

Reducing the Risk 

Kirby et al. (1991) 0 + 0 Moderate

Zimmerman et al. (2008) + Moderate

Reyna and Mills (2014) 0 0 0 Moderate

Abt Associates (2015a) 0 0 High

Cunningham et al. (2016) + 0 + 0 High

Safer Sex

Shrier et al. (2001) + 0 Moderate

Abt Associates (2015b) + + + High

Teen Outreach Program 

(TOP)

Allen et al. (1997) + High

Daley et al. (2015) + + Moderate

Francis et al. (2015) 0 0 Moderate

Note: 0, +, and – indicate evidence of null, favorable, and adverse effects, respectively. Empty cells indicate that the study did not 

examine program impacts on measures within that outcome domain.

a The review team established the rating of the supporting impact studies following pre-specified criteria to assess study design, 

attrition, baseline equivalence, reassignment of sample members, and confounding factors. See Table 3 for more details. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 
Page 7

Children’s Aid Society (CAS)–Carrera program. The CAS–Carrera program is a 

comprehensive, multi-component youth development program for middle and high school 

students. The efficacy of the program was first established in a 2002 study involving 484 

adolescents recruited from community-based agencies in New York City.9 The study found that 

three years after study enrollment, female adolescents participating in the program were less 

likely to be pregnant or report being sexually active. In a more recent study, researchers 

evaluated the Carrera program using a quasi-experimental design involving 204 students in 6th 

and 7th grade in Georgia.10 Researchers found that 12 months after enrollment, adolescents 

participating in the Carrera program were less likely than those in the study comparison group to 

report having initiated sexual activity. The study met the review criteria for a Moderate quality 

rating.

¡Cuídate! ¡Cuídate! is a six-lesson culturally tailored sexuality education program for Latino 

adolescents. The efficacy of the program was first established in a 2006 study involving 684 

adolescents in an after-school setting in northeast Philadelphia.11 The study found favorable 

impacts of the program on measures of sexual activity, number of sexual partners, and 

contraceptive use. A more recent study evaluated the program with a randomized controlled trial 

in three study sites involving 2,022 adolescents attending grades 8 to 12 in middle and high 

schools in Arizona, California, and Massachusetts.12 In two of the study sites, ¡Cuídate! was 

delivered in public school classrooms during the regular school day. In the third study site, the 

program was delivered in a variety of settings, including public school classrooms during the 

regular school day or after school, and in summer programs offered by community-based 

organizations. The study pooled data across the three study sites and found no evidence of 

positive, statistically significant program impacts on measures of sexual risk behavior. In 

addition, the study authors found evidence of one adverse effect: six months after study 

enrollment, adolescents in the treatment group who were sexually active at baseline were more 

likely to report they had sexual intercourse without a condom in the last 90 days, as compared 

with adolescents in the control group who were also sexually active at baseline. The study met 

the review criteria for a High rating.

Horizons. Horizons is a clinic-based STI/HIV prevention intervention for African American 

adolescent females, delivered through two four-hour small group sessions and 15-minute booster 

phone calls over the following year. The efficacy of Horizons was first established in a 2009 

9 Philliber, S., J. Williams Kaye, S. Herrling, and E. West. “Preventing Pregnancy and Improving Health Care 

Access Among Teenagers: An Evaluation of the Children’s Aid Society–Carrera Program.” Perspectives on Sexual 

and Reproductive Health, vol. 34, no. 5, 2002, pp. 244–251. 10 Tucker, T. “Evaluation of the Carrera Program: Findings from the Replication of an Evidence-Based Teen 

Pregnancy Prevention Program.” Atlanta, GA: Tressa Tucker and Associates, 2015. 11 Villarruel, A.M., J.B. Jemmott, and L.S. Jemmott. “A Randomized Controlled Trial Testing an HIV Prevention 

Intervention for Latino Youth.” Archives of Pediatrics & Adolescent Medicine, vol. 160, no. 8, 2006, pp. 772–777. 12 Abt Associates. “¡Cuídate!: Interim Impact Report, Teen Pregnancy Prevention Replication Study.” Report 

prepared for the Office of Adolescent Health and the Office of the Assistant Secretary for Planning and Evaluation, 

U.S. Department of Health and Human Services, September 2015c. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 

 
Page 8

study involving 715 African American adolescent females in Atlanta, Georgia.

13 The study 

found favorable impacts of the program on measures of STIs and condom use. In a more recent 

study, researchers examined the effectiveness of supplementing the Horizons program with a 

telephone counseling prevention maintenance intervention (PMI).14 PMI consists of 10-minute 

tailored phone calls provided by health educators every eight weeks over 36 months, for a total 

of 18 calls. In the study, researchers conducted a randomized controlled trial involving 429 

African American female adolescents recruited from health clinics in Atlanta, Georgia. The 

study authors report that at the 36-month follow-up, female adolescents who received the 

supplemental PMI were more likely to have condom-protected sex and less likely to test positive 

for chlamydial infection in the last 90 days and in the last six months. In addition, the study 

authors report that female adolescents in the treatment group had fewer sexual partners than 

study participants in the control group in the last 6 months. The study met the review criteria for 

a High rating.

It’s Your Game...Keep It Real (IYG). IYG is a 24-lesson, school-based program for middle 

school students. The efficacy of the program was first established in a 2010 study involving 

3,007 7th-grade students in Southeast Texas.15 In a subsequent study, the same group of 

researchers conducted a cluster randomized controlled trial involving 1,258 students from 15 

urban middle schools in a large south-central U.S. school district. The study findings for the 9thgrade follow-up replicated the findings produced in the initial study: students in the treatment 

schools were less likely than those in the control schools to report having initiated sexual 

activity.16 In a more recent study conducted by a separate group of researchers, IYG was 

evaluated in a cluster randomized controlled trial involving 2,487 students from 24 rural middle 

schools in South Carolina. The study found no evidence of favorable program effects for the 8th 

grade follow-up. In addition, for the 9th grade follow-up, the study reports evidence of an 

adverse effect: students in the treatment schools were more likely than those in the control 

schools to report having initiated sexual activity.17 The study met the review criteria for a 

13 DiClemente, R.J., G.M. Wingood, E.S. Rose, J.M. Sales, D.L. Lang, A.M. Caliendo, J.W. Hardin, and R.A.

Crosby. “Efficacy of Sexually Transmitted Disease/Human Immunodeficiency Virus Sexual Risk-Reduction 

Intervention for African American Adolescent Females Seeking Sexual Health Services.” Archives of Pediatric & 

Adolescent Medicine, vol. 163, no. 12, 2009, pp. 1112–1121. 14 DiClemente, R.J., G.M. Wingood, J.M. Sales, J.L. Brown, E.S. Rose, T.L. Davis, D.L. Lang, A. Caliendo, and 

J.W. Hardin. “Efficacy of a Telephone-Delivered Sexually Transmitted Infection/Human Immunodeficiency Virus 

Prevention Maintenance Intervention for Adolescents: A Randomized Clinical Trial.” JAMA Pediatrics, vol. 168, 

no. 10, 2014, pp. 938–946.

15 Tortolero, S.R., C.M. Markham, M. Fleschler Peskin, R. Shegog, R.C. Addy, S.L. Escobar-Chavez, and E. 

Baumler. “It’s Your Game: Keep It Real: Delaying Sexual Behavior with an Effective Middle School Program.” 

Journal of Adolescent Health, vol. 46, no. 2, 2010, pp. 169–179.

16 Markham, C.M., S.R. Tortolero, M. Fleschler Peskin, R. Shegog, M. Thiel, E.R. Baumler, R.C. Addy, S.L.

Escobar-Chaves, B. Reininger, and L. Robin. “Sexual Risk Avoidance and Sexual Risk Reduction Interventions for 

Middle School Youth: A Randomized Controlled Trial.” Journal of Adolescent Health, vol. 50, 2012, pp. 279–288.

17 Coyle, K., S.C. Potter, J.R. Glassman, and L. McDade-Montez. “Process and Impact Evaluation of Keep It Real 

South Carolina: Reducing Teen Pregnancy Among Middle School Youth.” ETR Associates, Report prepared for the 

Office of Adolescent Health, U.S. Department of Health & Human Services, August 2015a. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 

 
Page 9

Moderate quality rating. In a separate recent study, researchers conducted a cluster randomized 

controlled trial involving 1,912 students from 20 urban middle schools in Harris County, Texas. 

The study found no evidence of favorable effects on student sexual risk behaviors at the end of 

the 9th grade.18 The study met the review criteria for a Moderate quality rating.

Reducing the Risk (RtR). RtR is a 16-lesson comprehensive sex education program primarily 

for high school-aged students. The efficacy of the program was first established in a 1991 study 

involving 758 high school students in northern California.19 The study found that female 

adolescents participating in the program who were sexually inexperienced at baseline were 

significantly less likely to report having had unprotected sex. Two separate subsequent 

studies20,21 examined the effectiveness of the original RtR program and of modified versions of 

the program. In a more recent study, researchers conducted a cluster randomized trial involving 

2,689 students in 150 classrooms in public middle, junior high, and high school in three study 

sites in Missouri, Texas, and California, respectively.

22 The study examined program impacts 

based on the pooled data across the three study sites and found no statistically significant RtR 

program impacts on student sexual risk behaviors. Although site is not a subgroup included in 

the TPP Evidence Review, the study also examined program impacts in each of the study sites 

separately. From the analyses by site, the study found evidence of a statistically significant 

program impact in the Missouri site: significantly fewer students in the treatment group than in 

the control group engaged in sexual intercourse in the last 90 days. The study met the review 

criteria for a High quality rating. In a separate recent study, researchers conducted a cluster 

randomized controlled trial involving 939 students who were offered the RtR curriculum in

community-based organizations located in Louisville, Kentucky.

23 The study found that three 

months after the program ended, adolescents in the treatment group were significantly less likely 

to report ever having had sex or having been pregnant in the last three months. In addition, six 

months after the program ended, adolescents in the treatment group were significantly less likely 

to report having had sex without birth control in the last three months. The study found no 

statistically significant program impacts on sexual behavior outcomes for the follow-up 

18 Coyle, K., P. Anderson, B.A. Laris, T. Unti, H. Franks, and J. Glassman. “Evaluation of It’s Your Game...Keep It 

Real in Houston, TX” Final report. Scotts Valley, CA: ETR Associates, 2015b. 

19 Kirby, D., R.P. Barth, N. Leland, and J.V. Fetro. “Reducing the Risk: Impact of a New Curriculum on Sexual 

Risk-Taking.” Family Planning Perspectives, vol. 23, 1991, pp. 253–263. 

20 Zimmerman, R.S., P.K. Cupp, L. Donohew, C.K. Sionean, S. Feist-Price, and D. Helme. “Effects of a SchoolBased, Theory-Driven HIV and Pregnancy Prevention Curriculum.” Perspectives on Sexual and Reproductive 

Health, vol. 40, no. 1, 2008, pp. 42–51. 21 Reyna, V.F., and B.A. Mills. “Theoretically Motivated Interventions for Reducing Sexual Risk Taking in 

Adolescence: A Randomized Controlled Experiment Applying Fuzzy-Trace Theory.” Journal of Experimental 

Psychology. vol. 143, no. 4, 2014, pp. 1627–1648. 22 Abt Associates. “Reducing the Risk: Interim Impact Report, Teen Pregnancy Prevention Replication Study.”

Report prepared for the Office of Adolescent Health and the Office of the Assistant Secretary for Planning and 

Evaluation, U.S. Department of Health and Human Services, September 2015a. 

23 Cunningham, M.R., M.A. van Zyl, and K.W. Borders. “Evaluation of Love Notes and Reducing the Risk in 

Louisville, Kentucky.” Final Evaluation Report to the University of Louisville Research Foundation, Louisville, 

KY, 2016. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 

 
Page 10

conducted 12 months after the program ended. The study met the review criteria for a High 

quality rating.

Safer Sex. Safer Sex is an individual intervention for female adolescents and young adults that is 

delivered one-on-one through a 30- to 50-minute session with a health educator, followed by three 

10- to 30-minute follow-up sessions over the following six months. The efficacy of the program 

was first established in a 2001 study involving women younger than 24 who sought treatment for 

cervicitis or pelvic inflammatory disease at an urban children’s hospital adolescent clinic and who 

were not pregnant at the time of the visit.

24 In a more recent study, researchers conducted a 

randomized controlled trial involving 1,809 young women who sought treatment in one of three 

study sites (health clinics) in Florida, Minnesota, and Tennessee.25 The study pooled the data 

across the three study sites and reports that nine months after baseline, young women who were 

assigned to the treatment condition were less likely to report having sexual intercourse without 

birth control in the last 90 days than young women assigned to the control condition. In addition, 

nine months after baseline, the study reports that among the subgroup of young women who were 

sexually inexperienced at baseline, those assigned to the treatment condition were less likely to 

report having initiated sexual activity or having had more than one lifetime sexual partner. The 

study also examined program impacts by study site, and found that in the Minnesota site, 

significantly fewer adolescents in the treatment group than in the control group reported 

engaging in oral sex in the last 90 days. The study met the review criteria for a High quality 

rating.

Teen Outreach Program (TOP). TOP is a youth development program that incorporates 

weekly curriculum-guided lessons, community service learning, and positive adult guidance and 

support. TOP is provided in school, after-school, and community settings to adolescents 12 to 17 

years old. The efficacy of this program was first established in a 1997 study involving 695 high 

school students attending grades 9 to 12 in high schools in 25 cities across the United States.26

The study found that female adolescents participating in the program were significantly less 

likely to report a pregnancy during the academic year of the program. In two more recent studies, 

researchers examined the effectiveness of TOP based on cluster randomized trials. In the first 

study, involving 2,058 students in grades 9 to 12 in 26 high schools in Florida, researchers found 

that immediately after the intervention, students in the schools that provided TOP were less 

likely to report ever having sex or ever getting pregnant (or getting someone pregnant) than 

24 Shrier, L.A., R. Ancheta, E. Goodman, V.M. Chiou, M.R. Lyden, and S.J. Emans. “Randomized Controlled Trial 

of a Safer Sex Intervention for High-Risk Adolescent Girls.” Archives of Pediatrics & Adolescent Medicine, vol. 

155, no. 1, 2001, pp. 73–79.

25 Abt Associates. “Safer Sex Intervention: Interim Impact Report, Teen Pregnancy Prevention Replication Study.”

Report prepared for the Office of Adolescent Health and the Office of the Assistant Secretary for Planning and 

Evaluation, U.S. Department of Health and Human Services, September 2015b. 

26 Allen, J.P., S. Philliber, S. Herrling, and G.P. Kuperminc. “Preventing Teen Pregnancy and Academic Failure: 

Experimental Evaluation of a Developmentally Based Approach.” Child Development, vol. 68, no. 4, 1997, pp. 729–

742.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 

 
Page 11

students in schools assigned to the control condition.27 The study authors also found that the 

favorable impacts on pregnancy persisted 10 months after the intervention ended. The study met 

the review criteria for a Moderate quality rating. The second study involved 1,196 students 

attending 61 middle and high schools in Hennepin County, Minnesota.28 For the two follow-ups 

conducted 3 and 15 months after the program ended, the study authors found no evidence of

statistically significant program effects on student sexual risk behaviors. The study met the 

review criteria for a Moderate quality rating.

Review procedures

This update to the review findings followed procedures similar to those used for prior rounds of 

the review. In July 2015, the review team released a public call for studies requesting new 

research to consider. The team also identified studies through a comprehensive literature search 

that entailed keyword searches of electronic databases and manual searches of relevant academic 

journals. The identified studies were then screened against pre-specified eligibility criteria. 

For studies that met the eligibility criteria, the Mathematica review team assessed the quality and 

execution of each study’s research design. The reviewers assigned each study a quality rating of 

High, Moderate, or Low according to the risk of bias in the study's impact findings. A more 

detailed description of these ratings is provided in the review protocol available online at the 

review website.29 

For studies that achieved a Moderate or High quality rating, the review team extracted 

information on the program tested, evaluation setting, study sample, and research design, as well 

as detailed information on the program impact estimates. The review team then identified 

programs with evidence of effectiveness, defined as having a statistically significant favorable 

impact (and no adverse effects) on at least one priority outcome measured for either the full 

analytic sample or a subgroup defined by (1) gender or (2) sexual experience at baseline. The 

priority outcomes include sexual activity, number of sexual partners, contraceptive use, STIs, 

and pregnancy. 

Within these general procedures, this update to the review findings introduced two changes to 

the review criteria:

27 Daley, E.M., E.R. Buhi, W. Wang, A. Singleton, R. Debate, S. Marhefka, et al. “Evaluation of Wyman’s Teen 

Outreach Program® in Florida: Final Impact Report for Florida Department of Health.” Findings from the 

Replication of an Evidence-Based Teen Pregnancy Prevention Program, 2015. 28 Francis, K., M. Woodford, and M. Kelsey. “Evaluation of the Teen Outreach Program in Hennepin County, MN: 

Findings from the Replication of an Evidence-Based Teen Pregnancy Prevention Program.” Cambridge, MA: Abt 

Associates, 2015.

29 The review protocol is available at http://tppevidencereview.aspe.hhs.gov/ReviewProtocol.aspx. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 

 
Page 12

1. New eligibility criteria. To remain eligible for the review, programs must have at least 

one impact study conducted within the last 20 years. For any program that meets this 

requirement, evidence from all studies related to the program are considered for the 

review. However, programs for which the only impact study is more than 20 years old 

will now be excluded from the review. This “moving window” is designed to keep the 

review findings current and encourage continued research on established programs. 

2. Outcome-specific assessments of program effectiveness. Starting with this current 

update to the review findings, the review team is modifying the way in which the 

research evidence is presented. This update to the review findings will specifically 

indicate each program’s impact for each of five outcome domains: (1) sexual activity, (2) 

number of sexual partners, (3) contraceptive use, (4) STIs, and (5) pregnancy. As before, 

for each program eligible for review, the review team first assesses the program’s effects 

on measures of sexual activity, then conducts a separate assessment of the program’s 

effects on measures of number of sexual partners, then conducts a separate assessment for 

contraceptive use, and so on. Using this approach, a program may be identified as having 

favorable impacts on one type of outcome but null or no evidence for other outcomes. 

What is new is that these outcome-specific assessments will be available on the review 

website to help users of the review better understand the nature of the evidence 

supporting different programs.

More detailed information on the review process and criteria is available on the review website.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 
Page 13

Table 3. Summary of study quality ratings

Criteria category High study rating Moderate study

rating

Low study rating

Study design Random or functionally 

random assignment

Quasi-experimental design 

with a comparison group; 

random assignment design 

with high attrition or 

reassignment

Does not meet criteria for 

High or Moderate quality 

rating

Attrition What Works Clearinghouse 

standards for overall and 

differential attrition

No requirement Does not meet criteria for 

High or Moderate quality 

rating

Baseline

equivalence

Must control for statistically

significant baseline 

differences

Must establish baseline 

equivalence of research 

groups and control for 

baseline outcome measures

Does not meet criteria for 

High or Moderate quality 

rating

Reassignment Analysis must be based on 

original assignment to 

research groups

No requirement Does not meet criteria for 

High or Moderate quality 

rating

Confounding

factors

Must have at least two 

subjects or groups in each 

research group and no 

systematic differences in 

data collection methods

Must have at least two 

subjects or groups in each 

research group and no 

systematic differences in 

data collection methods

Does not meet criteria for 

High or Moderate quality 

rating

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov