Page 1

Updated Findings from the HHS Teen Pregnancy Prevention Evidence 

Review: April 2013 Through July 2014

February, 2015 

By Brian Goesling, Julieta Lugo-Gil, Joanne Lee, and Timothy Novak 

Mathematica Policy Research

Since 2009, the U.S. Department of Health and Human Services (HHS) has sponsored an 

ongoing systematic review of the teen pregnancy prevention research literature to help identify 

programs with evidence of effectiveness in reducing teen pregnancy, sexually transmitted 

infections (STIs), and associated sexual risk behaviors. The HHS Teen Pregnancy Prevention 

(TPP) Evidence Review was created in response to fiscal year (FY) 2010 Consolidated 

Appropriations Act legislation indicating that funded teen pregnancy prevention programs must 

have been “proven effective through rigorous evaluation to reduce teenage pregnancy, behavioral 

risk factors underlying teenage pregnancy, or other associated risk factors.” Mathematica Policy 

Research and Child Trends conduct the TPP Evidence Review, which is sponsored by the Office 

of the Assistant Secretary for Planning and Evaluation (ASPE), the Office of Adolescent Health 

(OAH) within the Office of the Assistant Secretary for Health, and the Family and Youth 

Services Bureau (FYSB) within the Administration for Children and Families (ACF). 

As of summer 2014, the review team, which is comprised of trained staff from Mathematica and 

Child Trends, had identified 35 programs meeting the review criteria for evidence of 

effectiveness. These criteria require programs to show evidence of at least one favorable, 

statistically significant program impact on at least one sexual behavior or reproductive health 

outcome of interest (sexual activity, contraceptive use, STIs, pregnancy, or birth). In addition, 

the supporting research studies must meet established criteria for the quality and execution of 

their research designs. The review team follows a pre-specified set of criteria in order to assess 

study design, attrition, baseline equivalence, reassignment of sample members, and confounding 

factors (see Table 1). To identify the 35 programs meeting these criteria, the review team 

identified and assessed over 200 studies released from 1989 through April 2013. 

The review team has recently updated the findings for this review to cover more recent research

published or released from April 2013 to July 2014. As part of this update, the review team 

identified two new programs meeting the review criteria for evidence of effectiveness, bringing 

the total number of programs meeting the review criteria for evidence of program effectiveness

to 37—the 35 programs from earlier rounds of the review and the two newly identified programs.

As discussed later in this brief, the review team also sought to identify and assess any newly 

available evidence for programs highlighted in previous rounds of the review.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 

Page 2

Newly Identified Programs

Two new programs met the review criteria for evidence of effectiveness:

1. Get Real. Get Real is a school-based, comprehensive sex education program for middle 

and high school students. The Get Real middle school curriculum is a three-year program 

comprising a total of 27 lessons—9 lessons delivered in each of the 6th, 7th, and 8th 

grades. At each grade level, the program also provides eight family activities for students 

to complete at home with their parents or guardians. The Get Real high school curriculum 

is a one-year program comprising eight required and three optional lessons for 9th or 10th 

grade students. The Get Real middle school curriculum was evaluated in a cluster 

randomized controlled trial involving 2,453 students from 24 middle schools in the 

greater Boston area.

http://www.etr.org/get-real/).

1

 Researchers found that students in schools that delivered the 

program were significantly less likely to initiate sexual activity by the end of 8th grade. 

The study met the review criteria for a “moderate” quality rating. The Get Real high 

school curriculum was not part of the study. Get Real was developed by Planned 

Parenthood League of Massachusetts and is currently distributed by ETR 

(

1 Grossman, J.M., Tracy, A.J., Charmaraman, L., Ceder, I., & Erkut, S. (2014). Protective effects of middle school 

comprehensive sex education with family involvement. Journal of School Health, 84(11), 739-747. 2 Sieving, R.E., McRee, A., Secor-Turner, M., Garwick, A.W., Bearinger, L.H., Beckman, K.J., McMorris, B.J., & 

Resnick, M.D. (2014). Prime Time: Long-term sexual health outcomes of a clinic-linked intervention. Perspectives 

on Sexual and Reproductive Health, 46(2), 91-100. 

2. Prime Time. Prime Time is a youth development program for adolescent females at high 

risk for teen pregnancy. The program involves a combination of one-on-one case 

management and group-based youth leadership activities delivered over an 18-month 

period. In a randomized controlled trial involving 253 sexually active adolescent females 

in Minneapolis and St. Paul, researchers found that adolescents participating in the 

intervention were significantly more likely to report having been sexually abstinent in the 

past six months.2 The study met the review criteria for a “high” quality rating. The study 

authors also reported statistically significant program impacts on measures of condom 

and contraceptive use. However, these findings were not considered for the review 

because the sample was restricted on the basis of sexual activity at follow-up. As detailed 

in the review protocol, the review considers evidence for only the full analytic sample or 

subgroups defined by (1) gender or (2) sexual experience at baseline. Prime Time was 

developed by researchers from the University of Minnesota. The program is still under 

development and not currently available for broader dissemination.

New Evidence for Previously Reviewed Programs

The recent update to the review findings also sought to identify and assess any newly available 

evidence for programs highlighted in previous rounds of the review. To date, most teen 

pregnancy prevention programs have been evaluated only once. However, a growing number of 

 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 

3 As part of the update to the review findings, the 

review team identified and assessed newly available evidence for two programs highlighted in 

previous rounds of the review: 

Page 3

studies have sought to test how these programs perform when implemented on a broader scale, in 

different settings, or with different populations.

3 Goesling, Brian. (forthcoming). Making Sense of Replication Studies: Guidance for the Field of Teen Pregnancy 

Prevention Research. Washington, DC: U.S. Department of Health and Human Services, Office of the Assistant 

Secretary for Planning and Evaluation. 4 Tortolero, S. R., Markham, C. M., Fleschler Peskin, M., Shegog, R., Addy, R. C., Escobar-Chavez, S. L., & 

Baumler, E. (2010). It’s Your Game: Keep It Real: Delaying sexual behavior with an effective middle school 

program. Journal of Adolescent Health, 46(2), 169–179. 5 Markham, C.M., Tortolero, S.R., Fleschler Peskin, M., Shegog, R., Thiel, M., Baumler, E.R., Addy, R.C., EscobarChaves, S.L., Reininger, B., & Robin, L. (2012). Sexual risk avoidance and sexual risk reduction interventions for 

middle school youth: A randomized controlled trial. Journal of Adolescent Health, 50, 279–288. 6 Markham, C.M., Peskin, M.F., Shegog, R., Baumler, E.R., Addy, R.C., Thiel, M., Escobar-Chaves, S.L., Robin, L., 

& Tortolero, S.R. (2014). Behavioral and psychosocial effects of two middle school sexual health education 

programs at tenth-grade follow up. Journal of Adolescent Health, 54, 151–159. 7 Kirby, D., Barth, R.P., Leland, N., & Fetro, J.V. (1991). Reducing the risk: Impact of a new curriculum on sexual 

risk-taking. Family Planning Perspectives, 23, 253-263. 8 Reyna, V.F., & Mills, B.A. (2014). Theoretically motivated interventions for reducing sexual risk taking in 

adolescence: A randomized controlled experiment applying fuzzy-trace theory. Journal of Experimental Psychology, 

143, 1627-1648. 

1. It’s Your Game...Keep It Real (IYG). IYG is a 24-lesson school-based program for 

middle school students. The efficacy of the program was first established in a 2010 study 

involving 3,007 7th-grade students in Southeast Texas.4 In a more recent study, 

researchers conducted a cluster randomized controlled trial involving 1,258 students from 

15 urban middle schools in a large south-central U.S. school district. The study findings 

for the 9th-grade follow-up show that students participating in the standard risk reduction 

version of the intervention were significantly less likely to initiate sexual activity or 

report having unprotected sex, and reported significantly lower frequency of vaginal and 

anal sex in the past three months.5 More recently released findings from the longer-term 

10th-grade follow-up, however, show a mix of positive, negative, and null effects.6 The 

study met the review criteria for a “moderate” quality rating.

2. Reducing the Risk (RtR). RtR is a 16-lesson comprehensive sex education program for 

primarily high-school-aged students. The efficacy of the program was first established in 

a 1991 study involving 758 high-school students in northern California.7 In a more recent 

study, researchers conducted a multi-group randomized controlled trial testing both the 

original RtR program and an adapted version of the program (RtR+) that places greater 

emphasis on the bottom-line summary or “gist” of program messages.8 The study 

involved a racially and ethnically diverse sample of 734 high-school-aged students from 

southern Arizona, northern Texas, and central New York. The study met the review 

criteria for a “moderate” quality rating. The study findings show no statistically 

significant impact on sexual risk behaviors of the original RtR program, relative to a 

control group. The study findings for the adapted version of the program (RtR+) did not 

 
Page 4

meet the review criteria because participants in this group differed from the control group 

in average age at baseline.

For these and other programs with more than one supporting research study, the review team is 

currently developing standards to support the comparison and synthesis of findings across 

studies. This type of cross-study synthesis is expected to become a growing focus of future 

updates to the review findings. 

Review Procedures

This update to the review findings followed the same procedures and criteria used for the prior 

update to the review. In September 2014, the review team released a public call for studies 

requesting new research to consider. The team also identified studies through a comprehensive 

literature search, which entailed keyword searches of electronic databases and hand searches of 

relevant academic journals. The identified studies were then screened against pre-specified 

eligibility criteria.

For studies that meet the eligibility criteria, teams of trained reviewers assessed each study for 

the quality and execution of its research design. As a part of this assessment, the reviewers

assigned each study a quality rating of high, moderate, or low according to the risk of bias in the 

study's impact findings. A more detailed description of these ratings is provided online at 

http://tppevidencereview.aspe.hhs.gov/ReviewProtocol.aspx. 

For studies that passed this assessment with either a “moderate” or “high” quality rating, the 

review team extracted information on the program tested, evaluation setting, study sample, and 

research design. The review team also extracted detailed information on the program impact 

estimates. On the basis of this information, the review team identified programs with evidence of 

effectiveness, defined as having a statistically significant favorable impact (and no adverse 

effects) on at least one priority outcome measured for either the full analytic sample or a 

subgroup defined by (1) gender or (2) sexual experience at baseline. The priority outcomes 

include: sexual activity, contraceptive use or consistency of use, STIs, or pregnancy or birth. 

More detailed information on these programs and the supporting research evidence is available 

online at http://tppevidencereview.aspe.hhs.gov. The website also provides more detailed 

information on the review process and criteria, and a complete listing of all studies included in 

this update to the review. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov 
Page 5

Table 1. Summary of HHS Teen Pregnancy Prevention Evidence Review Study Quality 

Ratings 

Criteria Category High Study Rating Moderate Study 

Rating

Low Study Rating

1. Study design Random or 

functionally random 

assignment

Quasi-experimental 

design with a 

comparison group; 

random assignment 

design with high 

attrition or 

reassignment

Does not meet 

criteria for high or 

moderate quality 

rating 

2. Attrition What Works 

Clearinghouse 

standards for overall 

and differential 

attrition

No requirement Does not meet 

criteria for high or 

moderate quality 

rating

3. Baseline 

equivalence 

Must control for 

statistically 

significant baseline 

differences

Must establish 

baseline equivalence 

of research groups 

and control for 

baseline outcome 

measures

Does not meet 

criteria for high or 

moderate quality 

rating

4. Reassignment Analysis must be 

based on original 

assignment to 

research groups

No requirement Does not meet 

criteria for high or 

moderate quality 

rating

5. Confounding 

factors

Must have at least 

two subjects or 

groups in each 

research group and no 

systematic 

differences in data 

collection methods 

Must have at least 

two subjects or 

groups in each 

research group and 

no systematic 

differences in data 

collection methods 

Does not meet 

criteria for high or 

moderate quality 

rating

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

http://tppevidencereview.aspe.hhs.gov