Page 1

Updated findings from the HHS Teen Pregnancy Prevention Evidence 

Review: August 2015 through October 2016 

April 2018

By Julieta Lugo-Gil, Amanda Lee, Divya Vohra, Jessica Harding, 

Lindsay Ochoa, and Brian Goesling 

Mathematica Policy Research

Since 2009, the U.S. Department of Health and Human Services (HHS) has sponsored an 

ongoing systematic review of teen pregnancy prevention research to identify programs with 

evidence of effectiveness in reducing teen pregnancy, sexually transmitted infections (STIs), and 

associated sexual risk behaviors. The HHS Teen Pregnancy Prevention (TPP) Evidence Review 

was created in response to the 2010 Consolidated Appropriations Act, which indicates that teen 

pregnancy prevention programs must be “proven effective through rigorous evaluation to reduce 

teenage pregnancy, behavioral risk factors underlying teenage pregnancy, or other associated risk 

factors.” Mathematica Policy Research conducts the TPP Evidence Review, which is sponsored 

by the Office of the Assistant Secretary for Planning and Evaluation (ASPE), the Office of 

Adolescent Health (OAH) within the Office of the Assistant Secretary for Health, and the Family 

and Youth Services Bureau (FYSB) within the Administration for Children and Families (ACF).

As of June 2016, Mathematica had identified 44 programs meeting the review criteria for 

evidence of effectiveness. These criteria require programs to show evidence of at least one 

favorable, statistically significant impact on at least one sexual risk behavior or reproductive 

health outcome of interest (sexual activity, number of sexual partners, contraceptive use, STIs, or 

pregnancy). In addition, the supporting research studies must meet established criteria for the 

quality and execution of their research designs. The review team follows pre-specified criteria to 

assess study design, sample attrition, baseline equivalence, reassignment of sample members, 

and confounding factors. We detail the review procedures later in this brief.

Mathematica has recently updated the review findings to include research released from August 

2015 to October 2016. As part of this update, the review team identified and assessed evidence 

for 15 programs not previously reviewed. Three of these 15 newly identified programs met the 

review criteria for evidence of effectiveness. The review team also identified and assessed newly 

available evidence for 25 programs assessed in previous rounds of the review. One of those 25 

programs met the review criteria for evidence of effectiveness for this first time. Therefore, the 

total number of programs meeting the review criteria for evidence of effectiveness is now 48: 44 

programs identified in earlier rounds of the review plus the 4 programs identified in the most 

recent update to the review findings. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 2

Newly identified programs 

The 15 newly identified programs (Table 1) varied in terms of their focus, target population, 

setting, and content. Three of these 15 programs met the review criteria for evidence of 

effectiveness, having a High or Moderate study rating and showing evidence of a favorable, 

statistically significant program impact on at least one sexual risk behavior or reproductive health 

outcome of interest (as designated by + in Table 1). A High or Moderate study rating is necessary 

for the findings to be deemed sufficiently credible evidence of program effectiveness. 

For nine other programs, the supporting impact studies met the review criteria for a study quality 

rating of High or Moderate, but the study findings did not show favorable, statistically significant 

program impacts. For the remaining three programs, the supporting impact studies did not meet 

the review criteria for a High or Moderate study quality rating. 

Nine of the 15 newly identified programs received funding from FYSB or from the OAH TPP 

program. 

Table 1. Newly identified programs: evidence by outcome domain and study rating

 Outcome domains 

Program

Sexual 

activity

Number 

of sexual 

partners

Contraceptive 

use STIs Pregnancy

Study 

ratinga

FYSB or 

OAH

fundedb

Programs with 

evidence of 

effectiveness

 

Crossroads + DNMS High Yes

POWER Through 

Choices 0 0 0 0 + High Yes

Possessing Your 

Power + Moderate Yes

Study quality 

meets standards, 

but no positive 

effects 

 

BUtiful 0 0 0 High Yes

Family oriented 

HIV/STI reduction 

intervention 

integrated within 

multidimensional 

family therapy 

(MDFT-HIV)

DNMS DNMS 0 High No

Respecting the 

Circle of Life 0 High No

Work It Out 

Together

0 0 High No

Be Yourself/Sé Tú 

Mismo 0 0 Moderate Yes

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 3

 Outcome domains 

Program

Sexual 

activity

Number 

of sexual 

partners

Contraceptive 

use STIs Pregnancy

Study 

ratinga

FYSB or 

OAH

fundedb

Health Education 

Counseling DNMS Moderate No

Need to Know (N2K) 0 0 0 Moderate Yes

Pono Choices 0 0 Moderate Yes

Web of Life 0 Moderate Yes

Study Quality Does

not meet evidence 

standards

 

Computer-Assisted 

Motivational 

Intervention (CAMI)

n.a. n.a. n.a. n.a. n.a. Low No

Middle School 

Horizon Program n.a. n.a. n.a. n.a. n.a. Low Yes

Web-based HIV, 

hepatitis, and STI 

prevention program

n.a. n.a. n.a. n.a. n.a. Low No

Note: 0, +, and – indicate evidence of null, favorable, and adverse effects, respectively. Empty cells indicate that the study did not 

examine program impacts on measures within that outcome domain. DNMS indicates that the findings for measures within the 

outcome domain did not meet the review standards (DNMS). 

a The review team established ratings for the supporting impact studies following pre-specified criteria to assess study design, attrition, 

baseline equivalence, reassignment of sample members, and confounding factors. See Table 2 below for a more detailed 

description.

b This column indicates which programs received funding from FYSB or from the OAH TPP program.

n.a. = not applicable; following the procedures specified in the review protocol, the review team did not assess evidence of 

effectiveness for programs with a supporting impact study that received a Low quality rating.

A description of the three new programs meeting the review criteria for evidence of 

effectiveness, as well as a summary of the research evidence around these programs, are as 

follows: 

Crossroads

Program summary: Crossroads is an intensive three-day intervention for older adolescents at risk 

of dropping out of school. Program participants meet for 6 to 7 hours per day over three 

consecutive days outside the regular school day (in the afternoon and during the weekend) in a 

community-based setting. Trained program facilitators lead a diverse range of activities, 

including small-group skill-building activities and discussion of how the program activities relate 

to the adolescents’ sexual health, outdoor experiential activities, a tour of a community college 

campus, and a community service project. During the second and third days of the program, 

facilitators deliver a version of the Be Proud! Be Responsible! curriculum that was adapted so 

that its entire content could be covered in two days. The focus of the Be Proud! Be Responsible! 

curriculum is building knowledge about STIs, and HIV/AIDS, and teen pregnancy prevention.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 4

Research summary: Crossroads was evaluated in a randomized controlled trial involving 1,174 

adolescents ages 17 to 19 classified as high risk for dropping out of high school (Slater and 

Mitschke 2015). The study recruited participants from one alternative and six traditional high 

schools in the Arlington Independent School District in Texas. The study found that six months 

after the end of the program, adolescents participating in the intervention were significantly less 

likely than those in the study control group to report having had vaginal intercourse without 

using a condom in the past three months (odds ratio = 0.38, confidence interval = 0.59 to 0.79). 

For the follow-up conducted 3 months after the program ended, the study found no evidence of 

statistically significant program impacts on having vaginal, anal, and oral sexual intercourse 

without a condom. The study also examined program impacts on having sexual activity without 

using a condom and on ever being pregnant 12 months after the program ended. However, the 

findings for the 12-month follow-up did not meet the review standards because they were based 

on a sample with high attrition. The study met the review criteria for a High quality rating.

POWER Through Choices

Program summary: POWER Through Choices is a comprehensive sexual health education 

program focused on teen pregnancy, HIV, and STI prevention, and is designed to serve

adolescents living in foster care, juvenile justice facilities, and other out-of-home care settings.

The program includes 10 sessions, each lasting 90-minutes, delivered once or twice a week for 5 

to 10 weeks. The sessions include interactive skill-building activities that emphasize selfempowerment and the impact of choices. 

Research summary: POWER Through Choices was evaluated in a cluster randomized controlled 

trial involving 885 adolescents ages 13 to 18 recruited from 44 residential group homes in 

California, Maryland, and Oklahoma (Covington et al. 2016). The evaluation found that 12 

months after the program ended, adolescents in the treatment group were significantly less likely to 

report ever been pregnant or gotten someone pregnant than the adolescents in the control group 

(odds ratio = 0.81, confidence interval = 0.75 to 0.88). In addition, 12 months after the end of the 

program the study found that the subgroup of female adolescents in the treatment group were 

significantly less likely to report ever been pregnant than their counterparts in the control group

(odds ratio = 0.66, confidence interval = 0.56 to 0.77). The study showed no statistically 

significant impacts on any other sexual behavior outcomes assessed. The study met the review 

criteria for a High quality rating.

Possessing Your Power 

Program summary: Possessing Your Power is a classroom-based youth character development 

curriculum designed for youth ages 10 to 18 (or in 6th to 12th grades). The curriculum includes 9 

lessons covering topics such as self-esteem, conflict resolution, bullying, decision making, and 

leadership skills. The program also addresses adolescent risk behaviors, such as sexual activity, 

drug and alcohol use, tobacco use, violence and crime, pornography, and gambling. The 

curriculum materials on sexual activity take an abstinence-centered approach. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 5

Research summary: Possessing Your Power was evaluated in a cluster randomized controlled 

trial involving 1,989 adolescents ages 13 to 17 (average age was 15 years) recruited from 48 

community-based organization summer programs in Hampton Roads, Virginia (JK Tanner Inc.). 

In the study, researchers examined program impacts on a measure the study authors describe as 

“risky sexual behavior”, which is a binary variable that takes the value of 1 if the adolescent had 

unprotected sex, meaning the adolescent did not use a condom or birth control every time for 

recent sexual activity (in the past three months), and takes the value of 0 if the adolescent used a 

condom or birth control every time for recent sexual activity or if the adolescent avoided sexual 

activity in the previous three months. Six months after the program ended, researchers found that 

adolescents participating in the intervention who were sexually inexperienced at baseline were less 

likely to report that they did not use a condom or birth control every time for sexual activity in the past 

three months than their counterparts in the control group (odds ratio = 0.31, confidence interval = 0.19 to 

0.52). The study found no evidence of statistically significant program impacts on that outcome 

for either the full sample or the subgroup of adolescents who reported being sexually 

experienced at baseline. The study met the review criteria for a Moderate quality rating.

New evidence for previously reviewed programs

The recent update to the review findings also sought to identify and assess any new evidence for 

programs examined in previous rounds of the review, several of which have been evaluated only 

once. However, a growing number of studies have sought to test how these programs perform 

when implemented on a broader scale, in different settings, or with different populations 

(Goesling 2015). As part of the update to the review findings, the review team identified and 

assessed 41 new publications on 25 programs examined in previous rounds of the review. Of 

those 41 new publications, 36 are reports or journal articles presenting the findings from studies 

funded through grants for 21 programs provided by FYSB or the OAH TPP program. Of the 

remaining 5 publications (on 4 programs), 1 received funding from the Centers for Disease 

Control and Prevention, 1 from the Health Resources and Service Administration, 2 from the 

National Institutes of Health, and 1 did not report its funding source. 

For three of the 25 programs examined in previous rounds of the review, the newly assessed 

publications provided new evidence of positive impacts. One of these three programs met the 

review criteria for evidence of effectiveness for the first time. For the other two programs—which 

had met the review criteria for evidence of effectiveness in previous rounds of the review—the 3 

newly assessed publications provide evidence of longer-term program impacts and/or youth 

outcomes not examined in previously assessed publications. 

For the other 22 of the 25 programs examined in previous rounds of the review, the new 

publications did not materially affect the review team’s previous assessment of these programs or 

their impacts on other youth outcomes. That is, the program was previously determined to have 

evidence of effectiveness (15 programs), and the newly assessed publications did not change that 

assessment (29 publications), or the program was previously determined to not have evidence of 

effectiveness (7 programs), and the new research maintained that assessment (8 publications). 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 6

Nonetheless, the 37 new publications present new results, adding to the evidence-base on those 22 

programs. 

Below we provide a more detailed description of the new evidence of positive impacts available 

for three programs, which are the following: (1) Familias Unidas (program meeting the review 

criteria for evidence of effectiveness for the first time), (2) Positive Potential, and (3) Teen 

Options to Prevent Pregnancy (T.O.P.P.). We also provide a more detailed description of the 

findings from the 8 studies of the Teen Outreach Program (TOP), including six new studies of the 

program reviewed for the first time. One of the six studies provided new evidence of statistically 

significant program effects and 5 studies found no significant effects. 

Familias Unidas

Program summary: Familias Unidas is a family-based, HIV prevention intervention specifically 

designed to serve Hispanic youth and their families. The Familias Unidas program includes eight 

2-hour group sessions with the parents (alone), and four 1-hour family visits with parents and 

adolescents that are focused on increasing parent-adolescent communication about sex and HIV 

risks. An adapted version of the intervention, Brief Familias Unidas, reduces the duration of the 

program, including just five parent sessions and one family visit. 

Research summary: The full version of the program has been evaluated in two studies, which 

were reviewed in previous rounds of the HHS TPP evidence review. The first, published in 2007, 

involved 266 Hispanic 7th grade students in Miami-Dade County middle schools and their 

families (Prado et al. 2007). The second, a 2012 study, involved 242 juvenile justice youth and 

their families in Miami-Dade County (Prado et al. 2012). Both studies met the review criteria for 

a High rating. However, neither of those two studies found evidence of statistically significant 

program impacts on student sexual risk behaviors. In the current round of the review, we 

assessed a more recently published study that involved 160 students in Miami-Dade County high 

schools and their families (Estrada et al. 2015). In this study, researchers examined the 

effectiveness of Brief Familias Unidas using a randomized controlled trial. The study reports that 

24 months after the baseline survey, the adolescents in the group that received Brief Familias 

Unidas and who were not sexually experienced at baseline were less likely to have had initiated 

sex than their counterparts in the group that did not receive the program (odds ratio = 0.42, 

confidence interval = 0.27 to 0.65). The study met the review criteria for a High rating.

Positive Potential 

Program summary: Positive Potential is a school-based, positive youth development program 

developed primarily for adolescents attending middle school in rural communities. The program 

is provided as a supplemental program to the health and physical education curricula adolescents 

receive as part of their middle school education. The Positive Potential program is a three-year 

program, offered in three middle school grades, where each grade level has a separate

curriculum. At each grade level, the program offers five 45- to 50-minute classroom sessions on 

consecutive days and one class assembly. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 7

Research summary: The program was evaluated in a randomized controlled trial involving 1,374 

students in 14 public middle and elementary schools in northwestern Indiana (Piotrowski and 

Hedeker 2015, 2016a, 2016b). Students were in 6th grade at the time of study recruitment, and 

were offered the full three-year curriculum, until the end of 8th grade. The study conducted two 

follow-ups: one at the beginning of the 7th grade (about 12 months after the baseline), and the 

second 12 months after the end of the three-year program, in spring of 9th grade. We reviewed 

the findings for the 7th grade follow-up in a previous round of the review (Piotrowski and 

Hedeker 2015, 2016a), and in the current round of the review we assessed the findings for the 

9th grade follow-up (Piotrowski and Hedeker 2016b). The study found that at the beginning of 

the 7th grade, adolescents in schools that delivered the program were significantly less likely to 

have had sexual intercourse, ever (odds ratio = 0.28, confidence interval = 0.15 to 0.55) and in 

the last three months (odds ratio = 0.21, confidence interval = 0.09 to 0.46). At the beginning of 

the 7th grade, the subgroup of males in the treatment group were also significantly less likely

than their counterparts in the control group to have had sexual intercourse, ever (odds ratio = 

0.08, confidence interval = 0.04 to 0.16) and in the last three months (odds ratio = 0.08 

confidence interval = 0.03 to 0.21). Those program effects persisted 12 months after the program 

ended, in spring of 9th grade: in schools that implemented Positive Potential, both the full 

sample of students and the subgroup of males were significantly less likely to ever have had 

sexual intercourse (odds ratio is 0.78 for the full sample and 0.67 for the subgroup of males) and 

to have had sexual intercourse in the last 12 months (odds ratio is 0.76 for the full sample and 

0.65 for the subgroup of males). In addition, the subgroup of males in the schools implementing 

Positive Potential were significantly less likely to have had sexual intercourse in the last three 

months than their counterparts in the schools not implementing the program (odds ratio = 0.62). 

The study met the review criteria for a Moderate quality rating. 

Teen Options to Prevent Pregnancy (T.O.P.P.)

Program summary: T.O.P.P. is a clinic-based intervention for pregnant and parenting adolescent 

females. The T.O.P.P. program is delivered individually to program participants over 18 months 

through telephone calls from trained nurse educators provided about once a month, with greater 

call frequency during the initial months of the program and whenever participants are actively 

seeking and adopting new forms of birth control. The program promotes healthy birth spacing 

and use of effective contraception, and it provides direct access to contraceptive services through 

a program clinic and access to a program social worker. 

Research summary: The program was evaluated in a randomized controlled trial involving 493 

low-income expectant or adolescent new mothers in the Columbus, Ohio area (Kim et al. 2015; 

Rotz et al. 2016a). The study collected outcome data six and 18 months after study enrollment. 

We reviewed the findings for the six-month follow-up in a previous round of the review (Kim et 

al. 2015) and the findings for the 18-month follow-up in the current round of the review (Rotz et 

al. 2016a). The study found that six months after study enrollment, adolescents participating in the 

intervention were significantly less likely to report having had sex without using birth control in 

the past three months (odds ratio = 0.51, confidence interval = 0.39 to 0.66). Those program 

effects on sexual activity without use of birth control in the past three months persisted 18 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 8

months after study enrollment (odds ratio = 0.56). The study also found that 18 months after 

study enrollment, adolescents participating in T.O.P.P. were significantly less likely to report 

having had sex without using a condom in the past three months (odds ratio = 0.65, confidence 

interval = 0.52 to 0.81), having had a repeated pregnancy (odds ratio = 0.56), an unwanted 

pregnancy (odds ratio = 0.38), a pregnancy ending in a live birth in the past 18 months (odds 

ratio = 0.46), and fewer lifetime pregnancies (difference in regression-adjusted means = -0.30). 

The study met the review criteria for a High quality rating. 

Teen Outreach Program (TOP). 

Program summary: TOP is a positive youth development program that incorporates weekly 

curriculum-guided lessons, community service learning, and positive adult guidance and support. 

TOP is focused on key topics related to adolescent health and development, including healthy 

relationships, communication, influence, goal-setting, decision making, values clarification, 

community service learning, and adolescent development and sexuality. The program is 

facilitated using the experiential learning cycle, multiple intelligences, service learning, and the 

"Quad-A" approach where facilitators are adaptable, age-appropriate, affective, and active. TOP 

is provided in school, after-school, and community settings to adolescents 12 to 17 years old. 

Research summary: The efficacy of this program was first established in a 1997 study that we 

assessed in a previous round of the review (Allen et al. 1997). The study involved 695 high 

school students attending grades 9 to 12 in high schools in 25 cities across the United States. The 

study found that female adolescents participating in the program were significantly less likely to 

report a pregnancy during the academic year of the program (odds ratio = 0.41, confidence 

interval = 0.26 to 0.64). A more recent study, published in 2015 and assessed in a previous round 

of the review, evaluated the program among a sample of 2,058 adolescents in 26 Florida high 

schools (Daley et al. 2015; Walsh-Buhi et al. 2016). The study successfully replicated the 

favorable program impact on pregnancy reported in the earlier 1997 study. In particular, for the 

follow-up conducted at the end of the program, the study found that students in the treatment 

group were less likely to report ever having been pregnant or gotten someone pregnant, and the 

reported effect size (odds ratio = 0.58) falls within the confidence interval for the effect size 

reported in the prior 1997 study. The study found a similar pattern of results for the longer-term 

follow-up survey conducted 10 months after the program ended (odds ratio = 0.51). The study 

also found a statistically significant program impact on sexual activity rates. In particular, for the 

follow-up conducted at the end of the program, the study found that students in the treatment 

group were statistically significantly less likely to report ever having had sex (odds ratio = 0.84, 

confidence interval = 0.78 to 0.90). 

Six additional more recent studies (described in Appendix A), which we assessed for the current 

update of the review findings, examined the effectiveness of TOP when implemented with 

different populations and in different settings. The studies were conducted in Denver (Bull et al. 

2015, 2016), Rochester (Crean et al. 2015; Robinson et al. 2016b), Kansas City (Philliber and 

Philliber 2016), Louisiana (Robinson et al. 2016a; Robinson et al. 2016b), Chicago (Seshadri et 

al. 2015), and across five northwestern states: Alaska, Idaho, Montana, Oregon, and Washington 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 9

(Philliber et al. 2015). All six studies used random assignment evaluation designs and met the 

review criteria for either a Moderate (five studies) or High (the study conducted across five 

northwestern states) quality rating. Four of the six studies failed to replicate the favorable 

impacts on pregnancy and sexual activity rates found in the earlier 1997 and 2015 studies, 

respectively. Specifically, the Kansas City and the Rochester study found that adolescents in the 

group receiving TOP were no less likely than adolescents in the control group to report ever 

having sex (odds ratio = 0.96 in the Kansas City study, odds ratio = 0.68 in the Rochester study). 

The Louisiana study found that immediately after the end of the program, the subgroup of 

adolescents in the group receiving TOP who were sexually inexperienced at baseline were no 

less likely than their counterparts in the control group to report they had become sexually active 

(odds ratio = 1.25). The study also found that 12 months after the program, adolescents in the 

group receiving TOP were no less likely than adolescents in the control group to report ever 

being pregnant (odds ratio = 0.96). The Chicago study found no evidence of program impacts on 

recent sexual activity or on having sex without using a condom in the last three months. The 

study conducted across five northwestern states found mixed effects (Philliber et al. 2015). The 

study found that immediately after the program ended, the subgroup of male adolescents in the 

schools that implemented TOP were less likely to report ever causing a pregnancy than their 

counterparts in schools that did not implement the program (odds ratio = 0.71). For the same 

time period, the study found that the subgroup of female adolescents in the treatment schools 

were more likely than those in the control schools to report having ever been pregnant (odds ratio 

= 1.27). The study found no significant impacts on pregnancy for the full sample immediately 

and 12 months after the program ended (odds ratio = 1.13 immediately after the program, odds 

ratio = 1.15 12 months after the program). Finally, the Denver study examined the effectiveness 

of supplementing TOP with a text message program. The study found no evidence of favorable 

impacts of the supplemental texting program on ever being pregnant or causing a pregnancy 

(odds ratio = 1.57) or on sexual activity with use of condoms or contraceptives Bull et al. 2015, 

2016). 

Summary

We identified 15 new programs in the current round of the review (Table 1). Three of these

newly identified programs—Crossroads, POWER Through Choices, and Possessing Your 

Power—met the review criteria for evidence of effectiveness, having a study with a High or 

Moderate quality rating that showed evidence of a favorable, statistically significant program 

impact on at least one sexual risk behavior or reproductive health outcome of interest. For nine 

other newly identified programs, the supporting impact studies met the review criteria for a study 

quality rating of High or Moderate, but the study findings did not show favorable, statistically 

significant program impacts. For the remaining three programs, the supporting impact studies did 

not meet the review criteria for a High or Moderate study quality rating. 

In this brief (Tables 2 and 3 below) and on the review website 

(https://tppevidencereview.aspe.hhs.gov/FindAProgram.aspx), we summarize the available 

evidence (positive, negative, and null) across all studies on a particular program model in each of 

five outcome domains: (1) sexual activity; (2) number of sexual partners; (3) contraceptive use; 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 10

(4) STIs or HIV; and (5) pregnancies. The strength of the body of evidence behind a given 

program model is represented by both the color and size of the icon (see Key below). Visitors to 

the review website may use this information to compare program models that match their 

outcome domain of interest, as well as the desired program type, population, age range, or 

implementation setting.

The HHS Teen Pregnancy Prevention Evidence Review has focused on the quality and rigor of 

the evaluations and the effect of the program models on a small set of outcomes since 2009. With 

the evidence that has been gathered, we have been able to identify many programs that met those 

requirements. Going forward, we will be strengthening the review criteria to identify robust 

programs with the goal of ultimately improving outcomes.

Key

Color indication Description

Uniformly positive impacts on outcomes in domain

A mix of positive, null, and/or adverse impacts on outcomes in domain

Uniformly null impacts on outcomes in domain

Uniformly adverse impacts on outcomes in domain

Size indication Description

Five or more studies

Two to four studies

One study

Note: A blank cell indicates the studies associated with the program did not examine any outcome measures within that particular 

outcome domain, or the studies examined outcome measures within that domain but the findings did not meet the review 

evidence standards.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 11

Table 2. Evidence by outcome domain and study rating for new publications on 

programs assessed in previous rounds of the review

 Outcome domains 

Program/publication

Sexual 

activity

Number 

of 

sexual 

partners

Contraceptive 

use STIs Pregnancy

Quality 

ratinga

FYSB or 

OAH 

fundedb

Program with new 

evidence, meeting the 

criteria for evidence of 

effectiveness for the 

first time

 

Familias Unidas 

Estrada et al. (2015) + 0 High No

Programs with evidence 

of longer-term impacts 

and/or youth outcomes 

not examined in 

previously assessed 

publications

 

Positive Potential 

Piotrowski and Hedeker 

(2016a); Piotrowski and 

Hedeker (2016b)

+ 0 0 Moderate Yes

Teen Options to Prevent 

Pregnancy (T.O.P.P.) 

Rotz el al. (2016a) 0 0 + + High Yes

Programs with evidence 

of effectiveness, new 

publications did not 

change that assessment

 

AIM 4 Teen Moms 

Covington et al. (2017) 0 0 0 0 Moderate Yes

Becoming a 

Responsible Teen 

The Policy & Research Group 

(2015a) 0 0 High Yes

Ruwe et al. (2016) 0 0 High Yes

Children’s Aid Society 

(CAS)–Carrera 

Herrling (2015) 0 0 High Yes

¡Cuídate! 

Kelsey et al. (2016a); Kelsey 

et al. (2016b) 0 0 0 0 High Yes

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 12

 Outcome domains 

Program/publication

Sexual 

activity

Number 

of 

sexual 

partners

Contraceptive 

use STIs Pregnancy

Quality 

ratinga

FYSB or 

OAH 

fundedb

Generations 

Lewin et al. (2016) + Moderate No

Healthy Futures 

Calise et al. (2016) + 0 Moderate Yes

It’s Your Game: Keep it 

Real 

Potter et al. (2016) - 0 Moderate Yes

Love Notes 

Barbee et al. (2016) + 0 + + Moderate Yes

Positive Prevention 

PLUS 

LaChausse (2016) + + 0 Moderate Yes

Promoting Health 

Among Teens! 

Abstinence-Only 

Intervention (PHAT-AO)

 

Walker et al. (2016) 0 High Yes

Promoting Health 

Among Teens! 

Comprehensive 

Abstinence and Safer 

Sex Intervention (PHATComprehensive)

 

Martin et al. (2015) 0 0 Moderate Yes

Reducing the Risk 

Barbee et al. (2016) + 0 + + Moderate Yes

Kelsey et al. (2016c); Kelsey 

et al. (2016d) 0 0 0 0 High Yes

Teen Outreach Program 

(TOP) 

Bull et al. (2015); Bull et al. 

(2016) 0 Moderate Yes

Crean et al. (2015); Robinson 

et al. (2016b) 0 0 Moderate Yes

Philliber and Philliber (2016) 0 0 Moderate Yes

Philliber et al. (2016) 0 0 + and - High Yes

Robinson et al. (2016a); 

Robinson et al. (2016b) 0 0 0 Moderate Yes

Seshadri et al. (2015) 0 0 Moderate Yes

Walsh-Buhi et al. (2016) DNMS DNMS DNMS Moderate Yes

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 13

 Outcome domains 

Program/publication

Sexual 

activity

Number 

of 

sexual 

partners

Contraceptive 

use STIs Pregnancy

Quality 

ratinga

FYSB or 

OAH 

fundedb

Safer Sex 

Kelsey et al. (2016e); Kelsey 

et al. (2016f) 0 0 0 0 0 High Yes

The Policy & Research Group 

(2015b) 0 0 High Yes

Seventeen Days

(formerly What Could 

You Do?)

 

Eichner et al. (2015); Downs 

et al. (2016) 0 0 High Yes

Programs with no 

evidence of 

effectiveness, new 

publications did not 

change that assessment

 

Choosing The Best 

JK Tanner Inc. (2016) 0 High Yes

Gender Matters (GEN.M) 

Smith et al. (2016) DNMS DNMS Moderate Yes

HealthTeacher 

Goesling et al. (2016) 0 Moderate Yes

READY-Girls 

Charron-Prochownik et al. 

(2013) 0 0 0 High No

Thurheimer (2016) n.a. n.a. n.a. n.a. n.a. Low No

Teen Prevention 

Education Program 

(Teen PEP)

 

Rotz el al. (2016b) 0 0 0 Moderate Yes

Will Power/Won’t Power 

Advanced Empirical Solutions 

(2015) n.a. n.a. n.a. n.a. n.a. Low Yes

Wise Guys 

Herrman et al. (2016) n.a. n.a. n.a. n.a. n.a. Low No

Note: 0, +, and – indicate evidence of null, favorable, and adverse effects, respectively. Empty cells indicate that the study did not 

examine program impacts on measures within that outcome domain. DNMS indicates that the findings for measures within the 

outcome domain did not meet the review standards (DNMS) because the quality rating for the findings is not consistent with the 

overall quality rating of the study. a The review team established quality ratings for the publications listed in this table following pre-specified criteria to assess study 

design, attrition, baseline equivalence, reassignment of sample members, and confounding factors. See Table 4 below for a more 

detailed description. b This column indicates which publications are reports or journal articles presenting the findings from studies funded through grants 

provided by FYSB or the OAH TPP program.

n.a. = not applicable; following the procedures specified in the review protocol, the review team did not assess evidence of 

effectiveness for programs with a supporting impact study that received a Low quality rating.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 14

Table 3. Evidence by outcome domain for programs assessed in previous rounds of 

the review

 Outcome domains

Program

Sexual

activity

Number of 

sexual 

partners

Contraceptive 

use STIs Pregnancy

Program with new evidence, 

meeting the criteria for evidence 

of effectiveness for the first time

 

Familias Unidas 

Programs with evidence of 

longer-term impacts and/or 

youth outcomes not examined in 

previously assessed 

publications

 

Positive Potential 

Teen Options to Prevent 

Pregnancy (T.O.P.P.) 

Programs with evidence of 

effectiveness, new publications 

did not change that assessment

 

AIM 4 Teen Moms 

Becoming a Responsible Teen 

Children’s Aid Society (CAS)–

Carrera 

¡Cuídate!

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 15

 Outcome domains

Program

Sexual

activity

Number of 

sexual 

partners

Contraceptive 

use STIs Pregnancy

Generations 

Healthy Futures 

It’s Your Game: Keep it Real 

Love Notes 

Positive Prevention PLUS 

Promoting Health Among Teens! 

Abstinence-Only Intervention 

(PHAT-AO)

 

Promoting Health Among Teens! 

Comprehensive Abstinence and 

Safer Sex Intervention (PHATComprehensive)

 

Reducing the Risk

Teen Outreach Program (TOP) 

Safer Sex

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 16

 Outcome domains

Program

Sexual

activity

Number of 

sexual 

partners

Contraceptive 

use STIs Pregnancy

Seventeen Days (formerly What 

Could You Do?) 

Programs with no evidence of 

effectiveness, new publications 

did not change that assessment

 

Choosing The Best 

Gender Matters (GEN.M) 

HealthTeacher 

READY-Girls 

Teen Prevention Education 

Program (Teen PEP) 

Will Power/Won’t Power 

Wise Guys n.a. n.a. n.a. n.a. n.a.

Note: n.a. indicates not applicable. Following the procedures specified in the review protocol, the review team did not assess 

evidence of effectiveness for programs with supporting impact studies that received a Low quality rating.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 17

Review procedures

This update to the review findings followed procedures similar to those used for prior rounds of 

the review. In October 2016, the review team released a public call for studies requesting new 

research for consideration. The team also identified studies through a comprehensive literature 

search that entailed keyword searches of electronic databases and manual searches of relevant 

academic journals. The identified studies were then screened against pre-specified eligibility 

criteria. 

For studies that met the eligibility criteria, the Mathematica review team first assessed the quality 

and execution of each study’s research design (Table 4). The reviewers assigned each study a 

quality rating of High, Moderate, or Low according to the risk of bias in the study's impact 

findings. A more detailed description of these ratings is provided in the review protocol available 

online at the review website (available at 

https://tppevidencereview.aspe.hhs.gov/ReviewProtocol.aspx). 

For the subset of studies that achieved a Moderate or High quality rating, the review team 

extracted information on the features of the program tested, evaluation setting, study sample, and 

research design, as well as detailed information on the program impact estimates. The review 

team then identified programs with evidence of effectiveness, defined as having a statistically 

significant favorable impact (and no adverse effects) on at least one priority outcome measured 

for either the full analytic sample or a subgroup defined by (1) gender or (2) sexual experience at 

baseline. The priority outcomes include sexual activity, number of sexual partners, contraceptive 

use, STIs, and pregnancy. 

This update to the review findings did not introduce any changes to the review criteria. More 

detailed information on the review process and criteria is available on the review website.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 18

Table 4. Summary of study quality ratings

Criteria category High study rating Moderate study

rating

Low study rating

Study design Random or functionally 

random assignment

Quasi-experimental design 

with a comparison group; 

random assignment design 

with high attrition or 

reassignment

Does not meet criteria for 

High or Moderate quality 

rating

Attrition What Works Clearinghouse 

conservative standards for 

overall and differential 

attrition

No requirement Does not meet criteria for 

High or Moderate quality 

rating

Baseline

equivalence

Must control for statistically 

significant baseline 

differences in age, gender, 

race/ethnicity, and outcome 

measures

Must establish baseline 

equivalence of research 

groups and control for 

baseline outcome measures

Does not meet criteria for 

High or Moderate quality 

rating

Reassignment Analysis must be based on 

original assignment to 

research groups

No requirement Does not meet criteria for 

High or Moderate quality 

rating

Confounding

factors

Must have at least two 

subjects or groups in each 

research group and no 

systematic differences in 

data collection methods

Must have at least two 

subjects or groups in each 

research group and no 

systematic differences in 

data collection methods

Does not meet criteria for 

High or Moderate quality 

rating

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 19

References

Abt Associates. “Reducing the Risk: Interim Impact Report, Teen Pregnancy Prevention 

Replication Study.” Report prepared for the Office of Adolescent Health and the Office of 

the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human 

Services, September 2015a. 

Abt Associates. “Safer Sex Intervention: Interim Impact Report, Teen Pregnancy Prevention 

Replication Study.” Report prepared for the Office of Adolescent Health and the Office of 

the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human 

Services, September 2015b. 

Abt Associates. “¡Cuídate!: Interim Impact Report”. Teen Pregnancy Prevention Replication 

Study, Report prepared for the Office of Adolescent Health and the Office of the Assistant 

Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, 

September 2015c. 

Advanced Empirical Solutions. Impact of a Teen Pregnancy Prevention Program in Los Angeles 

County School Districts, Evaluation of Will Power/Won’t Power in Los Angeles County, 

2015. 

Allen, J.P., S. Philliber, S. Herrling, and G.P. Kuperminc. “Preventing Teen Pregnancy and 

Academic Failure: Experimental Evaluation of a Developmentally Based Approach.” Child 

Development, vol. 68, no. 4, 1997, pp. 729–742. 

Barbee, A. P., Cunningham, M. R., van Zyl, M. A., Antle, B. F., and Langley, C. N. “Impact of 

two adolescent pregnancy prevention interventions on risky sexual behavior: A three-arm 

cluster randomized control trial.” American Journal of Public Health, 106, 2016, pp. S85-

S90. 

Bull, S., Schmiege, S., Devine, S. “Evaluation of Youth all Engaged (YAE) in Denver, CO.” 

Final Impact Report for the Denver Public Health Department. Written Report, 2015. 

Bull, S., Devine, S., Schmiege, S. J., Pickard, L., Campbell, J., and Shlay, J. C. “Text messaging, 

teen outreach program, and sexual health behavior: A cluster randomized trial.” American 

Journal of Public Health, 106, 2016, pp. S117-S124. 

Calise T.V., Chow, W., and Doré, K.F. JSI Research & Training Institute, Inc. “Evaluation of 

Healthy Futures in Three Northeastern Massachusetts Cities: Findings from an Innovative 

Teen Pregnancy Prevention Program. Final Impact Report for The Black Ministerial 

Alliance of Greater Boston, Inc.” Prepared for the Office of Adolescent Health, U.S. 

Department of Health and Human Services, 2015. 

Calise, T. V., Chow, W., Dore, K. F., O'Brien, M. J., Heitz, E. R., and Millock, R. R. “Healthy 

futures program and adolescent sexual behaviors in 3 Massachusetts cities: A randomized 

controlled trial.” American Journal of Public Health, 106, 2016, pp. S103-S109. 

Charron-Prochownik, D., Sereika, S.M., Becker, D., White, N.H., Schmitt, P., Powell, A.P., 

Diaz, A.M., Jones, J., Herman, W.H., Rodgers Fischl, A.F., McEwen, L., DiNardo, M., 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 20

Guo, F., and Downs, J. “Long-term Effects of the Booster-Enhanced READY-Girls 

Preconception Counseling Program on Intentions and Behaviors for Family Planning in 

Teens with Diabetes.” Diabetes Care, vol. 36, 2013, pp. 3870- 3874. 

Covington, R. et al. “Interim Impacts of the AIM 4 Teen Moms Program.” Princeton, NJ: 

Mathematica Policy Research, 2015.

Covington, Reginald D., Brian Goesling, Christina Clark Tuttle, Molly Crofton, Jennifer 

Manlove, Roy F. Oman, and Sara Vesely. “Final Impacts of the POWER Through Choices 

Program.” Washington, DC: Mathematica Policy Research, September 2016.

Covington, Reginald D., Dara Lee Luca, Jennifer Manlove, and Kate Welti. “Final Impacts of 

AIM 4 Teen Moms.” Washington, DC: U.S. Department of Health and Human Services, 

Office of Adolescent Health, February 2017. 

Coyle, Karin, K., Susan C. Potter, Jill R. Glassman, and Liz McDade-Montez. “Process and 

Impact Evaluation of Keep It Real South Carolina: Reducing Teen Pregnancy Among 

Middle School Youth.” ETR, Report prepared for the Office of Adolescent Health, U.S. 

Department of Health & Human Services, August 2015.

Crean, H. F., Seibold-Simpson, S. M., Jambon, M., and Kreipe, R. E. “Evaluation of the Teen 

Outreach Program® in Rochester, New York: Finding from the replication of an evidencebased teen pregnancy prevention program.” Rochester, New York: University of Rochester 

School of Nursing, 2015. 

Cunningham, M. R., van Zyl, M. A., and Borders, K. W. “Evaluation of Love Notes and 

Reducing the Risk in Louisville, Kentucky.” Final Evaluation Report to the University of 

Louisville Research Foundation, Louisville, KY, 2016. 

Daley, E.M., Buhi, E.R., Wang, W., Singleton, A., Debate, R., Marhefka, S., et al. “Evaluation of 

Wyman’s Teen Outreach Program® in Florida: Final Impact Report for Florida Department 

of Health.” Findings from the Replication of an Evidence-Based Teen Pregnancy Prevention 

Program, 2015. 

Downs, J. S., Ashcraft, A. M., and Murray, P. J. “Video for adolescent pregnancy prevention: 

Promises, challenges, and future directions.” American Journal of Public Health, 106, 2016, 

pp. S29-S31. 

Eichner, J., Salaway, J., Smith-Jones, J., and McCall, R. “Evaluation of Seventeen Days in Ohio, 

Pennsylvania, and West Virginia: Findings from the Replication of an Evidence-Based Teen 

Pregnancy Prevention Program.” Pittsburgh, PA: University of Pittsburgh, Office of Child 

Development, 2015. 

Estrada, Y., Rosen, A., Huang, S., Tapia, M., Sutton, M., Willis, L., Quevedo, A., Condo, C., 

Vidot, D. C., Pantin, H., and Prado, G. “Efficacy of a brief intervention to reduce substance 

use and human immunodeficiency virus infection risk among Latino youth.” Journal of 

Adolescent Health, 57(6), 2015, pp. 651-657. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 21

Goesling, Brian, Silvie Collman, Mindy Scott, and Elizabeth Cook. “Impacts of an Enhanced 

Family Health and Sexuality Module of the HealthTeacher Middle School Curriculum.” 

Final Report. Mathematica Policy Research, Princeton, NJ: May 2014. 

Goesling, Brian. “Making Sense of Replication Studies: Guidance for the Field of Teen 

Pregnancy Prevention Research.” Washington, DC: U.S. Department of Health and Human 

Services, Office of the Assistant Secretary for Planning and Evaluation, 2015. 

Goesling, B., Scott, M. E., and Cook, E. “Impacts of an enhanced family health and sexuality 

module of the HealthTeacher middle school curriculum: A cluster randomized trial.” 

American Journal of Public Health, 106, 2016, pp. S125-S131. 

Herrling, S. “Evaluation of the Children’s Aid Society (CAS)-Carrera Adolescent Pregnancy 

Prevention Program in Chicago, IL: Findings from the Replication of an Evidence-Based 

Teen Pregnancy Prevention Program.” Accord, NY: Philliber Research & Evaluation, 2015. 

Herrman, J. W., Moore, C., and Rahmer, B. “Focus on teen men: Evaluating the effectiveness of 

the Wise Guys program.” Journal of Child & Adolescent Psychiatric Nursing, 29(1), 2016, 

pp. 37-43. 

JK Tanner Inc. “Evaluation of Choosing the Best and Possessing Your Power: Comparison of an 

Evidence-Based Teen Pregnancy prevention Program and to a New Program.” Lighthouse 

Outreach, Inc. Yes! Dare2Dream Final Report, 2016. 

Kelsey, M., Layzer, C., Layzer, J., Price, C., Juras, R., Blocklin, M., and Mendez, J. “Replicating 

¡Cuídate!: 6-month impact findings of a randomized controlled trial.” American Journal of 

Public Health, 106, 2016a, pp. S70-S77. 

Kelsey, M., Layzer, J., Price, C., and Juras, R. “¡Cuídate!: Final Impact Report." Draft Teen 

Pregnancy Prevention Replication Study, Abt Associates: Cambridge, MA, November 

2016b. 

Kelsey, M., Blocklin, M., Layzer, J., Price, C., Juras, R., and Freiman, L. “Replicating Reducing 

the Risk: 12-month impacts of a cluster randomized controlled trial.” American Journal of 

Public Health, 106, 2016c, pp. S45-S52. 

Kelsey, M., Layzer, J., Blocklin, M., Price, C., and Juras, R. "Reducing the Risk: Longer-Term 

Impact Report." Draft Teen Pregnancy Prevention Replication Study, Abt Associates: 

Cambridge, MA, November 2016d. 

Kelsey, M., Walker, J. T., Layzer, J., Price, C., and Juras, R. “Replicating the Safer Sex 

intervention: 9-month impact findings of a randomized controlled trial.” American Journal 

of Public Health, 106, 2016e, pp. S53-S59. 

Kelsey, M., Layzer, J., Price, C., Juras, R., and Walker, J. "Safer Sex Intervention: Short-Term 

Impact Report." Draft Teen Pregnancy Prevention Replication Study, Abt Associates: 

Cambridge, MA, November 2016f. 

LaChausse, R. “Evaluation of the Positive Prevention PLUS Teen Pregnancy Prevention 

Program.” USDHHS, Office of Adolescent Health, 2015, TP2AH000007. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 22

LaChausse, R. G. “A clustered randomized controlled trial of the positive prevention PLUS 

adolescent pregnancy prevention program.” American Journal of Public Health, 106, 2016, 

pp. S91-S96. 

Lewin, Amy, Stephanie Mitchell, and Michel Boudreaux. “Improved Contraceptive Use Among 

Teen Mothers in a Family-Centered Medical Home.” Unpublished manuscript, University of 

Maryland, 2015. 

Lewin, A., Mitchell, S., Beers, L., Schmitz, K., and Boudreaux, M. “Improved contraceptive use 

among teen mothers in a patient-centered medical home.” Journal of Adolescent Health,

59(2), 2016, pp. 171-176. 

Martin, S., Hill, A., Nye M., and Hollman-Billmeier, K. “Evaluation of Alaska Promoting Health 

Among Teens, Comprehensive Abstinence and Safer Sex (AKPHAT) in Alaska.” Institute 

of Social and Economic Research, University of Alaska Anchorage, 2015. 

Philliber, A.E., Philliber, S., and Brown, S. “Evaluation of the Teen Outreach Program® in The 

Pacific Northwest.” Accord, NY: Philliber Research & Evaluation, 2015. 

Philliber, A.E. and Philliber, S. “Evaluation of the Teen Outreach Program® in Kansas City, 

Missouri.” Accord, NY: Philliber Research & Evaluation, 2016. 

Piotrowski, Z. Harry and Don Hedeker. “Evaluation of The Positive Potential Be The Exception 

Grade 6 Program in Predominantly Rural Communities: Findings from an Innovative Teen 

Pregnancy Prevention Program.” Report to the Office of Adolescent Health, U.S. 

Department of Health & Human Services, August 2015.

Piotrowski, Z. H., and Hedeker, D. “Evaluation of the Be the Exception sixth-grade program in 

rural communities to delay the onset of sexual behavior.” American Journal of Public 

Health, 106, 2016a, pp. S132-S139. 

Piotrowski, Z. H., and Hedeker, D. “Evaluation of Positive Potential Middle School Longitudinal 

Program in Rural Northwest Indiana Communities: Findings From an Innovative Teen 

Pregnancy Prevention Program.” Report Prepared for Office of Adolescent Health, U.S. 

Department of Health and Human Services (HHS), Washington DC, 2016b. 

Potter, S. C., Coyle, K. K., Glassman, J. R., Kershner, S., and Prince, M. S. “It's Your Game. 

Keep it Real in South Carolina: A group randomized trial evaluating the replication of an 

evidence-based adolescent pregnancy and sexually transmitted infection prevention 

program.” American Journal of Public Health, 106, 2016, pp. S60-S69. 

Prado, G., Pantin, H., Briones, E., Schwartz, S., Feaster, D., Huang, S., et al. “A randomized 

controlled trial of a family-centered intervention in preventing substance use and HIV risk 

behaviors in Hispanic adolescents.” Journal of Consulting and Clinical Psychology, 7(6), 

2007, pp. 914-926. 

Prado, G., Pantin, H., Huang, S., Cordova, D., Tapia, M. I., Velazquez, M. R., Calfee, M., 

Malcolm, S., Arzon, M., Villamar, J., Jimenez, G. L., Cano, N., Brown, C. H., and Estrada, 

Y. “Effects of a family intervention in reducing HIV risk behaviors among high-risk 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 23

Hispanic adolescents: A randomized controlled trial.” Archives of Pediatrics and Adolescent 

Medicine, 166(2), 2012, pp. 127-133. 

Robinson, W.T., Kaufman, R. and Cahill, L. “Evaluation of the Teen Outreach Program in 

Louisiana. New Orleans, LA.” Louisiana State University Health Sciences Center at New 

Orleans, School of Public Health, Initiative for Evaluation and Capacity Building, 2016a. 

Robinson, W. T., Seibold-Simpson, S., Crean, H. F., and Spruille-White, B. “Randomized trials 

of the Teen Outreach Program in Louisiana and Rochester, New York.” American Journal 

of Public Health, 106, 2016b, pp. S39-S44. 

Rotz, Dana et al. “Interim Impacts of the Teen PEP Program.” Mathematica Policy Research, 

Cambridge, MA: 2015. 

Rotz, Dana, Dara Lee Luca, Brian Goesling, Elizabeth Cook, Kelly Murphy, and Jack Stevens. 

“Final Impacts of the Teen Options to Prevent Pregnancy Program.” Cambridge, MA: 

Mathematica Policy Research, July 2016a.

Rotz, Dana, Brian Goesling, Molly Crofton, Jennifer Manlove, and Kate Welti. “Final Impacts of 

Teen PEP in New Jersey and North Carolina High Schools.” Cambridge, MA: Mathematica 

Policy Research, September 2016b. 

Ruwe, M.B., McCloskey, L., Meyers, A., Prudent, N., and Foureau-Dorsinville, M. “Evaluation 

of Haitian American Responsible Teen.” Findings from the Replication of an Evidencebased Teen Pregnancy Prevention Program in Eastern Massachusetts, August 2016. 

Seshadri, R., Smithgall, C., Goerge, R., Ippolito, J., Dasgupta, D., Wiegand, E., Guiltinan, S., 

and Wood, M. “Evaluation of Teen Outreach Program in Chicago: Final Impact Report for 

Chicago Public Schools”. Chicago: Chapin Hall at the University of Chicago, 2015. 

Slater, H.M., and Mitschke, D.B. “Evaluation of the Crossroads Program in Arlington, TX: 

Findings from an Innovative Teen Pregnancy Prevention Program” Arlington, TX: 

University of Texas at Arlington, 2015. 

Smith, Kim, et al. “Interim Impacts of the Teen Options to Prevent Pregnancy Program.” 

Mathematica Policy Research, Princeton, NJ: 2015a. 

Smith, Kim, et al. “Interim Impacts of the Gender Matters Program.” Mathematica Policy 

Research, Princeton, NJ: 2015b. 

Smith, Kimberly V., Claire Dye, Dana Rotz, Elizabeth Cook, Kristina Rosinsky, and Mindy 

Scott. “Final Impacts of the Gender Matters Program.” Princeton, NJ: Mathematica Policy 

Research, September 2016.

The Policy & Research Group. “Evaluation of Becoming a Responsible Teen: Findings from the 

Replication of an Evidence-based Teen Pregnancy Prevention Program. New Orleans, LA: 

Jenner, E., Walsh, S., Jenner, L. W., Demby, H., Gregory, A., and Davis, E., 2015a. 

The Policy & Research Group. “Evaluation of Safer Sex Intervention in New Orleans, LA: 

Findings from the Replication of an Evidence-Based Teen Pregnancy Prevention Program.” 

New Orleans, LA: Jenner, E., Walsh, S., Jenner, L. W., Demby, H., and Gregory, A., 2015b. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 24

Thurheimer, J. L. “Examining short- and long-term effects of preconception counseling delivered 

during adolescence on risk-taking behaviors, condom use, and sexually transmitted 

infections among females with type 1 diabetes.” Dissertation Abstracts International: 

Section B: The Sciences and Engineering, 77(1-B(E), Sefe), 2016. 

Walker, Elaine M., Inoa, R., and Coppola, N. “Evaluation of Promoting Health Among Teens 

Abstinence-Only Intervention in Yonkers, NY.” Sametric Research. Princeton, N.J. 08540, 

2016. 

Walsh-Buhi, E. R., Marhefka, S. L., Wang, W., Debate, R., Perrin, K., Singleton, A., Noble, C. 

A., Rahman, S., Maness, S. B., Mahony, H., Ziemba, R., Malmi, M., Marwah, E., Hall, K., 

Turner, D., Blunt-Vinti, H., Noble, S. M., and Daley, E. M. “The impact of the teen 

outreach program on sexual intentions and behaviors.” The Journal of Adolescent Health: 

Official Publication of the Society for Adolescent Medicine, 59(3), 2016, pp. 283-290. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 25

Appendix A. New publications identified and assessed for programs previously 

reviewed

Program/publication

Quality

ratinga Description

Program with new evidence, 

meeting the criteria for 

evidence of effectiveness for 

the first time 

 

 

Familias Unidas 

Estrada et al. (2015) High

This publication reports the findings from a randomized controlled trial of a 

brief version of the Familias Unidas program involving 160 students in 

Miami-Dade County high schools and their families. The study found a 

statistically significant program impact on sexual initiation for adolescents 

who had not had sex at baseline.

Programs with evidence of 

longer-term impacts and/or 

youth outcomes not examined 

in previously assessed 

publications 

 

 

Positive Potential 

Piotrowski and Hedeker (2016a); Piotrowski 

and Hedeker (2016b)

Moderate

These publications include a government report that presents findings from

a previously reviewed randomized controlled trial of Positive Potential, as 

well as an academic journal article that highlights findings from the study. 

The study involved 1,374 adolescents attending grades 6 to 8 in elementary

and middle schools in five northwestern Indiana counties (Piotrowski and 

Hedeker 2015). The study measured longer-term impact findings about 3 

and 12 months after the end of the program. For the follow-up occurring 3 

months after the end of the program, the study found no statistically 

significant findings. However, the study found evidence of favorable 

program impacts for the 12 months after end of program follow-up. 

Teen Options to Prevent 

Pregnancy (T.O.P.P.) 

 

Rotz el al. (2016a) High

This publication is a government report that presents the longer-term impact 

findings of a randomized controlled trial of T.O.P.P. (Smith et al. 2015). The 

study involved 493 low-income expectant or adolescent new mothers in the 

Columbus, Ohio area. The longer-term impact findings, measured 18 

months after program enrollment, show evidence of favorable, statistically 

significant program effects on measures of contraceptive use and 

pregnancy.

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 26

Program/publication

Quality

ratinga Description

Programs with evidence of 

effectiveness, new 

publications did not change 

that assessment 

 

 

AIM 4 Teen Moms 

Covington et al. (2017) Moderate

This publication is a government report that presents the longer-term 

impacts findings of a previously reviewed randomized controlled trial of AIM 

4 Teen Moms (Covington et al. 2015). The study involved 800 low-income, 

adolescent new mothers in Los Angeles County. The longer-term impact 

findings, measured about 21 months after the program ended, show no 

evidence of statistically significant, favorable program effects on sexual 

behavior outcomes.

Becoming a Responsible Teen 

The Policy & Research Group (2015a) High

This publication is a government report that presents the findings of a 

randomized controlled trial of Becoming a Responsible Teen (BART), a 

sexual education curriculum designed for African-American teens. The study 

involved 850 adolescents recruited from youth summer employment 

programs in New Orleans. The study examined the effects of the program 

on inconsistency of condom use and frequency of sexual activity. The study 

found no evidence of favorable program effects on those outcome measures 

six months after the end of the program.

Ruwe et al. (2016) High

This publication is a government report that presents the findings of a 

randomized controlled trial of Haitian-American Responsible Teen (HART), 

a cultural adaptation of BART designed for Haitian teens living in the U.S. 

The study involved 552 youth of Haitian descent attending 9th through 11th 

grades in schools located in the greater Boston area. The study examined 

program effects on ever having sex, having sexual intercourse in the past 

three months, and having sex without using a condom or using any birth 

control method in the past three months. Six months after the end of the 

intervention, the study found no evidence of favorable program effects on 

any of the sexual behavior outcome measures that were examined.

Children’s Aid Society (CAS)–

Carrera 

Herrling (2015) High

This publication is a government report that presents the findings of a 

randomized controlled trial of the CAS–Carrera program involving 338 

students in elementary and middle schools in Chicago. The study examined 

the effects of the program on measures of sexual initiation and sexual 

activity without the use of contraception. The study did not find any 

statistically significant program impacts on those measures at the follow-up 

conducted four years after the baseline. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 27

Program/publication

Quality

ratinga Description

¡Cuídate! 

Kelsey et al. (2016a); Kelsey et al. (2016b) High

These publications include a government report that presents findings from 

a previously reviewed randomized controlled trial of the ¡Cuídate!

curriculum, as well as an academic journal article that highlights findings 

from the study. The study involved 2,022 adolescents attending grades 8 to 

12 in middle and high schools in Arizona, California, and Massachusetts

(Abt Associates 2015c). The longer-term impact findings (16 months after 

the end of the program) show no statistically significant, favorable programs 

impacts on sexual behavior outcomes.

Generations 

Lewin et al. (2016) Moderate

This publication highlights key findings, in an academic journal article, from 

a previously reviewed quasi-experimental design study of the Generations 

program. The study involved 124 teen mothers and their children in 

Washington, DC, and it found statistically significant, favorable program 

effects on measures of condom and effective contraception use (Lewin et al. 

2015). 

Healthy Futures 

Calise et al. (2016) Moderate

This publication is an academic journal article that highlights key findings 

from a previously reviewed government report of a cluster randomized 

controlled trial of the Healthy Futures program. The study involved 2,346 

students from 15 middle schools in three cities in northeastern 

Massachusetts (Calise et al. 2015). At the end of 8th grade, the study found 

that female adolescents in the schools that delivered the intervention were 

significantly less likely to report ever having vaginal sex. 

It’s Your Game: Keep it Real 

Potter et al. (2016) Moderate

This publication is an academic journal article that highlights key findings 

from a previously reviewed government report on a cluster randomized 

controlled trial of the It’s Your Game: Keep it Real program (Coyle et al. 

2015). The study involved 2,487 students from 24 rural middle schools in 

South Carolina, and found no evidence of favorable program effects for the 

8th grade follow-up. For the 9th grade follow-up, the study reports no 

statistically significant impacts on measures of recent sexual activity, and 

having sex without using effective birth control or a condom in the last three 

months. It also found evidence of an adverse effect: students in the 

treatment schools were more likely than those in the control schools to 

report having initiated sexual activity. 

Love Notes 

Barbee et al. (2016) Moderate

This publication is an academic journal article that highlights key findings 

from a previously reviewed government report on a cluster randomized trial 

of the Love Notes program (Cunningham et al. 2016). The study involved

933 adolescents recruited from community-based organizations serving 

youth in low-income areas of Louisville, Kentucky. Six months after the 

program ended, the study found that adolescent participants were 

significantly less likely than youth in the study control group to report ever 

having had sex, ever having been pregnant, having had sex in the last three 

months, and having had sex without a condom or without birth control in the 

last three months. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 28

Program/publication

Quality

ratinga Description

Positive Prevention PLUS 

LaChausse (2016) Moderate

This publication is an academic journal article that highlights key findings 

from a previously reviewed government report on a randomized controlled 

trial of Positive Prevention Plus (LaChausse 2015). The study involved 

3,490 9th grade students in 21 public high schools in southern California. 

Six months after the end of the program, the study found that students in the 

schools that offered the program were significantly less likely to initiate 

sexual activity and to have had sex in the last three months without using 

birth control. 

Promoting Health Among 

Teens! Abstinence-Only 

Intervention (PHAT-AO)

 

 

Walker et al. (2016) High

This publication is a government report that presents the findings of a 

randomized controlled trial of PHAT-AO program involving 1,319 students in 

6th and 7th grades in public schools in Yonkers, New York. This study 

examined the program’s effectiveness on ever having sex at 3, 6, and 12 

months after the end of the program. The study found no evidence of 

favorable program effects on ever having sex at any of the follow-ups. 

Promoting Health Among 

Teens! Comprehensive 

Abstinence and Safer Sex 

Intervention (PHATComprehensive) 

 

Martin et al. (2015) Moderate

This publication is a government report that presents the findings of a 

randomized controlled trial of Alaska Promoting Health Among Teens, 

Comprehensive Abstinence and Safer Sex (AKPHAT), an adaptation of 

PHAT-Comprehensive. The study adapted the PHAT-Comprehensive 

program to: (1) use peer educators to deliver the program instead of adult

facilitators, (2) serve a different target population of older youth in rural 

areas, (3) use talking circles and talking sticks, and (4) use fingers rather 

than a penis model in the condom demonstrations module. The study 

involved 302 Alaskan Native youth recruited from four non-profit 

organizations serving youth in Alaska, and examined program effects on 

recent sexual activity and use of condoms during recent sexual intercourse. 

Six months after the end of the intervention, the study found no evidence of 

favorable program effects on any of those sexual behavior outcome 

measures.

Reducing the Risk 

Barbee et al. (2016) Moderate

This publication is an academic journal article that highlights key findings 

from a previously reviewed government report on a cluster randomized trial 

of Reducing the Risk (Cunningham et al. 2016). The study involved 939 

adolescents recruited from community-based organizations serving youth in 

low-income areas of Louisville, Kentucky. The study found that three months 

after the program ended, adolescents in the treatment group were 

significantly less likely to report ever having had sex or having been 

pregnant in the last three months. In addition, six months after the program 

ended, adolescents in the treatment group were significantly less likely to 

report having had sex without birth control in the last three months. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 29

Program/publication

Quality

ratinga Description

Kelsey et al. (2016c); Kelsey et al. (2016d) High

These publications include a government report that presents findings from

a previously reviewed cluster randomized controlled trial of Reducing the 

Risk, as well as an academic journal article that highlights findings from the 

study. The study involved 2,689 students in 150 classrooms in public 

middle, junior high, and high school in three study sites in Missouri, Texas, 

and California, respectively (Abt Associates 2015a). The longer-term impact 

findings of the study, measured about 24 months after program enrollment, 

show no evidence of favorable program impacts on sexual behavior 

outcomes.

Teen Outreach Program (TOP) 

Bull et al. (2015); Bull et al. (2016) Moderate

These publications include a government report that presents findings from

a cluster randomized controlled trial examining the effectiveness of 

supplementing TOP with a text message program called Youth All Engaged 

(YAE), as well as an academic journal article that highlights findings from 

the study. YAE consists of five to seven text messages a week while TOP is 

being delivered, and about three messages per week after TOP ends for 12 

weeks. All text messages reinforced the topics covered by TOP and 

delivered additional teen pregnancy prevention content. The study involved 

854 adolescents recruited from Boys & Girls Clubs in Denver, CO, and 

examined program effects on measures of condom and contraceptive use. 

Immediately after the program ended, the study found no evidence of 

favorable program impacts. 

Crean et al. (2015); Robinson et al. (2016b) Moderate

These publications include a government report that presents findings from

a cluster randomized controlled trial of TOP, as well as an academic journal 

article that highlights findings from the study. The study involved 1,188

adolescents recruited from recreation centers in Rochester, NY. The study 

examined the effects of TOP on measures of sexual initiation and 

contraceptive use, and found no evidence of favorable program impacts on 

those measures immediately after the program ended (about nine months 

after the baseline). 

Philliber and Philliber (2016) Moderate

This publication is a government report that presents the findings of a 

cluster randomized controlled trial of TOP involving 934 students in middle 

and high schools in Kansas City, Missouri. The study examined the effects 

of the program on measures of sexual initiation and contraceptive use. 

Twelve months after the end of the program, the study did not find any 

favorable, statistically significant program impacts on those measures.

Philliber et al. (2016) High

This publication is a government report that presents the findings of a

cluster randomized controlled trial of TOP involving 8,662 adolescents 

attending schools in five northwestern states. The study found evidence of 

mixed program effects. Immediately after the program ended, the study 

found a positive program impact on pregnancy rates for male adolescents in 

the sample. For the same time period, the study also found an adverse 

effect on pregnancy rates for female adolescents in the sample.

Robinson et al. (2016a); Robinson et al. 

(2016b) Moderate

These publications include a government report that presents findings from

a randomized controlled trial of TOP, as well as an academic journal article 

that highlights findings from the study. The study involved 4,769 adolescents 

recruited from Community Based Organizations in Louisiana. The study 

examined the effects of the program on measures of contraceptive use and 

pregnancy immediately and 12 months after the program ended. The study 

found no evidence of favorable program effects on those measures in any of 

the follow-ups. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 30

Program/publication

Quality

ratinga Description

Seshadri et al. (2015) Moderate

This publication is a government report that presents the findings of a 

cluster randomized controlled trial of TOP involving 5,633 9th grade 

students in Chicago Public Schools. The study examined the effects of the 

program on measures of recent sexual activity and contraceptive use. 

Immediately after the end of the program (about 10 months after the 

baseline), the study did not find evidence of any favorable, statistically 

significant program impacts.

Walsh-Buhi et al. (2016) Moderate

This publication is an academic journal article that highlights findings from a 

previously reviewed government report on a cluster randomized controlled 

trial of TOP (Daley et al. 2015). The study involved 2,058 students attending

9th through 12th grades in high schools in Florida, and examined program 

effects on measures of sexual initiation and pregnancy immediately and 10 

months after the end of the program. The study findings presented in the 

government report met the review criteria for a High rating. The findings 

presented in Walsh-Buhi et al. (2016) are based on sample sizes that 

indicate there was high attrition for that sample. Because of the high attrition 

these findings received a rating of Moderate, which is not consistent with the 

overall rating of the study (High rating, based on the findings presented in 

the government report). For that reason, these findings did not meet the 

review criteria for evidence of effectiveness. 

Safer Sex 

Kelsey et al. (2016e); Kelsey et al. (2016f) High

These publications include a government report that presents findings from

a previously reviewed randomized controlled trial of the Safer Sex 

intervention, as well as an academic journal article that highlights findings 

from the study. The study involved 1,809 young women who sought 

treatment in one of three study sites (health clinics) in Florida, Minnesota, 

and Tennessee (Abt Associates 2015b). The longer-term impact findings of 

the study, measured 18 months after baseline, show no statistically 

significant, favorable program impacts on measures of sexual initiation, 

recent sexual activity, number of partners, contraceptive use, STIs 

incidence, and pregnancy.

The Policy & Research Group (2015b) High

This publication is a government report that presents the findings of a 

randomized controlled trial of Safer Sex involving 319 young women who 

sought treatment in five health clinics in New Orleans, LA. The study 

examined the effects of the program on measures of inconsistency of 

condom use, inconsistency of use of any contraceptive, and frequency of 

sexual activity. Six months after the end of the program, the study found no 

evidence of favorable, statistically significant program effects. 

Seventeen Days (formerly 

What Could You Do?) 

 

Eichner et al. (2015); Downs et al. (2016) High

These publications include a government report that presents findings from

a randomized controlled trial of Seventeen Days, as well as an academic 

journal article that highlights findings from the study. The study involved

1,317 young women who sought treatment at health clinics in Ohio, 

Pennsylvania, and West Virginia. The study examined the effects of the 

program on measures of abstinence and contraceptive use three and six 

months after program enrollment. The study found no evidence of favorable, 

statistically significant program effects on those measures at any of the 

follow-ups. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 31

Program/publication

Quality

ratinga Description

Programs with no evidence of 

effectiveness, new 

publications did not change 

that assessment 

 

 

Choosing The Best 

JK Tanner Inc. (2016) High

This publication is a government report that presents the findings of a 

cluster randomized controlled trial of Choosing the Best. The study involved 

1,535 adolescents participating in the summer programs offered by 105 

community-based organizations in Hampton Roads, Virginia. The study 

found no evidence of favorable program effects on a measure of consistent 

use of a condom or birth control in the last three months, in follow-ups 

conducted six and 12 months after the program ended.

Gender Matters (GEN.M) 

Smith et al. (2016) Moderate

This publication is a government report that presents the longer-term 

impacts findings of a previously reviewed randomized controlled trial of 

GEN.M. (Smith et al. 2015). The study involved 754 adolescents in Travis 

County, Texas. Similarly to the shorter-term impact findings (measured 6 

months after study enrollment), the longer-term impact findings (measured 

18 months after program enrollment) show no evidence of favorable 

programs effects on sexual behavior outcomes. However, the longer-term 

impact findings did not meet the review criteria for evidence of effectiveness 

because they had high attrition and received a rating of Moderate, which is 

not consistent with the overall rating of the study (High rating, based on the 

shorter-term impact findings55).

HealthTeacher 

Goesling et al. (2016) Moderate

This publication is an academic journal article that highlights key findings 

from a previously reviewed government report on a cluster randomized 

controlled trial of the HealthTeacher program (Goesling et al. 2014). The 

study involved 1,189 students in 14 middle schools in Chicago and found no 

evidence of favorable programs impacts on sexual behavior outcomes. 

READY-Girls 

Charron-Prochownik et al. (2013) High

This publication presents the findings of a randomized controlled trial of the 

READY Girls program involving 113 female adolescents with type 1 or type 

2 diabetes and who were recruited from health clinics in Pittsburg, PA. The 

study examined the effects of the program on measures of sexual initiation, 

abstinence, number of partners, unprotected sexual intercourse, and 

condom use 3, 6, and 12 months after the end of the program. The study did 

not find any statistically significant program impacts on those measures at

any of the follow-ups. 

Thurheimer (2016) Low

This publication presents the findings of a quasi-experimental study of the 

READY-Girls program. The study received a low rating because the timing 

of data collection differed for the intervention and comparison groups. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov 
Page 32

Program/publication

Quality

ratinga Description

Teen Prevention Education 

Program (Teen PEP) 

Rotz el al. (2016b) Moderate

This publication is a government report that presents the longer-term impact 

findings of a previously reviewed cluster randomized controlled trial of Teen 

PEP, measured 18 months after the program ended (Rotz et al. 2015). The 

study involved 942 adolescents recruited from high schools in New Jersey 

and North Carolina. The longer-term impact findings show no evidence of 

favorable, statistically significant program impacts on sexual behavior 

outcomes. 

Will Power/Won’t Power 

Advanced Empirical Solutions (2015) Low

This publication presents the findings of a randomized controlled trial of the 

Will Power/Won’t Power program that involved 561 female adolescents 

attending middle schools in Los Angeles. The study received a low rating 

because it had high attrition and did not establish baseline equivalence for 

the final analysis sample. 

Wise Guys 

Herrman et al. (2016) Low

This publication presents the findings from a study of Wise Guys, a 

pregnancy prevention program aimed at male adolescents. The study 

received a low rating because it did not have an external comparison group. 

a The review team established quality ratings for the publications listed in this table following pre-specified criteria to assess study 

design, attrition, baseline equivalence, reassignment of sample members, and confounding factors. See Table 4 for a more 

detailed description of the study quality ratings. 

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.aspe.hhs.gov