U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

https://tppevidencereview.youth.gov/

Page 1 

Updated Findings from the HHS Teen Pregnancy Prevention Evidence 

Review: October 2016-May 2022

April 2023 

By Emily Forrester, Jamie Manzer, Kelsey Chesnut, Jean Knab, Russell Cole, Julieta Lugo-Gil,

Lexi Ouellette, Anna Gu, Astrid Harnack-Eber, Juliette Henke, Brandon Hollie, 

Lara Hulsey, and Christina Kent 

Mathematica Policy Research

Since 2009, the U.S. Department of Health and Human Services (HHS) has sponsored a 

systematic review of research on teen pregnancy prevention to identify programs with evidence 

of effectiveness in favorably impacting (1) teen pregnancy and sexually transmitted infections 

(STIs) and (2) sexual behaviors. The HHS Teen Pregnancy Prevention (TPP) Evidence Review 

was created in response to the 2010 Consolidated Appropriations Act, which states that teen 

pregnancy prevention programs must be “proven effective through rigorous evaluation to reduce 

teenage pregnancy, behavioral risk factors underlying teenage pregnancy, or other associated risk 

factors.” Mathematica conducts the TPP Evidence Review (TPPER), which is sponsored by the 

HHS Office of the Assistant Secretary for Planning and Evaluation, the Office of Population 

Affairs (OPA) within the HHS Office of the Assistant Secretary for Health, and the Family and 

Youth Services Bureau (FYSB) within HHS’s Administration for Children and Families. 

As of April 2018, when the last findings were released, the HHS Teen Pregnancy Prevention 

Evidence Review had identified 48 programs meeting the review criteria for evidence of 

effectiveness based on a review of studies released from August 2015 to October 2016. These 

criteria require programs to show evidence of at least one favorable, statistically significant 

impact on at least one outcome of interest reflecting sexual behavior (for example, whether 

teens have ever had sex) or reproductive health (for example, teens’ sexual activity, number of 

sexual partners, contraceptive use, STIs or HIV, or pregnancy). In addition, the supporting 

research studies must meet established criteria for the quality and execution of their 

research designs. The review team follows pre-specified criteria to assess study design, sample 

attrition, baseline equivalence, reassignment of sample members, and confounding factors. We 

detail the review procedures later in this brief.

Mathematica has recently updated the review findings to include research released from October 

2016 to May 2022. Due to resource constraints, we identified more studies than we reviewed. 

We used a prioritization process (described in the review protocol) to determine which studies to 

include in this round of the review. As part of this update, the review team identified and 

assessed studies of 18 programs that were not reviewed before and assessed studies for 7 

programs that were previously reviewed but also had more recent studies identified in this round. 

Of these 25 programs, 9 were new programs with evidence of effectiveness, 4 were previously 

reviewed programs with past evidence of effectiveness that have maintained their standing, and 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 2 

12 were programs (both new and previously reviewed) without evidence of effectiveness (Table 

1). One additional program was reviewed during this round and found to have evidence of 

effectiveness. However, the developer indicated that this program is not being disseminated or 

available to the public. It is available on the TPPER website list of inactive programs. Two other 

programs reviewed in previous rounds have also been moved to the list of inactive programs 

(Table 6).

The total number of programs meeting the review criteria for evidence of effectiveness is 

now 52: 43 programs identified in earlier rounds of the review (that have current evidence of 

effectiveness and are still currently available) plus the 9 programs identified in the most recent 

update to the review findings. 

Table 1. Overview of the 25 programs reviewed during this round of the TPP Evidence Review

New programs with evidence of effectiveness

Girl2Girl

High-school FLASH, version 3.0

IN·clued

LeadHer

Linking Families and Teens (LiFT)

Peer Group Connection-High School (PGC-HS)

Plan A

Pulse (two studies reviewed)

Vision of You

Previously reviewed programs with past and new evidence of effectiveness

Making Proud Choices! (MPC)

Power Through Choices (PTC)

Families Talking Together (FTT)

Previously reviewed programs with past evidence of effectiveness and no new evidence of effectiveness

Generations

New and previously reviewed programs without evidence of effectiveness

Achieving Condom Empowerment (ACE)-Plus

Ateyapi Identity Mentoring Program

Play Forward

Steps to Success

TEMPO

Decisions, Responsibility, Empowerment, Accountability, Motivation and Success (DREAMS)

Non-faith–based, abstinence-only intervention

Faith-based, abstinence-only intervention

In the Know 

Becoming a Responsible Teen1

Choosing the Best

Respecting the Circle of Life

1 Becoming a Responsible Teen did have evidence of effectiveness; however, the study that presents this evidence 

has a publication date that is now outside the 20-year eligibility window.
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 3 

New programs with evidence of effectiveness

We reviewed studies for 18 newly identified 

programs (Table 2) that varied in terms of their 

focus, target population, setting, and content. 

Studies of 9 of these 18 programs met the review 

criteria for evidence of effectiveness, having a 

rating of high or moderate and showing evidence 

of a favorable, statistically significant impact on 

at least one outcome of interest regarding sexual 

behavior or reproductive health (as designated by 

 in Table 2). A high or moderate study rating is 

necessary for the findings to be deemed as having 

sufficiently credible evidence of program 

effectiveness. 

For five other programs, the supporting impact 

studies met the review criteria for a study quality rating of high or moderate, but the study 

findings did not show favorable, statistically significant program impacts. For the remaining four 

programs, the supporting impact studies did not meet the review criteria for a high or moderate 

quality rating. 

Fifteen of the 18 newly identified programs were developed and tested with federal grant funding 

from FYSB or OPA. Federally funded studies were prioritized for review. 

Program-level findings 

In this brief (Table 2) and on the review website 

(https://tppevidencereview.youth.gov/EvidencePrograms.aspx) we summarize the findings 

(favorable, potentially favorable, indeterminate, conflicting, potentially unfavorable, and 

unfavorable) across identified studies of a particular newly-identified program model assessed in 

each of five outcome domains: (1) sexual activity; (2) number of sexual partners; (3) 

contraceptive use; (4) STIs or HIV; and (5) pregnancies. The strength of the body of evidence for 

a program model across individual studies is represented by the color, quantity, and size of the 

icons (Key 1). The symbols and ratings descriptions for the program models represent categories 

of evidence defined by TPPER and some categories do not apply to the newly identified 

programs in this round of review (e.g., none of the program models in Table 2 have “favorable 

evidence” or “conflicting evidence”). The size of the icon reflects the number of studies that 

have high or moderate quality study ratings, not necessarily stronger or weaker evidence of 

effectiveness. 

Categorization of Findings

This round of the TPP Evidence Review 

introduces a new categorization scheme for 

findings, which includes three key changes:

1. The addition of “potentially favorable” and 

“potentially unfavorable” findings categories

2. The replacement of the label “mixed 

impacts” with “inconsistent findings”

3. The use of two types of domain ratings 

labels, one for individual studies (Key 2) 

and one for programs (Key 1), which 

reflects results pooled across studies of 

each program
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 4 

Key 1. Strength of the body of evidence behind a given program model as illustrated by color, 

quantity, and size of icons

Symbol Description Criteria

Favorable evidence: Strong evidence 

of favorable findings with no 

overriding contrary evidence

1. Two or more studies show favorable findings

and

2. No studies have inconsistent findings, potentially 

unfavorable findings, or unfavorable findings

Potentially favorable evidence: 

Evidence of a favorable finding with 

no evidence of adverse findings

1. At least one study shows favorable findings or potentially 

favorable findings

and

2. No studies have inconsistent findings, potentially 

unfavorable findings, or unfavorable findings

Indeterminate evidence: No 

affirmative evidence of findings; 

uniformly null findings

1. All of the studies show indeterminate findings

Conflicting evidence: Evidence of 

both favorable and unfavorable 

findings 

1. At least one study shows inconsistent findings, 

or

2. At least one study shows favorable findings or potentially 

favorable findings, and at least one study shows 

unfavorable findings or potentially unfavorable findings

Potentially unfavorable evidence: 

Evidence of unfavorable findings with 

no overriding contrary evidence 

1. At least one study shows unfavorable findings or 

potentially unfavorable findings 

and

2. No studies have inconsistent findings, potentially 

favorable findings, or favorable findings

Unfavorable evidence: Strong 

evidence of unfavorable findings with 

no overriding contrary evidence 

1. Two or more studies show unfavorable findings 

and

2. No studies have inconsistent findings, potentially 

favorable findings, or favorable findings

Symbol Size Indication

Applies to all 

program ratings

1 study with a moderate or high study rating that examines 

outcomes in the domain

Applies to all 

program ratings

2–4 studies with moderate or high study ratings that 

examine outcomes in the domain

Applies to all 

program ratings

5 or more studies with moderate or high study ratings that 

examine outcomes in the domain
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 5 

Table 2. Newly identified programs: evidence by outcome domain and study rating 

Outcome domains

Program

Sexual 

activity

Number 

of sexual 

partners

Contraceptive 

use

STIs 

or HIV Pregnancy

Study 

ratinga

FYSB or 

OPA 

fundedb

Programs with evidence of effectiveness

Girl2Girl . . High Yes 

High-school FLASH, version 

3.0 . . . High Yes 

IN·clued . High Yes 

LeadHer . . High Yes 

Linking Families and Teens 

(LiFT) . . High Yes 

Peer Group Connection-High 

School (PGC-HS) . . . High Yes 

Plan A . . . High Yes 

Pulse (two studies reviewed) . . . High Yes 

Vision of You . . High Yes 

Programs for which all studies reviewed meet standards, but have no positive effects

Achieving Condom 

Empowerment (ACE)-Plus . . . . High Yes 

Ateyapi Identity Mentoring 

Program . . . . Moderate Yes 

Play Forward . . . . High No

Steps to Success . . . High Yes 

TEMPO . . . . High Yes 

Programs for which all studies reviewed do not meet evidence standards

Decisions, Responsibility, 

Empowerment, 

Accountability, Motivation and 

Success (DREAMS)

n.a. n.a. n.a. n.a. n.a. Low Yes 

Non-faith–based, abstinenceonly interventionn.a. n.a. n.a. n.a. n.a. Low No

Faith-based, abstinence-only 

intervention

n.a. n.a. n.a. n.a. n.a. Low No

In the Know n.a. n.a. n.a. n.a. n.a. Low Yes

Note: Refer to Key 1 for information about symbols in this table. Empty cells indicate that the study or studies did 

not examine program impacts on measures within that outcome domain. Programs are listed alphabetically 

within each evidence category.

a The review team established ratings for the supporting impact studies by following pre-specified criteria to assess 

study design, attrition, baseline equivalence, reassignment of sample members, and confounding factors. Table 2 has 

a more detailed description.

b This column indicates which programs were developed and tested with federal grant funding from FYSB or OPA.

n.a. = not applicable; following the procedures specified in the review protocol, the review team did not assess 

evidence of effectiveness for programs with a supporting impact study that received a quality rating of low
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 6 

A description of the 9 new programs meeting the review criteria for evidence of effectiveness, 

along with a summary of the research evidence around these programs, follows: 

Girl2Girl

Program summary: Girl2Girl is a 20-week, text-messaging–based program to prevent teen 

pregnancy. The program is designed for cisgender2 female youth ages 14–18 identifying as not 

exclusively heterosexual and provides information about unintended pregnancy prevention, 

different types of birth control, how to use contraception, and STI prevention. Some program 

content is tailored to participants’ self-reported sexual experiences and sexual orientation. Most 

texts simply provide information, whereas some are interactive, including links to videos. The 

program also offers an on-demand advice text line called G2Genie that provides information 

about sex, relationships, and the LGBTQ community. Finally, Girl2Girl participants are paired 

with another program participant, a “text buddy,” to give each other mutual support and help 

them practice skills they learn in the program.

Research summary: The program was evaluated using a randomized controlled trial involving 

948 cisgender lesbian, bisexual, or queer women ages 14–18 recruited across the United States 

through Facebook and Instagram advertisements (Ybarra et al. 2021). The study found that at 

program end (20 weeks), teen girls participating in the program reported having used condoms in 

the past three months while having sex significantly more often than the control group in the past 

three months (effect size = 0.03) and reported significantly fewer sex acts without a condom in 

the past three months (effect size = -0.05). Those program effects were larger in magnitude for 

the subgroup of girls who reported being sexually active at baseline (effect sizes = 0.05 and 

-0.29, respectively). The study showed no statistically significant impacts on any other sexual 

behavior outcomes assessed. The study met the review criteria for a high quality rating. 

High School FLASH, version 3.0 

Program summary: High School FLASH (FLASH) is a school-based comprehensive sexual 

health education program for youth in high school. The 15-session curriculum covers pregnancy, 

the reproductive system, healthy relationships, sexual orientation and gender identity, coercion 

and consent, abstinence, birth control, online safety, condoms, preventing HIV and STIs, STD 

testing, decision making, and communication. FLASH is delivered in schools by sexuality 

educators trained by FLASH curriculum developers on program implementation, curriculum 

components, and how to answer student questions. 

Research summary: FLASH was evaluated using a randomized controlled trial involving 1,734 

9th and 10th grade youth recruited from seven school districts in the midwestern and southern 

United States (Coyle et al. 2021). The study found that among the subgroup of youth who were 

sexually inexperienced at baseline, youth participating in FLASH were significantly less likely to 

report having had vaginal sex without condoms or other birth control in the prior three months 

than youth in the control group were (effect size = -0.67). This impact was measured three 

2 Cisgender is a term that describes individuals whose current gender identity is the same as the sex they were 

assigned at birth (CDC, n.d.).
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 7 

months after the program ended. The study showed no statistically significant impacts on any of 

the other sexual behavior outcomes that were assessed. The study met the review criteria for a 

high rating. 

IN·clued 

Program summary: The IN·clued: Inclusive Healthcare – Youth and Providers Empowered 

Program is a program focused on reducing STIs and unintended pregnancies among LGBTQ 

youth ages 14–19. IN·clued provides a one-time, in-person, three-hour workshop to LGBTQ 

youth and two 90-minute youth-led workshops for health care providers and clinic staff. Program 

content can be delivered in various community settings including schools and community-based 

organizations, and at conferences. It is delivered by trained youth peer educators and adult 

sexuality educators. The content emphasizes health disparities, safer sex practices, health care 

self-efficacy, exam room interactions, and patient rights. Youth can also opt into a text message 

component that sends texts once a week for 12 weeks to reinforce information from the program. 

Research summary: IN·clued was evaluated using a cluster randomized controlled trial involving 

1,401 LGBTQ youth ages 14–22 who were recruited through Planned Parenthood affiliates in 16 

states: Alaska, California, Connecticut, Iowa, Massachusetts, Michigan, Minnesota, Montana, 

Nebraska, North Carolina, Oregon, Rhode Island, Texas, Utah, Washington, and Wisconsin 

(Philliber 2021). The study found that 12 months after program end, participating youth were 

significantly less likely to report having had vaginal sex without using a condom in the past three 

months (effect size: -0.20). The study also found that, 12 months after program end, participating 

youth reported a significantly lower frequency of vaginal sex without using a condom in the past 

three months (effect size: -0.15). The study showed no statistically significant impacts on any of 

the other sexual behavior outcomes that were assessed. The study met the review criteria for a 

high rating. 

LeadHer 

Program summary: LeadHer, a female leadership and sexual health program, is designed for 

African American and Latina girls ages 14–19. LeadHer is composed of 30 hours of in-person 

group learning, two hourlong individual sessions with a case planner, and a smartphone app to 

reinforce program concepts. An adult facilitator and several peer educators (also youth) provide 

six five-hour lessons weekly for six weeks. LeadHer is implemented at schools (after school) or 

at community centers. The program focuses on more than a dozen topics including positive selfesteem, healthy relationships, birth control use, and STIs. 

Research summary: LeadHer was evaluated using a randomized controlled trial involving 501 

female--identifying youth ages 14–19 recruited through various events held by Alternatives For 

Girls (AFG) in the Wayne County, Michigan metro area (Malofeeva et al. 2022). The study 

found that 12 months after program end, girls participating in LeadHer reported a statistically 

significant reduction in unprotected sex compared with participants assigned to the control group 

(effect size = 0.50). The study showed no statistically significant impacts on any other assessed 

sexual behavior outcomes. The study met the review criteria for a high rating. 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 8 

Linking Families and Teens (LiFT) 

Program summary: LiFT is a family-based positive youth development program for families 

with youth ages 13-19 years old. In this six-hour in-person and text-based program, youth and 

their adult caregivers receive curriculum lessons both together and separately about 

communication skills, contraception, and how to effectively access sexual health care resources 

in the community. The program is completed in one or two sessions. At program end, youth can 

opt into receiving 12 text messages meant to reinforce program content. Adult participants can 

receive a booster phone call three to five weeks after the workshop to receive additional 

resources and support. 

Research summary: LiFT was evaluated using a cluster randomized controlled trial involving 

786 high school youth ages 13–19 and the adults parenting them (Brown et al. 2021). 

Recruitment spanned 21 rural community sites (such as schools and health care settings) across 

nine states. The study found that three months after program end, youth participating in the 

program were significantly less likely than youth who were not offered the program to report a 

pregnancy (effect size = 0.37). The study showed no statistically significant impacts on any of 

the other sexual behavior outcomes it assessed. The study met the review criteria for a high 

rating. 

Peer Group Connection-High School (PGC-HS) 

Program summary: PGC-HS is a school-based positive youth development program designed to 

support 9th grade youth with their transition into high school. The program focuses on topics 

related to youth development, such as goal setting, decision making, school attachment, resisting 

peer pressure, relationship skills, and motivation. PGC-HS program content is delivered by peer 

leaders in 18 45-minute sessions over the course of one semester or a whole school year. 

Research summary: PGC-HS was evaluated using a randomized controlled trial involving 1,523 

ninth grade youth in 18 high schools in New York City and rural North Carolina (Walsh et al. 

2022). The study found that 12 months after baseline, youth participating in the program were 

significantly less likely to report ever having had vaginal sex (effect size = -0.14). The study 

showed no statistically significant impacts on any of the other sexual behavior outcomes that 

were assessed. The study met the review criteria for a high rating. 

Plan A

Program summary: Plan A is a clinic-based teen sexual health education program for Black and 

Latina women ages 18 and 19. The content is delivered using a 23-minute video shown to clients 

in private or semi-private areas as they wait for their clinical appointment with a reproductive 

health provider. The video encourages effective contraceptive use, including long-acting 

reversible contraception and dual method use to protect against STIs and HIV. 

Research summary: Plan A was evaluated using a randomized controlled trial involving 1,770 

Black and Latina women recruited from eight health centers in California's Central Valley and 

the greater Oakland area (Jenner et al. under review). The study found that at nine months after 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 9 

study enrollment, women who were offered the Plan A video as they waited for an appointment 

reported significantly fewer instances of vaginal sex without a condom (effect size = -0.10) and 

sex of any type without a condom (effect size = -0.09) in the preceding three months. The study 

showed no statistically significant impacts on any other assessed sexual behavior outcomes. The 

study met the review criteria for a high rating. 

Pulse 

Program summary: Pulse is a self-paced mobile health app designed for Black and Latina 

women ages 18–20. Pulse has six interactive modules covering reproductive health topics such 

as birth control, healthy relationships, consent, anatomy and physiology, STIs, and pregnancy 

and pregnancy testing. Within the app, there are 16 core activities available to users through text 

messages, interactive graphics, videos, and quizzes. Pulse also includes a clinic locator and 

calendar tools to facilitate accessing reproductive care. Six weeks after starting the program, 

Pulse delivers 16 pre-programmed text messages to reinforce program content. 

Research summary: The program was evaluated using two randomized controlled trials. The first 

study involved 1,304 mostly Black or Latina women between the ages of 18 and 20 living in the 

United States or U.S. territories (Manlove et al. 2020). Participants were recruited online through 

social media. The study found that women participating in the program were significantly less 

likely to report having had sex without using a hormonal or long-acting contraceptive method in 

the past six weeks (effect size = -0.24). The study showed no statistically significant impacts on 

any of the other sexual behavior outcomes that were assessed. The study met the review criteria 

for a high rating. 

The second study involved 1,013 Black and Latina women ages 18–20 living in the United States 

(Manlove et al. 2021). The study also recruited participants through social media advertisements. 

The study showed no statistically significant impacts on any assessed sexual behavior outcomes. 

It met the review criteria for a high rating. 

Vision of You (VOY) 

Program summary: VOY is a comprehensive sexual education program for youth ages 13-19. 

VOY is a self-paced, interactive, online curriculum that consists of nine 45-minute modules 

completed over a period of four to six weeks. The program focuses on topics related to 

abstinence and contraception, anatomy, STIs, consent, communicating with adults, healthy life 

skills and relationships, and future goals. 

Research summary: VOY was evaluated using a randomized controlled trial involving 790 youth 

ages 13-19 who were recruited from nontraditional education settings in several rural regions in 

Virginia (Dainis 2021). The study found that nine months after the program ended, youth 

participating in the program were significantly less likely to report having had vaginal 

intercourse without using a condom or birth control in the past three months (effect size = -0.41). 

Nine months after the program ended, youth participating in the program also reported having 

significantly fewer sexual partners in the past three months (effect size = -0.19). The study 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 10

showed no statistically significant impacts on any of the other sexual behavior outcomes that 

were assessed. The study met the review criteria for a high rating. 

Previously reviewed programs with past evidence of effectiveness and new studies reviewed 

In this most recent update to the review findings, we sought to identify and assess any new 

evidence for programs examined in previous rounds of review. Many of the 43 previouslyreviewed programs have been evaluated only once. However, a growing number of studies have 

sought to test how these programs perform when implemented on a broader scale, in different 

settings, or with different populations (Goesling 2015). In addition, a number of studies have 

extended their evaluations to incorporate other outcomes and time points. 

As part of this update to the review findings, we identified nine new publications on seven

programs examined in previous rounds of the review (Table 3). 

• Two programs (Making Proud Choices! and Power Through Choices) had new evidence of 

longer-term impacts and/or youth outcomes that were not examined in publications we had 

reviewed before. 

• One program (Families Talking Together) had new evidence of youth outcomes examined in 

previously assessed publications. 

• One of these programs, Generations, has recent evidence of effectiveness but a new study did 

not meet quality standards so the assessment of Generations did not change. 

• For the other three programs (Becoming a Responsible Teen, Choosing the Best, and 

Respecting the Circle of Life) examined in previous rounds of the review, the new 

publications did not meet the review criteria for evidence of effectiveness and did not change 

the conclusions of the previous reviews’ assessments. 

– Becoming a Responsible Teen (BART) has evidence of effectiveness from data that was collected 

more than 20 years ago. Therefore, it exceeds the 20-year eligibility window to be considered 

“recent” and eligible to be included in the TPP Evidence Review list of programs with evidence 

of effectiveness. 

Three of the nine new publications are reports or journal articles presenting the findings from 

studies funded through FYSB or OPA. 

Summary of updated review findings 

The programs that have been reviewed during both the previous and current rounds of review 

have more than one supporting impact study. Therefore, we present their findings both at the 

study level (Table 3) and at the program level, which pools the findings across studies for a given 

program (Table 4). Program-level ratings reflect the strength of findings from one or more 

studies that present findings on the same or different eligible outcomes. The program-level 

ratings are presented separately for each of the outcome domains on which a program has been 

evaluated.
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 11

Study-level findings 

In this brief (Table 3) and on the review website 

(https://tppevidencereview.youth.gov/EvidencePrograms.aspx), we summarize the findings 

(favorable, potentially favorable, indeterminate, conflicting, potentially unfavorable, and 

unfavorable) within new studies of a particular program model assessed in both current and 

previous rounds in each of five outcome domains: (1) sexual activity; (2) number of sexual 

partners; (3) contraceptive use; (4) STIs or HIV; and (5) pregnancies. The strength of the body of 

evidence within the study is represented by both the color and quantity of the icons (Key 2). 

Key 2. Strength of the findings from individual studies of programs by color and quantity of icons

Symbol Description

Favorable findings: Two or more favorable impacts and no unfavorable impacts, regardless of null 

findings

Potentially favorable findings: At least one favorable impact and no unfavorable impacts, regardless of 

null findings

Indeterminate findings: Uniformly null findings

Conflicting findings: At least one favorable and at least one unfavorable impact, regardless of null findings

Potentially unfavorable findings: At least one unfavorable impact and no favorable impacts, regardless of 

null findings

Unfavorable findings: Two or more unfavorable impacts and no favorable impacts, regardless of null 

findings
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 12

Table 3. Individual study ratings by outcome domain for new publications on programs assessed 

in the current and previous rounds of the review 

Outcome domains

Program and publication

Sexual 

activity

Number of 

sexual 

partners

Contraceptive 

use

STIs or 

HIV Pregnancy

Study 

ratinga

FYSB or 

OPA 

fundedb

Programs with evidence of longer-term impacts and/or youth outcomes not examined in previously assessed 

publications

Making Proud Choices! 

(MPC) 

. . . . . . .

Cole et al. (2022) High Yes 

Power Through Choices 

(PTC) . . . . . . .

Oman et al. (2018) . . . High Yes 

Programs with evidence of effectiveness; new publications did not change that assessment

Families Talking Together 

(FTT)

. . . . . . .

Guilamo-Ramos et al. 

(2020)

. . . . High No

Santa Maria (2020) . . . . High No

Generations . . . . . . .

Lewin et al. (2019) n.a. n.a. n.a. n.a. n.a. Low No

Programs with no evidence of effectiveness, new publications did not change that assessment

Becoming a Responsible 

Teen3

. . . . . . .

Shepherd et al. (2017) n.a. n.a. n.a. n.a. n.a. Low No

Choosing the Best . . . . . . .

Floren et al. (2022) n.r. . . . . High No

Shepherd et al. (2017) n.a. n.a. n.a. n.a. n.a. Low No

Respecting the Circle of 

Life

. . . . . . .

Tingey et al. (2021) . . . . High Yes 

Note: Refer to Key 2 for icon descriptions. Empty cells indicate that the study did not examine program impacts 

on measures within that outcome domain. Programs are listed alphabetically within each evidence 

category. Becoming a Responsible Teen had evidence of effectiveness previously, but the study that 

presents this evidence has a publication date that is now outside of the 20-year eligibility window.

a The review team established quality ratings for the publications listed in this table by following pre-specified criteria 

to assess study design, attrition, baseline equivalence, reassignment of sample members, and confounding factors. 

Table 4 has a detailed description.

b This column indicates which publications are reports or journal articles presenting the findings from studies of 

programs that were developed and tested with federal grant funding from FYSB or OPA.

n.a. = not applicable; following the procedures specified in the review protocol, the review team did not assess 

evidence of effectiveness for programs with a supporting impact study that received a quality rating of low.

n.r. = no findings presented that meet TPPER reporting standards. 

3 Becoming a Responsible Teen did have evidence of effectiveness before; however, the study that presents this 

evidence has a publication date now outside the 20-year eligibility window. 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 13

Detailed descriptions of the new evidence of positive impacts available for three programs 

follow: (1) Families Talking Together (FTT), (2) Making Proud Choices! (MPC), and (3) Power 

Through Choices (PTC). 

Families Talking Together (FTT)

Program summary: Families Talking Together is a single-session, 45-to-60-minute parent-based 

program that aims to increase parents’ effective communication and parenting skills, build 

parent-adolescent relationships, and teach adolescents assertiveness and refusal skills. The 

program is designed for African American and Hispanic caregivers of youth ages 10–14 and can 

be delivered by a health care professional to caregivers, individually or in small group settings. 

The caregivers receive a follow-up check-in call placed one month after the session. 

Research summary: FTT was evaluated in two previous studies, which were reviewed in earlier 

rounds of the HHS TPP Evidence Review. The first study was a randomized controlled trial 

involving 264 African American and Hispanic youth recruited in the waiting room of a 

community-based pediatric health clinic in New York City (Guilamo-Ramos et al. 2011a). This 

first study found that nine months after the program ended, youth in the treatment group were 

significantly less likely to report ever having engaged in vaginal intercourse and reported 

significantly lower frequency of sexual intercourse in the past 30 days than youth in the control 

group. The second study was a randomized controlled trial involving 2,016 African American 

and Hispanic youth recruited from New York City middle schools (Guilamo-Ramos et al. 

2011b). This study did not show statistically significant impacts on any assessed sexual behavior 

outcomes. Both of these studies met the review criteria for a high rating. 

In the current round of review, we assessed two more recent studies of FTT. The first evaluated 

FTT in a randomized controlled trial involving 900 mother-child dyads from a pediatric clinic in 

New York City (Guilamo-Ramos et al. 2020). The study found that 12 months after study 

enrollment, youth in the program group were significantly less likely to report having had 

vaginal sex (effect size= -0.84) and having had their sexual debut in the past 12 months (effect 

size = -0.76). This study did not show statistically significant impacts on any other assessed 

sexual behavior outcomes. The study met the review criteria for a quality rating of high. 

The second study evaluated a different version of FTT that included an education program on 

HPV vaccination (FTT + HPV) in a randomized controlled trial involving medically underserved 

youth ages 11–14 (Santa Maria 2020). Parents and their children were recruited as dyads from 22 

after-school programs and 19 charter schools in the Houston, Texas, area. The study found at 

both one month and six months after study enrollment that youth who had received FTT + HPV 

and those in the control group were not significantly different in their reports of ever having had 

oral, vaginal, or anal sex. This study met the review criteria for a high quality rating. 

Making Proud Choices! (MPC) 

Program summary: Making Proud Choices! is a comprehensive sexuality education program that 

aims to provide youth with the knowledge, confidence, and skills necessary to abstain from sex 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 14

or use condoms if they choose to have sex. The MPC program has been revised five times over 

the past two decades, but the overall goals and messages of the curriculum have not changed. At 

the time of the current round of review, there were three versions of the MPC curriculum for 

different implementation settings or populations: original, school, and out-of-home. The original 

edition consisted of eight one-hour lessons implemented in community settings; the school 

edition consisted of 14 40-minute lessons implemented in classroom settings; and the out-ofhome edition consisted of 10 75-minute lessons implemented in juvenile justice settings, foster 

homes, or independent and transitional living facilities. 

Research summary: The original version of the program was evaluated in one study, which was 

reviewed in a previous round of the HHS TPP evidence review. That study, published in 1998, 

examined the effectiveness of MPC using an individual-level random assignment design 

involving 6th- and 7th-grade African American youth in Philadelphia (Jemmott et al. 1998). The 

study found that three months after the end of the program, youth participating in MPC who 

were sexually experienced at baseline were significantly less likely to report having had sexual 

intercourse without a condom in the past three months, and reported a lower frequency of sexual 

intercourse without a condom. The study met the review criteria for a high rating. In the current 

round of review, we assessed a more recently published study of the current version of MPC that 

involved 2,035 male and female youth recruited from 15 high schools in Mobile, Alabama; 

Detroit, Michigan; St Louis, Missouri; and Cincinnati, Ohio (Cole et al. 2022). In this study, 

researchers examined the effectiveness of the MPC school edition using a cluster randomized 

controlled trial. The study found that six months after the end of the program, youth participating 

in MPC reported having had sex significantly fewer times in the past three months compared 

with youth in the control group (effect size = -0.10). The study met the review criteria for a high 

rating. 

POWER Through Choices (PTC)

Program summary: POWER Through Choices is a comprehensive sexual health education 

program focused on teen pregnancy, HIV, and STI prevention, and is designed to serve youth in 

out-of-home care settings such as foster care and juvenile justice facilities. The program is 10 

sessions long, with each session lasting 90 minutes and delivered once or twice a week for 5 to 

10 weeks. The sessions include interactive skill-building activities that emphasize selfempowerment and the impact of choices.

Research summary: The results of an evaluation of PTC were presented in two publications, one 

of which met the review criteria and was reviewed in a previous round of the HHS TPP evidence 

review. That manuscript, published in 2016, documented an evaluation of PTC using a cluster 

randomized controlled trial involving 885 youth from 44 residential group homes in California, 

Maryland, and Oklahoma (Covington et al. 2016). Twelve months after the program ended, 

youth participating in PTC were significantly less likely to report having ever been pregnant or 

gotten someone pregnant. The study met the review criteria for a high rating. In the current round 

of review, we assessed a publication that presented six-month outcome data from the same study 

(Oman et al. 2018), which had not previously been reported. According to the study, six months 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 15

after the end of the program the youth participating in PTC were significantly less likely to have 

had sex without using birth control in the preceding three months (effect size = -0.20). In 

addition, when making slightly different analytic decisions, this study confirmed the 12-month 

findings for youth being significantly less like to have ever been pregnant or gotten someone 

pregnant. The manuscript met the review criteria for a high rating. 

Program-level findings 

Below (Table 4) and on the review website 

(https://tppevidencereview.youth.gov/EvidencePrograms.aspx), we summarize the available 

evidence (favorable, potentially favorable, indeterminate, conflicting, potentially unfavorable, 

and unfavorable) across studies on a particular program model in each of five outcome domains: 

(1) sexual activity; (2) number of sexual partners; (3) contraceptive use; (4) STIs or HIV; and (5) 

pregnancies. The strength of the body of evidence behind all studies that meet evidence 

standards for a given program model is represented by the color, quantity, and size of the icons 

(Key 1). Visitors to the review website may use this information to compare program models that 

match their outcome domain of interest, as well as the desired program type, population, age 

range, or implementation setting. In this brief, we only include program-level findings for the 

programs assessed in both the current and previous rounds of the review. 

As shown in Table 4: 

• Making Proud Choices! added data to each outcome domain (the previous study only had 

evidence in the sexual activity and contraceptive use domains). It now has a potentially 

favorable rating for the sexual activity domain (changed from a previous rating of 

indeterminate (null)) and has maintained its potentially favorable rating for the contraceptive 

use domain. 

• Power Through Choices now has a potentially favorable rating for the contraceptive use 

domain and has maintained its potentially favorable evidence rating for the pregnancy 

domain. 

• Families Talking Together has added data to the sexual activity domain and has maintained 

its potentially favorable rating for this domain. 

• Although we reviewed new research on the Generations program, that research received a 

rating of low and therefore it did not add data or change the rating of the program. 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 16

Table 4. Evidence by outcome domain for programs assessed in the current and previous rounds 

of the review 

Outcome domains

Program/publication

Sexual 

activity

Number of 

sexual 

partners

Contraceptive 

use STIs or HIV Pregnancy

Programs with evidence of longer-term impacts and/or youth outcomes not examined in previously 

assessed publications

Making Proud Choices! (MPC)

Power Through Choices (PTC)

Programs with evidence of effectiveness, new publications did not change that assessment

Families Talking Together (FTT) . . . .

Generations . . . .

Programs with no evidence of effectiveness, new publications did not change that assessment

Becoming a Responsible Teen n.a. n.a. n.a. n.a. n.a.

Choosing the Best n.r. . . . .

Respecting the Circle of Life . . . .

Note: Refer to Key 1 for icon descriptions. Empty cells indicate that the study did not examine program impacts 

on measures within that outcome domain. Programs are listed alphabetically within each evidence 

category.

n.a. = not applicable; following the procedures specified in the review protocol, the review team did not assess 

evidence of effectiveness for programs with a supporting impact study that received a low quality rating.

n.r. = no findings presented that meet TPPER reporting standards. 

Review procedures 

For this update to the review’s findings, the review team followed procedures similar to those 

used for prior rounds of the review. In March 2022, the team released a public call for studies, 

requesting new research for consideration. The team also identified studies through a 

comprehensive literature search that entailed keyword searches of electronic databases, manual 

searches of relevant academic journals and professional conference proceedings, and reviewed 

citations in recently published literature reviews and meta-analyses. The team screened the 

resulting studies against pre-specified eligibility criteria. 

For studies that met the eligibility criteria, the team first assessed the quality and execution of 

each study’s research design (Table 5). The reviewers assigned each study a quality rating of 

high, moderate, or low according to the risk of bias in the study's impact findings. A more 

detailed description of these ratings is in the version 6.0 review protocol, available online at the 

review website (https://tppevidencereview.youth.gov/ReviewProtocol.aspx). This round of the 

TPPER did not include a rereview of studies that were previously reviewed under earlier 

versions of the standards. 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 17

For the subset of studies that achieved a moderate or high quality rating, the review team 

extracted information on the features of the program being tested, evaluation setting, study 

sample, and research design, and detailed information on the program impact estimates. The 

review team then identified programs with evidence of effectiveness, defined as having a 

statistically significant favorable impact (and no adverse effects) on at least one priority outcome 

measured for either the full analytic sample or a subgroup defined by (1) gender or (2) sexual 

experience at baseline. The priority outcomes include sexual activity, number of sexual partners, 

contraceptive use, STIs or HIV, and pregnancy. 

When the study team updated the review findings, we also changed the review criteria. We 

applied the changes when we reviewed the studies of the 25 programs described above. The first 

change to the review criteria is the new requirement for baseline equivalence for cluster 

randomized controlled trials (RCTs) to achieve a moderate rating. The second change is to the 

baseline equivalence criterion; TPPER now relies on the magnitude, not the statistical 

significance, of the difference in key characteristics to determine whether baseline equivalence 

has been satisfied or if a statistical adjustment is necessary to address small underlying 

differences between a treatment and control group. The third change is increased transparency in 

TPPER review approaches, as TPPER now provides detailed guidance on aspects of its review 

process (for instance, allowable imputation of missing data and how baseline equivalence is 

assessed for categorical data). In addition, TPPER is using a new approach to categorize and 

present credible evidence. There is an expanded rating scheme for outcome domains and 

program ratings that pool within outcome domains across studies for specific programs. More 

details on the review process and criteria are available on the review website. 

Table 5. Summary of study quality ratings

Criteria category

Features of studies with the 

high study rating

Features of studies with the

 moderate study rating

Study design Random or functionally random 

assignment

Random assignment design with high attrition or 

reassignment; Quasi-experimental design with a 

comparison group

Attrition Random assignment studies that do not 

exceed What Works Clearinghouse 

standards for overall and differential 

attrition (cautious assumption)

Random assignment studies that exceed What 

Works Clearinghouse attrition standards;

Attrition is not assessed in quasi-experimental 

designs

Baseline equivalence Not assessed – samples are assumed to 

be equivalent by virtue of random 

assignment and low levels of sample 

attrition

The equivalence of the research groups is 

demonstrated at baseline, and systematically 

adjusted for in impact analyses

Reassignment Analysis is based on original assignment 

to research groups

Not assessed, given the baseline equivalence 

requirement described above that ensures 

equivalence of the research groups

Confounding factors At least two subjects or groups in each 

research group and no systematic 

differences in data collection methods

At least two subjects or groups in each research 

group and no systematic differences in data 

collection methods

Note: Studies that do not achieve the high or moderate rating are given a “low” study rating. Refer to the protocol

for more details on applying the study quality ratings.
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 18

As described in the introduction, 3 programs with evidence of effectiveness were moved to the 

inactive programs list on the TPPER website (Table 6). Implementation information and 

reviewed evidence are still available to view on the website. Inactive programs may become 

active again in the future, at which point they will be moved to the active list of programs. 

Table 6. Summary of programs moved to inactive list

Program name Reason for inactive classification

Guy2Guy Program is not being disseminated 

Making a Difference! Program’s only evidence is older than 20 years

Respeto/Proteger Program is not being disseminated
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 19

References

Brown, Sally A., Rita E. Turner, and Cindy Christensen. “Linking Families and Teens: 

Randomized Controlled Trial Study of a Family Communication and Sexual Health 

Education Program for Rural Youth and Their Parents." Journal of Adolescent Health, vol. 

69, no. 3, 2021, pp. 398–405. 

Centers for Disease Control and Prevention (CDC). “Adolescent and School Health: 

Terminology.” CDC, n.d. 

Cole, R., T. Schulte Neelan, A. Langan, B. Keating, J. Walzer, S. Asheer, and S. Zief. “The 

Impact of the Making Proud Choices! Teen Pregnancy Prevention Curriculum.” Office of 

Population Affairs, Office of the Assistant Secretary for Health, U.S. Department of Health 

and Human Services, 2022. 

Covington, Reginald D., Brian Goesling, Christina Clark Tuttle, Molly Crofton, Jennifer 

Manlove, Roy F. Oman, and Sara Vesely. “Final Impacts of the POWER Through Choices 

Program.” Mathematica Policy Research, September 2016.

Coyle, K., P. Anderson, B.A. Laris, M. Barrett, T. Unti, and E. Baumler. “A Group Randomized 

Trial Evaluating High School FLASH, a Comprehensive Sexual Health Curriculum." Journal 

of Adolescent Health, vol. 68, no. 4, 2021, pp. 686–695. 

Dainis, A. Evaluation of Vision of You in the Commonwealth of Virginia. Dainis & Company, 

Inc, 2021. 

Floren, M., and K. Floren. “Impact of the Choosing the Best Program on Sexual Activity Among 

Students in Communities Committed to Abstinence Education: Additional Analysis 

Findings.” Practical Statistics, LLC, 2022. 

Goesling, Brian. “Making Sense of Replication Studies: Guidance for the Field of Teen 

Pregnancy Prevention Research.” U.S. Department of Health and Human Services, Office of 

the Assistant Secretary for Planning and Evaluation, 2015. 

Guilamo-Ramos, V., A. Benzekri, M. Thimm-Kaiser, P. Dittus, Y. Ruiz, C.M. Cleland, and W. 

McCoy. “A Triadic Intervention for Adolescent Sexual Health: A Randomized Clinical 

Trial.” Pediatrics, vol. 145, no. 5, 2020. 

Guilamo-Ramos, V., A. Bouris, J. Jaccard, B. Gonzalez, W. McCoy, and D. Aranda. (2011a). “A 

Parent-Based Intervention to Reduce Sexual Risk Behavior in Early Adolescence: Building 

Alliances Between Physicians, Social Workers, and Parents.” Journal of Adolescent Health, 

vol. 48, no. 2, 2011, pp. 159–163. 

Guilamo-Ramos, V., J. Jaccard, P. Dittus, A. Bouris, B. Gonzalez, E. Casillas, and S. Banspach. 

(2011b). “A Comparative Study of Interventions for Delaying the Initiation of Sexual 

Intercourse Among Latino and Black Youth.” Perspectives on Sexual and Reproductive 

Health, vol. 43, no. 4, 2011, pp. 247–254. 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 20

Jemmott, J. B., L.S. Jemmott, and G.T. Fong. “Abstinence and Safer Sex HIV Risk-Reduction 

Interventions for African American Adolescents: A Randomized Controlled Trial.” JAMA: 

Journal of the American Medical Association, vol. 279, no. 19, 1998, pp. 1529–1536. 

Jenner, E., S. Walsh, C. Henley, H. Demby, R. Leger, and G. Falk. (Under review). 

“Randomized Trial of a Sexual Health Video Intervention for Black and Hispanic Adolescent 

Females.”

Lewin, A., S.J. Mitchell, D.A. Quinn, T.M. Street, K. Schmitz, and L.S. Beers. “A Primary Care 

Intervention to Prevent Repeat Pregnancy Among Teen Mothers.” American Journal of 

Preventive Medicine, vol. 56, no. 3, 2019, pp. 404–410. 

https://doi.org/10.1016/j.amepre.2018.10.015

Malofeeva, E., A.S. Warnke, H. Rogers, and K. Browning. “Evaluation of LeadHer, a Pregnancy 

Prevention Curriculum, in Detroit, Michigan.” Evaluation Strategies, 2022. 

Manlove, Jennifer, Elizabeth Cook, Brooke Whitfield, Makedah Johnson, Genevieve MartinezGarcia, and Milagros Garrido. "Short-Term Impacts of Pulse: An App-Based Teen 

Pregnancy Prevention Program for Black and Latinx Women.” Journal of Adolescent Health, 

vol. 66, no. 2, 2020, pp. 224–232. 

Manlove, J., Brooke Whitfield, Jane Finocharo, and Elizabeth Cook. "Lessons Learned from 

Replicating a Randomized Control Trial Evaluation of an App-Based Sexual Health 

Program." International Journal of Environmental Research and Public Health, vol. 18, no. 6, 

2021, pp. 3305. https://doi.org/10.3390/ijerph18063305. 

Oman, R. F., S.K. Vesely, J. Green, K. Clements-Nolle, and M. Lu. “Adolescent Pregnancy 

Prevention Among Youths Living in Group Care Homes: A Cluster Randomized Controlled 

Trial.” American Journal of Public Health, vol, 108, S1, 2018, pp. S38–S44. 

Philliber, Ash. "The IN·clued Program: A Randomized Control Trial of an Effective Sex 

Education Program for Lesbian, Gay, Bisexual, Transgender, Queer, and Questioning 

Youths." Journal of Adolescent Health, vol. 69, no. 4, 2021, pp. 636–643, 

Rotz, Dana, Dara Lee Luca, Brian Goesling, Elizabeth Cook, Kelly Murphy, and Jack Stevens. 

“Final Impacts of the Teen Options to Prevent Pregnancy Program.” Mathematica Policy 

Research, July 2016. 

Santa Maria, Diane. “Efficacy of a Student-Nurse Brief Parent-Based Sexual Health Intervention 

to Increase HPV Vaccination Among Adolescents.” Journal of Adolescent Health, vol. 66, 

no. 2, Supplement, 2020, S4–S5. 

Shepherd, L. M., K.F. Sly, and J.M. Girard. “Comparison of Comprehensive and AbstinenceOnly Sexuality Education in Young African American Adolescents.” Journal of 

Adolescence, vol. 61, 2017, pp. 50–63. https://doi.org/10.1016/j.adolescence.2017.09.006

Smith, Kim, Dana Rotz, Brian Goesling, Elizabeth Cook, Kelly Murphy, and Jack Stevens. 

“Interim Impacts of the Teen Options to Prevent Pregnancy Program.” Mathematica Policy 

Research, 2015. 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 21

Stevens, J., R. Lutz, N. Osuagwu, D. Rotz, and B. Goesling. “A Randomized Trial of 

Motivational Interviewing and Facilitated Contraceptive Access to Prevent Rapid Repeat 

Pregnancy Among Adolescent Mothers.” American Journal of Obstetrics and Gynecology, 

vol. 217, no. 4, 2017, pp. 423.e1–423.e9. 

Tingey, L., R. Chambers, H. Patel, S. Littlepage, S. Lee, A. Lee, D. Susan, L. Melgar, A. Slimp, 

and S. Rosenstock. “Prevention of Sexually Transmitted Diseases and Pregnancy Prevention 

Among Native American Youths: A Randomized Controlled Trial, 2016–2018.” American 

Journal of Public Health, vol. 111, no. 10, 2021, pp. 1874–1884. 

Walsh, S., E. Jenner, N. Qaragholi, C. Henley, H. Demby, R. Leger, and K. Burgess. “The 

Impact of a High School-Based Positive Youth Development Program on Sexual Health 

Outcomes: Results from a Randomized Controlled Trial.” Journal of School Health, vol. 92, 

2022, pp. 1155–1164. DOI: 10.1111/josh.13216. 

Ybarra, M. L., T.L. Prescott, G.L. Phillips II, S.S. Bull, J.T. Parsons, and B. Mustanski. “Pilot 

RCT Results of an mHealth HIV Prevention Program for Sexual Minority Male 

Adolescents.” Pediatrics, vol. 140, no. 1, 2017, DOI: 10.1542/peds.2016-2999. 

Ybarra, Michele, Carol Goodenow, Margaret Rosario, Elizabeth Saewyc, and Tonya Prescott. 

"An mHealth Intervention for Pregnancy Prevention for LGB Teens: An RCT." Pediatrics, 

vol. 147, no. 3, March 2021. 
U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation 

Page 22

Appendix

Table A1. Descriptions of studies reviewed that did not change assessments for programs 

previously reviewed

Program/publication

Quality 

ratinga Description

Programs with evidence of effectiveness, new publications did not change that assessment

Generations

Lewin et al. (2019) Low This publication presents the findings from a prospective quasiexperimental evaluation of the Generations program, a patientcentered medical home program for families that include teenage 

parents. This study received a low study quality rating because it is 

a quasi-experimental design that does not demonstrate that the 

treatment and control groups were sufficiently similar before the 

introduction of the program.

Programs with no evidence of effectiveness, new publications did not change that assessment

Becoming a Responsible Teen4

Shepherd et al. (2017) Low This publication presents the findings from a study of two programs: 

Becoming a Responsible Teen, a comprehensive sexuality 

education program, and Choosing the Best Path, an abstinence-only 

sexuality education program. This study received a low study quality 

rating because there was only one school in each condition.

Choosing the Best

Floren et al. (2022) High This publication presents findings from a cluster randomized 

controlled trial of Choosing the Best, an 8-session sexual risk 

avoidance program delivered to 9th graders during regular health 

classes. This study involved 1,143 youth ages 14 to 17 in two school 

districts in suburban Georgia. The study did not report any findings

on sexual initiation that meet TPPER reporting standards – the study 

findings do not account for the clustered study design. 

Shepherd et al. (2017) Low This publication presents the findings from a study of two programs: 

Becoming a Responsible Teen, a comprehensive sexuality 

education program, and Choosing the Best Path, an abstinence-only 

sexuality education program. This study received a low study quality 

rating because there was only one school in each condition.

Respecting the Circle of Life

Tingey et al. (2021) High This publication presents findings from a randomized controlled trial 

of Respecting the Circle of Life, an eight-lesson comprehensive 

sexual and reproductive health education program for Native 

American youth. The study involved 534 youth ages 11–19 in a rural 

reservation community in the southwestern United States. The study 

found no evidence of favorable program effects on participants’ 

history of vaginal sex at 9 or 12 months after the program. 

a The review team established quality ratings for the publications listed in this table by following pre-specified criteria 

to assess study design, attrition, baseline equivalence, reassignment of sample members, and confounding factors. 

Table 5 has a more detailed description of the study quality ratings.

4 Becoming a Responsible Teen did have evidence of effectiveness before; however, the study that presents this 

evidence has a publication date now outside the 20-year eligibility window.