An Interactive Education Program to Reduce High Risk Behavior in Adolescents Ph I
Conditions: Risky Sexual Behavior, Emotional Regulation, Adolescent Behavior
Sponsor: Klein Buendel, Inc.

PROTOCOL AND STATISTICAL ANALYSIS PLAN 
 
 
 
 
An Interactive Education Program to Reduce High Risk Behavior 
in Adolescents Phase I (1R42HD110333-01) 
 
 
NCT05607784  
 
 
09/27/2022 
 
 

 
Version 1.2  9/27/22                                                                              1 
 
Note. We have highlighted information in the protocol relating to Phase 2 activities in grey. Such information is included 
for reference but does not pertain to the current package submission. We are only seeking approval for Phase 1 activities 
at this time. 
 
SIGNIFICANCE 
Early Adolescence is a Critical Time for Sexual Health Intervention. Partnered sexual behaviors often 
begin in early adolescence (ages 12-14 years). Youth Risk Behavior Surveillance data indicate that while only 
3% of adolescents have had intercourse before age 13, by 10th grade, over a third have had sex, highlighting 
the importance of the early adolescent years.2 The behavioral patterns established in adolescence have pow-
erful implications for current and future health,3 yet CDC data indicate that only 53% of Americans receive for-
mal sex education before age 18.144 Those who have sex in early adolescence continue to exhibit greater sex 
risk (more sex partners, substance use with sex, less condom use) than their peers as they age,4-6 suggesting 
a persistent lifestyle pattern with negative outcomes. Indeed, early sexual debut is associated with more unin-
tended pregnancies7Error! Hyperlink reference not valid. and STIs.8 Early adolescent girls who have sex 
are also less likely to use contraception and wait longer to start doing so.9 These effects extend to other health 
behaviors; early adolescent sex predicts higher frequency of alcohol use at age 16 and problematic use at age 
22,10 even controlling for confounds such as puberty, parental monitoring, and antisocial behavior. Delaying 
sexual initiation and enhancing skills related to decision making can have profound effects on long-term health. 
Early adolescence is a critical time to intervene, as attitudes about sex change dramatically in this develop-
mental window.11 Cognitions about sex change before adolescents engage in sex,11 and values and intentions 
in early adolescence are predictive of sex in middle adolescence.12 The relationship between positive feelings 
about sex and sexual initiation is strongest when adolescents are younger.13 Intervening during early adoles-
cence is an opportunity to target sexual values, cognitions, and feelings before they are well established.  
Biological changes in early adolescence also put youth at risk via both sexual development and emotion 
regulation (ER). Physical changes during puberty can elicit sexual advances, and hormonal changes are linked 
to adolescent sexual behavior.14,15 Relatedly, brain development in early adolescence may affect skills related 
to self-regulation, often leading to impulsive decisions. MRI data show that teens’ frontal lobes, which manage 
executive functions such as impulse control, ER, and reasoning, mature later than other brain areas.16,17 For 
example, young adolescents’ performances on tasks of emotion recognition tend to be poor in early adoles-
cence, and this poor performance has been linked to high activation of the amygdala and low activation of the 
frontal lobes when processing these tasks. Performance improves as teens age, when their frontal lobes be-
come more active in this processing.18 These difficulties in emotion processing likely impact early teens’ under-
standing of their own (and others’) feelings, making it difficult to recognize and regulate emotions in affect-
laden situations, such as those involving sex or substance use (which has been shown to influence sexual 
risk).19-21 Indeed, poor ER in adolescence is related to more sex partners22 and more substance use.23-25 Be-
cause psychosocial and biological factors predispose early adolescents to emotion dysregulation and sexual 
risk, improving ER during this developmental period is likely to affect their lifetime health trajectories.  
Emotion Regulation (ER) Mitigates Risky Behavior. The most widely used and supported model26 of ER is 
Gross’ process model,27 which conceptualizes ER as the process of “shaping which emotions one has, when 
one has them, and how one experiences or expresses these emotions.”27 ER uses multiple skills, including 
identifying affect in others, recognizing one’s own emotions, and using strategies to manage one’s emotional 
response. Targeted emotions are often negative, but positive emotions can require regulation too, as when cu-
riosity or attraction leads to sex. The process model identifies five families of ER strategies: situation selection, 
situation modification, attentional deployment, cognitive change, and response modulation. These target emo-
tion management at various points in the emotion process (e.g., attending to a stimulus, appraising a stimulus, 
physical/ cognitive responses). 
Research has demonstrated that when people experience high levels of negative emotion, they act impul-
sively and prioritize short-term soothing to decrease distress.28 Indeed, many studies document a lack of rela-
tionship between knowledge of potential negative outcomes and behavior,29-31 suggesting that other processes 
interfere with the use of facts. Teens who engage in risk behaviors are more likely to report difficulty with ER,32 
and better ER has been longitudinally associated with less adolescent risk taking.33 Moreover, poorer self-reg-
ulation of emotions and behavior in early adolescence is associated with sexual risk in later adolescence.22,34 
These relationships likely exist because adolescents’ ER difficulties increase the likelihood of impulsive self-
soothing behaviors, such as sex. However, this work is hampered by a preponderance of associational studies 
and few developmentally tailored interventions that examine ER. 
Most sexual risk prevention interventions target knowledge and behavioral skills.35 Without ER skills train-
ing, these approaches are unlikely to influence the ways in which teens respond to risk situations. Poor ER 
may short-circuit effective use of knowledge and behavioral skills in risk situations, which are typically marked 
by high emotionality. This is particularly important because decision making surrounding risky sexual behavior 

 
Version 1.2  9/27/22                                                                              2 
 
occurs during strong positive and negative emotions. ER training addresses the heightened emotions of these 
situations, enabling teens to implement knowledge and skills to better navigate these moments. The proposed 
intervention moves the field forward by focusing on emotion regulation to enhance sexual health education to 
reduce negative health outcomes. While a few digital sexual health interventions have been developed for ado-
lescents, they do not focus on ER, and do not show impact on sexual initiation,36-38 leaving a gap in the field. 
Other recent interventions have focused on brief face-to-face interventions with parents to encourage sexual 
health communication39,40 or group educational interventions for high risk adolescent subgroups,19,41 but they 
do not target ER. Similarly, programs targeting social emotional learning (SEL) exist, but these typically have 
lengthy curricula administered in groups, with a wide range of goals (e.g., organizational skills, personal goal 
setting, compassion, conflict resolution) that do not explicitly connect to health behaviors. These broad reach-
ing programs serve important but distinct functions to those directly teaching efficacious, evidence-based ER 
skills. Without ER skills training, sexual health education is unlikely to influence the ways in which teens act in 
high-emotion risk situations. Learning facts is not enough; ER training teaches adolescents to regulate emo-
tions in ways that allow them to implement other skills (e.g., communication skills, condom use skills) to posi-
tively influence sexual health and outcomes. 
Targeting ER during early adolescence, when ER and cognitions about sex are developing and dynami-
cally interacting, increases the likelihood of improving ER throughout adolescence and increasing healthy deci-
sion making. While a few other group-based programs have been developed to specifically target ER con-
structs (rather than broader SEL) among adolescents42-44, the proposed program goes beyond existing ones by 
directly relating ER to sexual behavior and eliminating challenges inherent in group-based interventions.45 It is 
based on the only proven efficacious program to target ER and demonstrate longitudinal impact on sexual risk 
reduction among adolescents from middle school into high school (Project TRAC). Our team has demonstrated 
that ER skills can be effectively taught at the developmental level of early adolescence and that these skills 
change sexual behavior over 2.5-year follow-up. We seek to extend this work by completing the technology 
translation of our novel, effective intervention and testing it in a randomized controlled trial.  
Preliminary Studies. Project TRAC (Talking about Risk and Adolescent Choices; R34 MH07875, R01 
NR011906) is an efficacious, developmentally tailored intervention teaching ER skills for early adolescents that 
focuses on strategies to use in moments of sexual decision making (e.g., peer pressure to have sex, decisions 
to use a condom). It contains 3 main components: ER training, sex education, and the link between emotions 
and risk behaviors. The program was developed using qualitative research with early adolescents46 and em-
phasizes emotion recognition and labeling, connecting emotions and behavior, and the use of ER strategies 
(Get Out, Let It Out, and Think It Out) consistent with Gross’ process model. Sexual health information, includ-
ing sexual development, safer sexual behaviors, and condoms, is also included in the program. Project TRAC 
integrates ER into all topics (e.g., discomfort with condoms, managing emotions that occur in sexual possibility 
situations, managing peer pressure) as the key construct that is absent from previous adolescent sexual health 
interventions. In an efficacy trial (n=420) of urban early adolescents (ages 12-14), Project TRAC’s ER condi-
tion outperformed a rigorously matched health promotion (HP) comparison, which provided sexual and 
general health education without ER content, in delaying sexual initiation and reducing sexual risk.47-49 
The sample was diverse in terms of ethnicity (38% Latino), race (28% Black), and SES (30% < $20,000) and 
balanced in terms of gender (53% male). The study tar-
geted those at highest risk due to mental health symp-
toms. There were no significant differences between 
the conditions on baseline demographics, sexual ac-
tivity, emotional and behavioral symptoms, or attend-
ance at the 12 after school sessions (9.1 vs. 9.0).  
The TRAC ER condition was efficacious in 
slowing the time to sexual initiation. Time-to-event 
analyses showed that ER participants were signifi-
cantly less likely to begin having vaginal sex (the pri-
mary outcome) over the 1-year follow-up (Adjusted 
Hazard Ratio [AHR]=.58 [95% CI: .36 to .94])49 and 
this effect maintained to 30 months (AHR=.61 [.42 to 
.89];47 See Figure 1). Further analyses indicate rela-
tively similar effects between those with and without 
clinically significant mental health symptoms (AHRsub-
clinical= .60 [.31-1.17]; AHRclinical=.65 [.42-.99]47), suggesting a promising intervention that works for multiple audi-
ences, including those at higher risk. 
The intervention also reduced sexual risk behavior. ER participants reported significantly fewer con-
domless vaginal or anal sex acts (Adjusted Rate Ratio [ARR]= .36 (.14-.90) and fewer total vaginal or anal sex 
Figure 1.  TRAC Sexual Initiation by Intervention 

 
Version 1.2  9/27/22                                                                              3 
 
acts (ARR=.39 [.20-.77]). They also were less likely than HP to endorse multiple partners (Adjusted Odds Ratio 
[AOR]=.54 [.30-.99]) or substance use before sex (AOR=.42 [.23-.75] during the 30-month follow up.5 In short, 
TRAC ER delayed sexual initiation and reduced sexual risk when youth became sexually active. 
TRAC ER also affected measures of ER abilities, the targeted mechanism. On a computerized measure of 
distress tolerance, ER participants were more likely to persist through 80% of the task than those in HP (OR= 
1.55, p= .05), suggesting better ER skills. Similarly, on a validated task of emotion recognition49,50 ER partici-
pants correctly labeled more emotions based on facial expressions than HP (Rate Ratio= 1.10, p<.01).49 These 
findings provide performance-based evidence of the efficacy of TRAC ER in affecting emotional competence 
that correspond to adolescent self-reports of greater use of ER strategies, Cohen’s d= .30 [.25-.34].48 
Barriers to Implementation and Solutions. TRAC ER reduces adolescent sexual risk, but is time-, re-
source-, and training-intensive, and thus possesses challenges to dissemination. Its interactive activities were 
designed for coordinated, small groups and require significant facilitator training in ER concepts. Our school 
partners have indicated that while they want to continue offering this program, these challenges are prohibitive. 
Indeed, it is common for evidence-based interventions (EBIs) to encounter problems during broader dissemi-
nation due to passive diffusion techniques.51-53 Active techniques, such as technology use, offer an opportunity 
to expand EBI reach.54-56 The mood and anxiety literatures for both adults and children have demonstrated that 
the translation of traditionally delivered EBIs to technology 1) can be effective,57-67 2) has high feasibility and 
acceptability,68-70 and 3) can achieve high retention and adherence.41, 66, 69, 70 Technology is nearly ubiquitous in 
teens’ lives, and over 90% report playing video games.71 Further, middle schoolers’ preferred device for learn-
ing at school is tablets.72 A tablet-based adaptation of TRAC offers many advantages over group-based train-
ing. Logistically, a tablet intervention reduces staff and training in addition to long-term costs as tablets become 
more accessible. For learning uptake, tablet administration provides all individuals opportunities to practice 
skills, whereas the group format often could not accommodate practice for everyone. It also provides individu-
alized feedback at the participant’s pace, improving comprehension, and allows evaluation of interaction with 
the program. Scheduling barriers are removed, so participants do not miss groups and thus receive all content. 
Finally, some adolescents may prefer sexual health content that is provided in a less public setting than group. 
Findings for the preliminary development of an internet-based TRAC (iTRAC) support that the ER constructs 
can be taught through these games and that early adolescents liked it. In fact, as expected, findings support 
that our tablet-based intervention had stronger effects on ER constructs than the group format. 
iTRAC Pilot Study (R21 HD089979). To establish that the ER skills training from TRAC ER could be effec-
tively translated to a game format, the investigative team completed a study to 1) translate the ER components 
of TRAC ER into interactive web app activities, 2) assess usability and acceptability, and 3) conduct a small, 
randomized trial to assess intervention feasibility and the impact on ER constructs. Due to the focus on demon-
strating feasibility and preliminary efficacy of digital ER training, as well as the scope of funding, sexual health 
components were not developed. With the help of two youth advisory boards (YABs; one male, one female) 
and input from an expert panel (psychologists and school professionals), TRAC ER components (only) were 
adapted to develop the foundation of an internet-based TRAC (see Table 1). With guidance from the YABs (14 
meetings) and expert panel (3 meetings), activities comprising 
four 30- to 45-minute modules were developed to target TRAC’s 
core ER constructs. Using YAB feedback, an engaging space 
theme was chosen. Users are “cadets” in a mandatory training on 
a planet in which the young aliens are making unhealthy deci-
sions because of unmanaged feelings; they must complete a se-
ries of challenges to learn ER skills. After the translation phase, 
two rounds of acceptability testing were conducted with ten more 
7th graders to incorporate their feedback on program functionality, 
usability, and acceptability. The iTRAC modules were judged to 
be easy to use and acceptable, with average ratings of 4.7 (on a 
5-point scale) for ease of use, 4.3 for information helpfulness, 
and 4.3 for “overall experience.” After establishing usability and acceptability, 85 7th graders (58% female; 33% 
White) were randomized to iTRAC or a waitlist control. Participants completed study procedures after school 
under supervision. Modules were completed about 1 week apart. This format proved highly feasible; 88% 
(37/42) of those randomized to iTRAC completed all 4 modules (93% completed at least 3).  
Results from the final, 3-month follow-up showed moderate to strong effect sizes on ER constructs (d=.36-
.72), stronger than the small to moderate effect sizes on ER measures observed in the original TRAC 
trial, suggesting better efficacy on ER for the tablet-based intervention. Specifically, emotional compe-
tence for ER participants improved relative to the control group on scales assessing belief in emotions as mal-
leable (Implicit Theories of Emotion Scale; d=.72), emotional self-efficacy (Children’s Self-Efficacy Question-
naire; d=.63), emotional awareness (Difficulties in Emotion Regulation Scale (DERS); d=.59), access to ER 

 
Version 1.2  9/27/22                                                                              4 
 
strategies (DERS; d=.36), and use of ER strategies taught in TRAC (Emotion Regulation Behaviors Scale-Re-
vised (ERBS-R); d=.45). These data indicate that teaching TRAC ER’s constructs in a technology format can 
affect emotional awareness and regulation in this developmental period. The proposed STTR will improve 
upon the rigor of the pilot R21 by completing the iTRAC program to include the sexual health content and con-
tent linking ER to sexual health from the original program. It will evaluate its ability to influence self-efficacy for 
sexual risk prevention, a key construct related to sexual risk among adolescents. The study will also collect 
preliminary data regarding whether it delays sex, which was not collected during the short pilot study. The com-
pletion of iTRAC to include sex education and its content linking ER to health represents a novel, efficacious 
strategy (ER) for promoting healthy behaviors that can reach a wider audience than group-based interventions 
and has substantial commercialization potential given societal interest in healthy sexual development to avoid 
unintended pregnancy, relationship violence, and sexually transmitted diseases in adolescence and beyond.   
History of successful collaboration between the study teams. Dr. Houck has collaborated with Klein 
Buendel on three previous federally-funded projects involving the creation of digital products teaching ER skills 
to adolescents (2014-MU-CX-0002, R21 HD 089979, R01 HD 097126). These teams have established work 
processes that have led to successful completion of these projects that was built on a foundation of communi-
cation of the sophisticated concepts of ER and digital translation of behavioral interventions. Similarly, the 
Rhode Island Hospital (RIH) team has a 20-year history of collaboration with Dr. Hadley, previously at RIH and 
now at the University of Oregon, with whom they continue to collaborate (R01 HD 097126; R01 DA 050603). 
 
INNOVATION 
Integrating ER Training in Sexual Health Education for Early Adolescents: Project TRAC innovated the 
field as the first program to target ER in the context of sexual risk for early adolescents, leading to positive sex-
ual health outcomes. Although ER is touted as a promising approach for health interventions (primarily from 
work with preschoolers or clinical adults),73,74 it has been rarely implemented in sexual risk prevention, which 
has typically focused on cognitions or skill building,45,75 thus weakening the rigor of previous prevention re-
searchError! Hyperlink reference not valid.. Only two other programs76, 77 have used ER to address sexual 
risk; both targeted older teens with severe mental health problems in group formats. While a few other group-
based programs have been developed to target ER constructs generally (e.g., Learning to Breathe,42 Mindful-
ness-based cognitive behavioral therapy78), TRAC goes beyond existing programs by directly relating ER to 
sexual health.  
While we recognize that digitizing an efficacious in-person intervention is not innovative in itself, creating a 
marketable tool that can be used by schools and/or other agencies and does not require staff training to admin-
ister does represent an innovation to the field and the commercial landscape. Although there are a number of 
programs designed to scaffold Social Emotional Learning (SEL) within school settings, the vast majority are 
resource intensive, require trained facilitators, and do not integrate sexual health education. A search of 61 of 
the most popular SEL programs (most of which are focused on SEL constructs that emphasize social skills for 
positive school climate) indicates that about half (31) distribute materials relevant to early adolescents. Of 

 
Version 1.2  9/27/22                                                                              5 
 
those, only 12 are primarily web-based, and only one (Get Real) focuses on sexual risk behaviors79. Though 
there are significant distinctions that set iTRAC-Sexual Risk Prevention (SRP) apart from this program, most 
notable is iTRAC’s focus on teaching ER skills and practicing those skills within the program. Get Real de-
scribes itself as comprehensive sexual education that includes elements of SEL such as “relational skill-build-
ing,” but does not teach ER skills. iTRAC has innovated the teaching of ER skills through technology by includ-
ing activities that provide opportunities to regulate emotions through experiences that evoke feelings (e.g., 
watching a sad video clip, looking at “gross” pictures, playing stressful video games while receiving negative 
feedback) while coaching participants to use the strategies they have been taught. In this way, teens develop 
self-efficacy for emotion regulation, which is key to its use in daily life and is reflected in the moderate effect 
sizes observed in adolescents’ self-efficacy for emotion regulation (SEQ; d=.63), beliefs that they are able to 
change their emotions (ITES; d=.72), and their use of the strategies taught (ERBS-R; d=.45). iTRAC-SRP will 
further enhance these opportunities by connecting ER skills to sexual situations.  
For example, standard sexual health education includes learning about reproductive anatomy of both 
sexes with visual aids for understanding. This often elicits feelings of discomfort that can interfere with engage-
ment with programs and, thus, knowledge acquisition. Connecting the ER strategies learned in the program to 
this content may enhance understanding and improve application of this information to later sexual health con-
tent. Other examples of connecting ER to sexual health education include recognizing and regulating emotions 
evoked when learning how to use a condom or managing emotions during conversations with partners in which 
one is negotiating sex or condom use. Explicitly making these connections for young adolescents, before pat-
terns of behavior around sex are entrenched, can improve the likelihood that they will apply the ER skills they 
are learning to sexual situations. Sexual activity is an emotional experience for people, especially in the devel-
opmental period of adolescence, and the absence of emotion regulation skill building in sexual health educa-
tion represents a key opportunity for innovation.  
 
COMMERCIAL POTENTIAL 
iTRAC-SRP has strong commercial potential, having few direct competitors in the diverse landscape of SEL 
and ER curricula for middle school students (see Commercialization Plan). The vast majority of existing pro-
grams are resource-intensive, require trained facilitators, and do not integrate sexual health education. Fore-
sight Science & Technology, Inc. will serve as a Technical and Business Assistance vendor for this Fast-Track 
application. Additionally, we secured a letter of interest from Cranston Public School District to aid in imple-
mentation of research activities as well as evaluate the program for future adoption (see Letters of Support). 
 
RESEARCH PLAN 
We propose a Fast-Track STTR research project. We have already developed the efficacious iTRAC interven-
tion to enhance emotion regulation competencies (based on the original efficacious group intervention). This 
was well-received and resulted in high uptake, even without the sexual health content (which is often of high 
interest to middle schoolers). Thus, we believe the potential feasibility of iTRAC-SRP (with its sexual health 
content) is high. We propose to achieve Phase 1 milestones of conducting focus groups with advisory boards, 
updating the TRAC sexual health content using current sexual health education standards, creating prototypes 
of the iTRAC-SRP intervention and receiving approval from the advisory boards and creating a specifications 
document that will guide Phase 2 production. Milestones in Phase 2 include full production of iTRAC-SRP fol-
lowed by acceptability testing with 16 students and a randomized trial with 120 more. 
Research Team: The investigative team brings a wealth of complementary experience to their roles (see 
Biosketches). Drs. Houck and Hadley developed TRAC ER and collaborated with Klein Buendel on the devel-
opment and pilot testing of iTRAC, providing experience with the procedures of the proposed application. Dr. 
Houck has successfully recruited and retained participants for several middle school-based projects, including 
the pilot. Each investigator brings necessary expertise. Dr. Houck will oversee the adaptation and the clinical 
trial with expertise in ER interventions and early adolescence. Dr. Hadley will assist in the translation of the re-
maining content as well as assist in the oversight of the clinical trial design and execution. Dr. Barker will direct 
the statistical analyses. Ms. Berteletti will contribute expertise in the technology transfer of interventions. She 
will oversee the KB development team during the Translation phase as well as during the management of the 
intervention during the RCT. Dr. Houck and Ms. Berteletti have collaborated previously on an intervention to 
prevent adolescent dating violence (R01 HD097126) that integrated emotion regulation concepts into health 
behavior content. Thus, we are uniquely positioned to successfully achieve the goals of this program. 
Overview of Fast-Track Research: To fill a need in the area of sexual education, which fails to integrate 
emotion regulation skills training, the goal of this study is to complete adaptation of TRAC to a web app format 
and assess its efficacy for improving self-efficacy for sexual risk prevention behaviors (e.g., sexual refusal, 
condom use). Consistent with the process of the pilot study (R21 HD089799), this will be accomplished 
through three stages. In the Translation Stage (Phase 1), the remaining core constructs of the original TRAC 

 
Version 1.2  9/27/22                                                                              6 
 
sessions will be adapted and iteratively reviewed by community advisory panels (2 adolescent advisory 
boards, 1 expert panel of health professionals). In the Acceptability Testing Stage (Phase 2), participants 
(n=16) will provide feedback on acceptability and usability for the completed iTRAC for Sexual Risk Prevention 
(iTRAC-SRP), followed by revisions. Finally, the RCT Stage (Phase 2) will individually randomize 120 partici-
pants to iTRAC-SRP or a waitlist control group. Teens will complete 8 tablet-based modules after school in 
small, monitored groups. Adolescent assessments (baseline, post-intervention, 6-month follow-up) will include 
computer-administered self-interviews (ACASI) and computerized performance tasks. Parents will also com-
plete ratings of adolescent ER. The effect size of iTRAC-SRP relative to the waitlist control group will be exam-
ined, and results will be published. See Timeline. 
Settings and Participants (for both Phases 1 and 2): Subjects will be recruited from four public schools in 
Providence County, which is rated a Level 1 Urban Influence Code by the U.S.D.A. Health and Education Pro-
fessionals Panel (HEPP) members in Phase 1 will also be recruited through a Rhode Island Department of 
Health (RIDOH) school nurses listserv and the Planned Parenthood Federation of America. Eligibility criteria 
will include 1) attending the 7th grade and 2) being 12-14 years of age. Adolescents will be excluded if they are 
unable to read at a 4th grade level, have a sibling who has participated in the program, or have developmental 
delays. The proposed schools have previously participated in our studies and are familiar with our procedures 
(see Letters). Racial and ethnic minorities represent 50% of students, and 45% are from socioeconomically 
disadvantaged homes. Thus, these data will generalize to large segments of the population. Together, these 
schools serve over 750 7th graders each year. The RCT will enroll 120 participants (and their parents), requir-
ing a recruitment rate of 16%. This is feasible; our rate of recruitment in Project TRAC was 40% from a sample 
that presented additional challenges (i.e., teens with mental health symptoms), and the pilot study recruited 86 
students in 4 months from fewer schools (see Recruitment). 
 
Overview of Phase 1: Translation Stage (months 1-12): This phase will complete the TRAC adaptation to a 
web app. The pilot study (R21 HD089979) effectively translated the ER content of TRAC (see Table 1) and the 
same procedures will be used to translate the remaining content. In month 1, protocols will be reviewed and 
approved by the Institutional Review Boards. In months 1-4, content will be reviewed and advisory boards will 
participate in focus groups to guide development of iTRAC-SRP. Wireframe prototypes will be developed in 
months 5-8. Advisory boards will review prototypes and provide approval of feasibility and acceptability in 
months 9-12. Finally, a specifications document will be created in month 12 to guide full development in Phase 
2, if approved by the advisory boards. 
Phase 1a- Sexual Risk Prevention Content Development Updates: The TRAC content to be translated will 
include sexual health information such as reproductive anatomy, pubertal development, STIs, sexual identity, 
pregnancy, abstinence, condom use and contraception, accessing healthcare, and sexual behaviors that con-
fer risk of STIs. TRAC content will be updated with consideration of the National Sex Education Standards 
(NSES) published by the Focus on Sex Education Initiative80, which emphasize functional skills through a lens 
that recognizes the influences of inclusivity, intersectionality, trauma, and social justice. Other sources will be 
consulted (e.g., Gender, Sexuality & Inclusive Sex Education Tip Sheet81, Vermont Agency of Education Sex-
ual Health Education Resource Guide)82 as well as input from the HEPP of middle school health professionals. 
This information will guide decisions related to adapting sexual health content for iTRAC-SRP and will ensure 
consistency with professional recommendations for inclusive sexuality education. Possible technology transla-
tions include a digital anatomy game in which participants are asked to label body parts and match their biolog-
ical functions and a game in which teens correctly order videos of steps for condom use. Similarly, content 
connecting emotions and sexual behavior from the TRAC intervention remains to be translated. Examples in-
clude the influence of sexuality in the media on feelings, attitudes, and behavior; emotions’ influences on deci-
sions to have sex and/or use condoms; and the influence of perceived peer norms on emotions and their influ-
ence on risk behaviors. Other content includes connecting these concepts with the ER strategies taught in the 
intervention. For example, an activity might include coaching participants to use previously learned ER strate-
gies while watching a video in which a condom is correctly placed on a plastic penis model, which can be 
dysregulating to some and interfere with learning in many settings. As in the R21 iTRAC pilot, games, puzzles, 
quizzes, and video scenarios used in the original TRAC intervention to convey these concepts will serve as the 
starting point for translation ideas with the advisory groups for the current study. Per NSES recommendations, 
inclusive language that recognizes the spectrum of gender and sexuality will be used throughout. Further, we 
will not be including content that may be upsetting for individuals with a history of trauma (e.g., descriptions of 
sexual violence), within this digital program (See also Trauma-informed Approach in Human Subjects).  
Deliverable 1: A summary report of the recommended content based on review of sexual education stand-
ards, integrated with recommendations from advisory groups, will be developed. 
Phase 1b- Formative Research through Advisory Focus Groups: The iTRAC program created during the 
pilot R21 study was developed using principles consistent with Designing for Dissemination (D4D), which 

 
Version 1.2  9/27/22                                                                              7 
 
promotes the design of interventions with consideration of the elements most important to external validity.83 
Stakeholders from the target population (early adolescents) were included as key advisors in the adaptation of 
the program and testing of delivery approaches (after school). This proposal will continue this approach and 
form a Health and Education Professionals Panel (HEPP) of middle school health professionals to complement 
two Adolescent Advisory Boards (AABs; 1 male, 1 female, n=16) during translation. Research summaries will 
also be generated to communicate with participating communities. Using these recommended D4D processes, 
the research will be better positioned for dissemination (and commercialization).  
Content translation will be guided by the AABs, who will meet 2 times to assist with brainstorming transla-
tion of content and 2 times to review the wireframe prototypes generated from that process. AABs will contrib-
ute ideas for translation, provide feedback about appeal, and attend to cultural, developmental, and gender 
considerations. The HEPP will provide feedback on the proposed content and ensure contextual acceptability 
(i.e., schools). Each time an AAB or HEPP meeting is scheduled, participants will be notified in advance of the 
time, data, and location to minimize conflicts. At least 50% of participants will need to indicate their ability to 
attend each meeting. Otherwise, meetings will be rescheduled for a more convenient time. If municipal, state, 
or federal guidelines prohibit face to face communication, or if it is not logistically feasible to get the majority of 
participants together in the same physical space, AAB and HEPP meetings will be conducted virtually using 
secure video-conferencing technology. Research staff will provide portable wireless internet and a video-com-
pliant device to all participants who require them in order to fully participate. As in the R21, the design develop-
ment cycles will begin with advisory focus group feedback about how the existing activities convey the content 
(i.e., what and how they are learning). They will advise on activity design for the remaining content (sexual 
health and content connecting this with ER) to ensure these are engaging, appropriate, and informative. They 
will also provide perspectives about design, structure, navigability, and approach. Established protocols, suc-
cessful in previous projects, will be used and led by Dr. Houck. Groups will be recorded, transcribed, and ana-
lyzed for content and themes, as in the pilot.  
Deliverable 2: A report of the focus group data summarizing and integrating recommendations for the de-
velopment of remaining modules will be created. 
Phase 1c- Prototype Development: Theoretical 
frameworks. The technology design process will 
use a multi-theoretical framework to guide pro-
duction. Persuasive System Design (PSD) is a 
well-established guide for translating clinical aims 
to technology frameworks for supporting health-
related behavior change.84-86 Along with PSD, 
User-Centered Design (UCD)87, 88 and the ADDIE 
Model,89-91 a user interface and instructional de-
sign approach, will be used. Design elements 
such as space (colors, sounds, visual space), 
components (characters, objects), and mechan-
ics (actions) will be determined for program fea-
tures (i.e., activities).  
Iterative design process. The conceptual and technological web architecture will be developed using PSD. 
The AABs, HEPP, and investigative team will identify key elements to be translated. Using UCD, AAB input will 
be solicited through iterative cycles, with adjustments based on feedback,92 resulting in greater user experi-
ences and more effective results.93, 94 Game design theory and the MDA framework will guide intervention 
“gamification.” Consistent with mHealth research with teens,95-99 adaptation will emphasize gaming as a teach-
ing strategy and interactive components will be the centerpiece of the intervention, complemented by role play 
simulations and hosted videos. To reward learning and bolster adherence, “badges” (visual reward icons), will 
be awarded intermittently. Badges are a highly effective tool used to encourage user engagement and build 
game “loyalty.”100-103 Quizzes for the modules will also provide corrective normative feedback. TRAC content 
will be integrated into 8 (approx. 45-minute) modules in Phase 2. Though full programming will not be com-
pleted in Phase 1, the formative research and iterative design process will guide plans for future development. 
Multimedia programming will follow standard production steps. Investigators will develop instructional and 
behavioral objectives based on focus group results, the previously validated intervention, and existing web 
platforms. Interface design ideas will be created in writing, combined with scripts, flowcharts, mock-ups, and 
storyboards using Adobe XD wireframing tools in Phase 1. Acceptability of the prototypes will be determined 
with a second round of advisory board meetings to review the wireframes generated from the ideas provided 
during the first round. A specifications document, with a full outline of modules and activities, will be created to 
guide full development in Phase 2.  
Deliverables: Deliverable 3a: Prototype wireframes for use in a second round of advisory board meetings 

 
Version 1.2  9/27/22                                                                              8 
 
will be created. Deliverable 3b: A summary report of advisory board feedback will be generated to guide final 
scripting. Deliverable 4: Specification documents for the digital modules to complete the iTRAC intervention for 
sexual risk prevention (iTRAC-SRP) will be created. 
 
Summary of Phase I Milestones: 
1. Milestone 1: Review of Sexual Education Standards. TRAC sexual health content will be updated using 
the recommendations of the health and education professional advisory board and by a review of the National 
Sex Education Standards80. Deliverable 1: A summary report of the recommended content based on review of 
sexual education standards, integrated with recommendations from advisory groups, will be developed. 
2. Milestone 2: Focus Groups. Each of three advisory boards (female adolescents, male adolescents, health 
and education professionals) will meet to provide recommendations for the adaptation of the intervention, in-
cluding key sexual health content and content linking sexual health to ER as well as approaches to translating 
this material via technology. Deliverable 2: A report of the focus group data summarizing and integrating these 
recommendations for the development of remaining modules will be created. 
3. Milestone 3: Prototype wireframes. Static screenshots will be created to show the look and feel of the new 
content (aiming to match that of the existing content), demonstrate storyboards of the proposed games and 
activities, and illustrate the user interface via wireframes. These will be used in a second round of advisory 
board meetings to get feedback. Deliverable 3: a) Prototypes of intervention modules will be created. b) A 
summary report of advisory board feedback will be generated to guide final scripting.  
4. Milestone 4: Specifications Document. Scripts of the remaining iTRAC module content will be created, in-
tegrating the existing emotion regulation content with the identified sexual health skills and education content. 
Deliverable 4: Specification documents for the digital modules to complete iTRAC-SRP will be created. 
Completing the Phase 1 Specific Aims and these milestones will provide measurable outcomes reflecting a 
successful trajectory of the project indicating a high probability of accomplishing the Phase 2 goals. 
 
Overview of Phase 2: Acceptability Testing and Randomized Trial (months 13-36): This phase will create 
iTRAC-SRP as a web app from the specifications document created in Phase 1, thus completing the transla-
tion of the original group program to a mobile intervention that can be more easily disseminated and fill a con-
tent gap (emotion regulation) in the area of sexual education. Once completed, acceptability testing will be 
completed with early adolescents in two rounds to allow for modifications based on participant feedback. Once 
finalized, a small RCT will assess impact on adolescents’ self-efficacy for preventing sexual risk as well as en-
gagement in sexual behaviors. All of these activities will follow procedures successfully used in prior collabora-
tions between the research institution (RIH) and technology partner (KB). 
Phase 2a- Full Program Development (months 13-18): will take place at the beginning of Phase 2, using 
content outlined in Phase 1 and approved by the advisory boards. For this STTR Fast-Track study, the sexual 
health and connecting ER to risk decisions content will be completed using the same successful procedures of 
the pilot study (including D4D principles).83 The iTRAC-SRP program will be produced for mobile platforms, 
including smartphones, tablets, and desktop computers. It will be alpha tested in-house for stability and code 
errors, beta tested (see below), and revised following KB’s iterative design process. Federal Section 508 
standards for accessible information technology will be 
considered so program functionality can be compliant in 
future dissemination trials and commercialization. Web 
app production will use a Responsive Web Design ap-
proach that provides an optimized user experience for 
tablets. The web app front-end interface components will 
be developed using HTML5, CSS and JavaScript. A Mi-
crosoft SQL database will enable user identification and 
store and deliver tailored content. Server-side compo-
nents will be developed on the Microsoft dotNet platform 
using the C# programming language. Hosting will occur 
on KB’s state-of-the-art server farm (see Resources). 
Click stream data will be tracked and monitored on the 
backend. Sensitive information is protected by a hardware firewall (Cisco ASA 5505) and daily backups protect 
against data loss. The program will be developed for tablets for the current project but optimized for use on 
desktops and smartphones as well, which will enhance commercialization opportunities.  
The iTRAC intervention aims to enhance ER skills to reduce poor decision making that can lead to risk be-
haviors. Incorporating the ER content developed in the pilot, we plan to create 45-minute digital modules, time 
consistent with computerized executive function training programs used with adolescents (e.g., Cogmed, 
Braingame Brian) and typical class lengths. This length was acceptable in the pilot trial. Consistent with re-
search on mHealth strategies for adolescents,95-99 the design emphasizes gaming as a teaching strategy. 

 
Version 1.2  9/27/22                                                                              9 
 
These interactive components (4-6 per module) are the centerpiece of the intervention, complemented by 
hosted videos connecting the games. As in the original TRAC group program, each module will include activi-
ties related to emotions, sexual health, and the link between the two (the iTRAC pilot modules contained emo-
tion content only). As with the pilot, the complete iTRAC-SRP intervention will begin each module with teens 
labeling their feelings that day (to practice emotion recognition and labeling skills) before moving on to a brief 
review of previous module content. The first four modules will focus on illustrating the relationship between 
emotions and behaviors and providing education about emotions, such as strategies for labeling feelings in 
others, identifying emotional arousal in oneself through somatic cues, labeling these feelings, and recognizing 
their source (“triggers”). The last four modules of the program will focus on teaching developmentally appropri-
ate strategies for regulating emotions during difficult situations, particularly those related to health risks, using 
three primary groups of strategies identified during qualitative work in the development stage of Project TRAC 
(R34 MH078750)45: 1) getting away (physically or cognitively) from triggers for strong emotions, 2) releasing 
emotional energy in healthy ways (verbally or physically), or 3) changing cognitions and appraisals about emo-
tional triggers. These strategies, labeled “Get Out,” “Let It Out,” and “Think It Out,” correspond to four of the 
“families” of ER processes in Gross’s process model.27 Activities also emphasize practice with the unique focus 
of the intervention, managing emotions as they occur, as cued by frustrating games or emotionally arousing 
visuals. Much of this content exists through the pilot study (see Table 1), including games navigating a space-
ship through emotional triggers, a virtual slingshot target game highlighting places in the body where somatic 
emotional cues are experienced, and activities exposing adolescents to emotional cues with coaching to regu-
late these emotions. Emphasis is also placed on activities that encourage participants to personalize this infor-
mation. This existing ER training will be integrated with the rest of the program content (sexual health educa-
tion and activities linking ER to health behaviors) to be developed with guidance from the AABs and HEPP. 
This content will include strategies for (and practice with) recognizing and managing emotions in sexual health 
situations, such as those involving peer pressure, condom use, media/pornography exposure, or sexual deci-
sion making, to enhance the likelihood that the ER and sexual health education provided can be applied to ex-
periences that are emotionally arousing and lead to risk. 
Deliverable: A complete digital version of the adapted TRAC intervention for sexual risk prevention, 
iTRAC-SRP, will be created. 
Phase 2b- Acceptability Testing and Finalizing Revisions (months 15-18): A pilot test of the completed 
iTRAC-SRP will be conducted to confirm that the components perform as designed. Assessment and interven-
tion procedures to be used in the RCT will also be tested. Sixteen 7th grade students (50% female, racially and 
ethnically diverse) will be recruited from the study schools in two cohorts to test iTRAC-SRP and make iterative 
revisions. These participants will be excluded from the RCT.  
Acceptability will be assessed several ways for each module. First, teens will complete a brief standardized 
questionnaire (used in the pilot project) to assess acceptability (e.g., How much did you like the program?) and 
feasibility (e.g., How easy was it to use?). Mean scores of 3.5 or greater (on a scale of 1-poor to 5-excellent), 
will be deemed acceptable (the existing iTRAC received scores over 4). Second, adolescents will complete the 
System Usability Scale (SUS)104. This validated, 10-item scale provides a global view of subjective usability for 
programs such as web tools. SUS scores above 68 will be considered acceptable. Third, researchers will ob-
serve participants playing the program using screen sharing software (e.g., GoToMeeting) to assess for confu-
sion with functionality. In the pilot, this proved informative for identifying usability concerns. Finally, a brief inter-
view with teens after each module will collect impressions of the primary messages conveyed in each of the 
developed activities. Interviews will be reviewed for consistency of themes and content comprehension by Drs. 
Houck and Hadley; both have experience in qualitative methods.46 Each activity will have an identified primary 
message to which participant responses will be compared for understanding, as in the pilot. This interview will 
also collect feedback regarding suggested improvements. Recruitment will occur in two groups of eight, to al-
low for iterative changes in response to participant feedback. The final revisions will incorporate all feedback 
and resolve remaining technical issues prior to the RCT. This process was successful in the pilot study.  
Deliverable: A summary report of quantitative and qualitative data from the Acceptability testing question-
naires and interviews will accompany a table of recommended changes that will guide Klein Buendel’s revi-
sions to the program, leading to the finalized iTRAC-SRP program. 
Phase 2c- Randomized Control Trial (months 19-36): Recruitment: For the RCT, project staff will recruit 120 
7th grade students from two waves of enrollment over an eight-month period that will span across two school 
years, providing two unique cohorts of 7th graders from which to recruit. We will examine sample demographics 
after the first cohort to ensure diversity in terms of race, ethnicity, and SES. Oversampling will be conducted if 
sampling is not representative of diversity at the schools, though our previous projects have not experienced 
this. Participants will be recruited through several methods consistent with those used to successfully recruit in 
the pilot trial and in our other school-based studies (see Form F- Recruitment and Retention). 1) Students 
will be introduced to the program at school via small-group presentations emphasizing the content (sexual and 

 
Version 1.2  9/27/22                                                                              10 
 
emotional health) and approach (gaming), both of which are of interest to this age group. The introductions will 
be conducted by project staff, who will distribute flyers with QR codes that connect to a REDCap link to the 
“consent to contact” form, to avoid students having to remember to return the form. Parents can also sign and 
return the forms to the school. 2) Principals will introduce the project to parents via school announcement 
emails that contain the REDCap link to the consent to contact form. 3) Project staff will recruit at school events 
attended by parents (open houses, sporting events, etc.).  
Once permission to contact the family is received, trained staff arrange a meeting to describe the project, 
obtain parental consent, and complete baseline parent measures. Separate meetings for adolescents are ar-
ranged to explain the project and obtain assent (separate from their parent). The Translation and Acceptability 
stages will also use the classroom recruitment procedures described above. Parents and teens will be com-
pensated for time and effort in all stages of the project.  
Randomization: Participants will be individually randomized to either iTRAC-SRP or waitlist control after 
baseline, following consent. Per recommendations for achieving between-group comparability in clinical tri-
als,105 randomization will be stratified by school and gender and blocked using predetermined size-4 blocks. 
Individual randomization avoids nesting of conditions within schools by each school hosting both conditions, 
which is feasible given individual program completion. We considered randomization by school, but a group 
randomized design would require >30 schools to achieve adequate power. Minor concern exists that teens 
from different conditions may discuss the intervention; as in Project TRAC, we will measure this “cross-talk” via 
assessment. Project staff will log participants into the password-protected site to avoid outside access. Acces-
sibility will be an ultimate goal of iTRAC, but use will be monitored during the study to control for dose. 
Intervention Condition: iTRAC-SRP will consist of eight, approximately 45-minute, “gamified” digital mod-
ules of 4-6 activities (games, videos, etc.). Teens will complete modules in after-school supervised groups. 
Students will be allowed to complete 1-2 modules each week (total intervention completion time 4-8 weeks); 
project staff will be present at each school at least three days per week, allowing flexible scheduling/attend-
ance. No instruction is needed to use the program. Teens will come to a common area (e.g., classroom, li-
brary) where they will be provided a project tablet connected to Wi-Fi. They will have an individual account for 
the program (password controlled by project staff to prevent outside use), by which they will resume where 
they left off at the previous module (thus allowing flexible administration and breaks, as needed). We have cho-
sen this strategy for the research for pragmatic reasons; after-school formats are common in schools, and this 
time, location, and format were successful in TRAC and iTRAC. However, we recognize that the program may 
be administered differently (e.g., during class time, in waiting rooms, or at home) once it is commercialized. 
Content will use gender- and sexuality-inclusive language and avoids heteronormative descriptions of risk. 
All modules will include activities that personalize main concepts (some were developed in the pilot). iTRAC’s 
programming structure does not permit the skipping of content (unless configured to allow it in predetermined 
instances). Modules will conclude with a quiz (with corrective feedback) that must be passed before moving on.  
Waitlist Control Condition: Control participants will be assessed on the same schedule as the treatment 
condition and offered the intervention after 6-month follow-up. This is an appropriate strategy given that ado-
lescents will still be in the developmental window of early adolescence at the conclusion of the follow-up win-
dow, thus the intervention will still be age appropriate. 
Fidelity: To preserve internal validity, assurances for consistent delivery will be taken. 1) Research staff 
complete comprehensive training to assure proper administration, and drift will be monitored through regular 
team meetings. 2) The computerized nature of iTRAC-SRP will ensure standardized delivery of content. 3) A 
trained RA will be available to answer questions and problem solve at all appointments. 4) Each module will 
conclude with a quiz of key content, with corrective information that re-presents concepts relevant to incorrect 
responses. Participants will not be able to advance to the next module until successfully answering all quiz 
items. Results will also be saved as data to inform regarding program comprehension. 5) The iTRAC program 
prevents advancing through video content, to ensure that teens do not skip challenging material. 6) Tracking 
software will track time spent on each module and activity to assess dose. Summaries of these data (means, 
ranges) will be reviewed to understand usage patterns that may also inform commercialization. 
Intervention Retention: Retaining participants and ensuring a maximum intervention dose is critical to as-
sessing efficacy. 1) The gaming approach to the interventions will encourage participants to return weekly. Pi-
lot teens were easily engaged; 88% completed all modules. 2) The health content, especially that related to 
sex, is of interest to this age group. 3) In-program reinforcers (e.g., “badges,” virtual prizes) mark progress and 
encourage completion, especially as they relate to knowledge valued in this developmental period (e.g., sexual 
anatomy, STI knowledge). Badges were noted by participants during the pilot project. 4)  As in the pilot, the 
sessions will be held at a convenient location and time (after school), with options to accommodate special cir-
cumstances (e.g., library meetings during school breaks). 5) Snacks will be provided. 6) Finally, providing mul-
tiple chances to complete modules on a flexible schedule is an advantage over traditional group formats, which 
are restricted to a specific time and can lead to missed content when participants miss group. Mobile delivery 

 
Version 1.2  9/27/22                                                                              11 
 
allows teens to receive 100% of the content at their own pace, which may enhance intervention effects. 
Follow-up Retention: The investigative team has previously demonstrated strong retention capacity (e.g., 
Project TRAC: 86% of adolescents, 77% of parents through 30-months; iTRAC pilot: 94% through 3-months). 
We will facilitate retention over the proposed two-year follow-up using successful methods from our own work 
and the literature.106-111 1) The value of their participation will be conveyed (teen and parent compensation; ed-
ucation during consent regarding the importance of follow-ups; problem solving barriers). 2) Rapport will be 
established (contacts conducted by the same staff over time; follow-ups conducted at times/locations conven-
ient to families; holiday cards). 3) Participant relocation will be monitored (assessment of potential moves at all 
contacts; families complete locator forms). 4) Logistic barriers will be minimized (parent assessments kept 
brief; scheduling to include evenings/weekends). See Form F- Recruitment and Retention for details.  
 
Assessment Procedures: Assessments for both conditions will occur at baseline, post-intervention (2 
months) and 6 months after baseline. They will minimize participant burden while assessing key constructs of 
the intervention. Assessments will be administered in a quiet location at school; we expect the battery of self-
report and performance measures will take 60-90 minutes. Questionnaires will be administered via tablet, us-
ing REDCap to reduce errors and missing values. We will use calendar-aided recall cues and anticipate that 
recall will be excellent given the salience of risk events for early adolescents. The proposed battery is shorter 
than that of Project TRAC, which was well-tolerated. Performance measures of emotional competence will be 
collected via laptop computer. The brief parent battery will be collected via REDCap and can be administered 
in person, via email link, or, if needed, by phone. 
Measures 
Primary Outcome: It is well-established that sexual self-efficacy (SSE) is correlated with positive sexual 
behavior outcomes and sexual well-being among a diverse array of populations112-115 . As such, measuring 
self-efficacy for safer sexual activity as a main outcome will indicate iTRAC-SRP’s efficacy and ability to pro-
mote healthier sexual behaviors with the full sample (not just those who are sexually active). Adolescents’ per-
ceived abilities to engage in behaviors that mitigate sexual risk will be assessed by the Self-Efficacy for HIV 
Prevention scale (12 items)116. This continuously scaled measure includes items such as “If you decide to have 
sexual intercourse with your girlfriend/boyfriend, how sure are you that you could talk to your girlfriend/boy-
friend about safer sex?” The measure assesses a range of behaviors related to prevention of HIV, other STIs, 
or unintended pregnancy, including refusing sexual behaviors, discussing sexual histories with partners, buying 
condoms, taking free condoms, carrying condoms, and asking a partner to use a condom. Participants respond 
on a 4-point scale (couldn’t do it, unsure, sure, very sure). This measure demonstrated good reliability in the 
TRAC study with this age group (α= .90). 
Secondary Outcomes: Sexual Risk Cognitions: The intervention is expected to have an impact on other 
cognitive constructs related to sexuality (e.g., knowledge, attitudes), as well as behaviors. The Sexuality Ques-
tionnaire for Adolescents (34 items) is a multiple choice test to assess sexual health knowledge and has been 
shown to be sensitive to intervention impact.12 The Abstinence Attitudes (10 items117) questionnaire assesses 
agreement with values related to abstinence (α= .86).  
Sexual Behaviors: The Adolescent Risk Behavior Assessment (ARBA)118 is a computer-assisted structured 
interview for self-reported sexual and drug behaviors, successfully used in Project TRAC. The ARBA employs 
a skip structure so that questions answered in the negative are not followed by more detailed questions. Ques-
tions are asked in behavioral terms, without reference to sex of partner. Except at 2-month follow-up (which will 
use an abbreviated version), the ARBA will assess sexual behavior occurrence and frequency in the past 6 
months. Given the relative infrequency of these behaviors and their significance to adolescents, we anticipate 
that recall will be excellent for this time frame. For early adolescents, we have modified the ARBA to include 
behaviors relevant to this developmental period (e.g., genital fondling) using previously validated items from 
the Psychosexual Development Interview (PDI)119. Other questions cover condom use and number of partners. 
It is expected that, as in TRAC, the ER skills taught in iTRAC-SRP will result in less sexual behavior and less 
sexual risk when compared to the waitlist control group over the same time period. Occurrence of vaginal, 
anal, and oral sex, as well as condomless sex, multiple partners, and substance use before sex, assessed via 
ARBA, will be examined. We recognize that the study will be underpowered to detect significant group differ-
ences on these variables, given that most early adolescents will not transition to sex in this 6-month time frame 
of 7th grade and that larger studies with longer follow-ups will be needed to demonstrate effects on these varia-
bles. Nonetheless, gathering preliminary information will be useful for understanding the impact of iTRAC-SRP 
and planning for future studies.   
Non-sexual risk behaviors may also be impacted by a generalized effect of ER on health behavior. Thus, 
we will conduct brief assessment of behaviors that may be more common in this developmental window.  
Youth Risk Behavior Surveillance System1 items (n=9) will be used to assess tobacco/vape use, violence (e.g., 
fighting), and substance use behaviors. Further, acceptability of the program will be assessed at immediate 
posttest using the acceptability questionnaire used in stage 2b, to provide additional information for commercial 

 
Version 1.2  9/27/22                                                                              12 
 
viability and marketing direction. 
Treatment Mechanisms (Mediators): Emotion regulation will be measured via self-report, performance 
measures, and parent observation. Teacher measures were considered; our experience in Project TRAC was 
that students changed teachers too frequently for long-term follow-ups and that short-term follow-ups (2 
months) did not provide time for the ER skills learned for risk situations to generalize to non-risk settings, like 
the classroom48. Self-report: The Difficulties in Emotion Regulation Scale (36 items120) uses six subscales 
(e.g., lack of emotional awareness, limited access to ER strategies; all α > .80) to assess perceptions of skill in 
ER based on Linehan’s theoretical work.121 The Affect Dysregulation Scale (6 items122) assesses the frequency 
of difficulties with ER (α= .72), and is shown to be related to adolescent risk behaviors.122 The Emotion Regula-
tion Behaviors Scale- Revised (9 items123) measures use of the specific emotion regulation strategies taught in 
iTRAC (α= .73). The emotional self-efficacy subscale of the Self-Efficacy Questionnaire for Children (8 
items124) assesses perception of one’s ability to cope with negative emotions (α= .83). It has been shown to be 
valid, reliable, and related to risk behaviors among adolescents.124 The Implicit Theories of Emotions for Chil-
dren- Self subscale (6 items)125 is based on Dweck and colleagues’ work on implicit theories of intelligence126 
and Tamir’s adult measure of emotion malleability._ENREF_146127 It assesses adolescents’ beliefs about the 
controllability of their emotions (α=.86). All of these measures were successfully used in the iTRAC study. Per-
formance measures: a) Diagnostic Analysis of Nonverbal Accuracy–250 (DANVA2) is a computer-based 
measure that asks participants to identify the emotion of facial expressions displayed in photographs. The 
measure has shown one-month test-retest reliability over .850 and had an alpha of .83 in Project TRAC. b) Be-
havioral Indicator of Resiliency to Distress (BIRD)128 is a 5-minute computerized distress tolerance task in 
which participants are given the option to quit at any time. It generates a score of time spent persisting on a 
frustrating task that provides negative feedback (aversive noise) when users fail at the task.128-130 Parent re-
port: The Adolescent Self-Regulatory Inventory (29 items)131 measures perceptions of adolescents’ abilities to 
regulate over the short-term and long-term, separately; both adolescents and parents will complete it about the 
adolescent (α= .75-.90). Parents will complete this measure at baseline and 6-months, to reduce burden.  
Treatment Moderators: Demographic information will be collected, allowing for analysis of demographic 
factors, including sex as a biological variable, as moderating influences of treatment effects. Self-Rating Scale 
for Pubertal Development (PDS; 5 items)132 will be used, as pubertal status and sexual activity are related. The 
PDS uses gender-specific items to assess physical maturation. It is reliable (α= .67) and strongly correlated 
with pediatrician-rated physical development.132 The Columbia Impairment Scale (CIS; 13 items)133 provides a 
global measure of functional impairment with good reliability and validity.131, 133 A score of > 15 suggests clinically 
significant impairment. Intervention Cross Talk will be assessed with 3 items from our previous studies that ask 
whether participants have talked with others about the intervention, how often, and about what content. 
Other constructs shown to influence early adolescent sexual behavior will be measured to assess modera-
tors of intervention impact.29, 134-140 Perceived Peer Approval (3 items)141 measures perceptions of peer ap-
proval of sexual behaviors (α= .81). Normative Beliefs (5 items)142 assesses perceptions of peers’ experience 
with and attitudes toward sexual behaviors (α= .76). The Neighborhood Environment Scale (6 items)143 as-
sesses conditions of an adolescent’s neighborhood (α= .83).  
Data Analysis: A more detailed description of the analytic plan and justification for power calculations is pre-
sented in Section 4.4 Statistical Design and Power, including approaches to evaluate model assumptions, po-
tential imbalance between treatment groups in baseline characteristics, and missing data. Analysis of covari-
ance will be used to evaluate changes in self-efficacy for sexual risk prevention skills over 6-months post inter-
vention with baseline as a covariate. All participants who were randomized will be included in all analyses (i.e., 
intent-to-treat). The size of the treatment effect will be estimated using standardized mean differences. The 
same approach will be used to evaluate sexual health knowledge and emotional competencies. Sexual behav-
iors (number of sexual behaviors, number of condomless sex acts) will each be aggregated across the two- 
and six-month follow-up assessments and analyzed using generalized linear models with log link functions and 
negative-binomial distributions for behavioral counts that are expected to follow a zero-altered count distribu-
tion. Exploratory analyses: Mediation: Causal mediation models will be used to test how much of the treatment 
effect is accounted for by measures of emotional competencies and sexual attitudes. Causal mediation pro-
vides a general framework for evaluating mediation in both linear and nonlinear models, including count out-
comes1. Moderation: To better understand who is benefiting most from the intervention we will use causal ran-
dom forests, a non-parametric data-mining procedure2. Although this principled exploratory approach cannot 
definitively define subgroups, it has better statistical properties than looking at individual moderators and will 
help suggest subgroups that differ in terms of their response to the intervention.  
Power and Sample Size: This trial was powered to the primary and other continuously scaled outcomes (self-
efficacy, knowledge, and emotional competencies). With 120 participants, this study is powered at .80 to detect 
moderate effect sizes (standardized mean difference d ≥ .47). The power analysis was run using the R pack-
age simstudy v0.2.1 and assumed a two tailed type-1 error rate of .05 and a correlation between baseline and 

 
Version 1.2  9/27/22                                                                              13 
 
the 6-month outcome of r=.6. Effect sizes from our previous work with TRAC and iTRAC showed effect sizes 
ranging from .20 to .70.  
Deliverable: A summary report of the trial data will be created and submitted for publication. 
 
Summary of Phase 2 Milestones:  
1. Milestone 5: Digital Modules. Based on the specifications document, Klein Buendel will create the digital 
activities that will be integrated with the existing iTRAC intervention. Deliverable 5: A complete digital version 
of the adapted TRAC intervention for sexual risk prevention, iTRAC-SRP, will be created. 
2. Milestone 6: Acceptability Testing. Acceptability of iTRAC-SRP will be assessed by recruiting youth in the 
target age range to complete the full intervention and provide feedback after each module via questionnaires 
and interviews. Deliverable 6: A summary report of quantitative and qualitative data from the Acceptability 
testing questionnaires and interviews will accompany a table of recommended changes that will guide Klein 
Buendel’s revisions to the program, leading to the finalized iTRAC-SRP program. 
3. Milestone 7: Efficacy Data. A randomized controlled trial with assessments at baseline, post-intervention 
(2 months), and 6-months follow-up will provide information regarding adolescents’ perceived self-efficacy for 
sexual risk prevention behaviors, an important construct at the intersection of emotion regulation and sexual 
health. Other variables of interest will include sexual health knowledge, sexual behaviors, and emotion compe-
tencies. Further, acceptability of the program will be assessed at immediate posttest, to provide additional in-
formation for commercial viability. Deliverable 7: A report of the trial data will be submitted for publication.  
 
RECRUITMENT AND RETENTION PLAN 
The approach for recruitment will be similar for both phases of the project, with the exception that, given the 
small numbers of students needed for the Translation and Acceptability stages of the project and adolescent 
enthusiasm for participation, sufficient numbers of participants can be recruited with a few classroom presenta-
tions (described below).  
Adolescent Recruitment: Several considerations for recruitment have been included in the proposed plan, 
including the time needed to enroll the target number of participants. First, to maximize the number of students 
approached, we will recruit over 8 months, crossing two school years. In this way, two cohorts of 7th graders at 
each school will be eligible for participation during the recruitment period. Participants will be approached dur-
ing the school year, and school vacations during summer have been accounted for in the recruitment plan. 
Our team has successfully recruited diverse families in previous projects, including the pilot trial (33% White; 
30% endorsed Latino ethnicity). The participating schools serve students from diverse racial and ethnic back-
grounds (see Planned Enrollment Table). Our participant sample will reflect current Cranston school de-
mographics; we expect at least 50% of the sample to endorse racial or ethnic minority identity.  
As noted in the application, participants will be recruited through several methods, all of which are facilitated by 
our excellent relationships with the participating schools. All of the methods described have been used in our 
intervention research laboratory to successfully recruit samples. First, project staff will introduce students to the 
study of iTRAC-SRP via a brief presentation describing the program, arranged by project staff with school staff 
assistance. This presentation emphasizes the content (health topics relevant to middle schoolers, such as sex) 
and approach (gaming), both of which are of interest to adolescents in this age group. The presentations are 
conducted in groups (e.g., homerooms), and project staff members distribute flyers that depict engaging im-
ages of the intervention with “consent to contact” forms for parents to sign and return if interested in participa-
tion. Homeroom staff will provide reminders, and project staff will be present at schools on a regular basis to 
serve as reminders to students as well. Recruitment for the Advisory Boards and Acceptability Trial is often 
enhanced by adolescents’ interest in providing their opinions to advise on the development of a product, lead-
ing to a short recruitment period. Recruitment for the RCT is staggered over time, and anecdotal evidence sug-
gests that positive word of mouth about the program (from those classes that receive the presentation and thus 
participate first) enhances recruitment during this phase.  
Our previous studies have also demonstrated that having an identified school staff member who serves as a 
“project champion” and liaison to our study team is key to successful recruitment, thus we will identify such a 
“champion” at each school during the startup period of iTRAC-SRP. The participating schools are familiar with 
this model; we will reimburse schools for staff time and school space used by the project, which eases the bur-
den of recruitment procedures and enhances the likelihood of success.  

 
Version 1.2  9/27/22                                                                              14 
 
Another strategy for recruitment that avoids relying on adolescents to transport paperwork is the use of email 
communication with parents. Principals at participating schools send regular updates to parents and have in-
cluded an introduction to our projects with a link to a REDCap page for parents to electronically complete the 
consent to contact form that allows project staff to contact them with details about the project.  
Finally, project staff recruit at school events attended by parents (open houses, awards nights, school plays, 
sporting events, etc.). Again, having a presence at events where some families have previously participated in 
the project and spontaneously refer other parents to the project table has proven beneficial to recruitment. 
Recruitment is also enhanced by the respect for participants that is a core component of the training received 
by research assistants responsible for interactions related to recruitment. Families are encouraged to take time 
to discuss the project and to ask questions at all phases, including when deciding whether to schedule a con-
sent appointment. Respect for families’ time is conveyed by the reimbursement schedule that compensates 
both adolescents and parents separately for the time invested in completing assessment procedures for the 
project. Respect for school procedures and personnel has also fostered positive, ongoing relationships be-
tween our research lab and the local school districts; these supportive relationships enhance collaborative 
problem solving around recruitment when issues arise. 
Health Education Professionals Panel (HEPP) Recruitment: Health teachers and other school health pro-
fessionals (e.g., nurses, counselors) will be recruited (n=8) for the HEPP to advise on the adaptation of the 
TRAC intervention during Phase 1. School staff directly involved in health of middle school students at each 
participating school, in addition to school health personnel who are members of the Rhode Island Department 
of Health (RIDOH) school nurses listserv and representatives of the Planned Parenthood Federation of Amer-
ica, will be invited to participate in the panel meetings during Phase 1. Staff will be contacted through the 
RIDHO listserv or through their publicly available organization or school district email accounts, a strategy that 
has been successful in our current research.  
Intervention Retention: After families are enrolled in the project, retention to completion of the intervention 
will be critical to assessing its efficacy. The pilot trial showed excellent retention to the intervention, with 88% of 
adolescents randomized to the intervention condition completing all 4 sessions of the intervention. For the 
group-based format of TRAC, average attendance was 75%.  
Several strategies will be used to engage and retain participants in iTRAC-SRP.  
1) iTRAC was designed with intervention retention in mind by emphasizing an interactive gaming approach to 
the intervention that is appealing to adolescents and encourages participants to complete the program.  
2) “Badges” are awarded for completing activities in the program. Used in the pilot with success, these badges 
are given names that reflect the construct of the game but are also engaging (e.g., “Emotion Commotion,” 
“Truth Sleuth”). They mark progress through the program, and a tally of badges earned is noted in the upper 
corner of the game. Other in-program reinforcers will be employed in the complete program, such as virtual 
prizes (e.g., dressing up their character in the game) that enhance connection and individualize experience 
with the program.  
3) Retention to the intervention is enhanced by the health content of iTRAC-SRP, especially the topics of rela-
tionships and sexual health. These topics are of interest to early adolescents, who anecdotally complain that 
information received in health class regarding sex and relationships is inadequate. In addition, the program 
teaches strategies for managing emotions that often interfere with relationships for early adolescents, thus cre-
ating interest in learning more through continued participation in the intervention.  
4) Adolescents complete the interventions in locations convenient, familiar, and comfortable for them. As in the 
pilot, sessions will be held immediately after school at the school (e.g., library or classroom; all schools have 
procedures allowing project staff to use their facilities after hours). This makes attendance for students feasible 
and convenient. Accommodations can be made for students who have after school commitments, though 
these are rarely daily events at this developmental stage. Alternative locations can include the local library (all 

 
Version 1.2  9/27/22                                                                              15 
 
local libraries have rooms that can be reserved for privacy of assessment or intervention procedures), adoles-
cents’ homes (staff travel in pairs to ensure safety), and the research office.  
5) Providing families with flexibility in timing of appointments is maintained as a priority by project staff. Several 
days each week will be offered for program completion right after school. This flexibility is an advantage over 
group-administered interventions, which are restricted to a specific time and can lead to missed content when 
participants miss group. Mobile formats allow participants to receive 100% of the content at their own pace.  
6) Reminders will be sent via text messages to parents (and adolescents who have phones and permission to 
receive texts from research staff) regarding scheduled appointments to complete intervention sessions. 
7) Snacks will be provided. Work from our previous trials has shown that hunger can lead adolescents to leave 
school at the end of the day; providing snacks reinforces attendance and focus.  
Follow-up Retention: Retaining participants for follow-ups is critical to assessing efficacy. The PI has demon-
strated strong retention to study completion with challenging populations in other studies (e.g., middle school-
ers with mental health symptoms: 85% at 30-month follow-up (R01 NR011906); dyads of adolescent boys and 
their parents: 88% at 9-month follow-up (2014-MU-CX-0002). Similarly, the pilot study of iTRAC retained 94% 
of adolescents for the final assessment (3-months). We will facilitate retention over the proposed 6-month fol-
low-up using successful procedures from our own studies and the literature.  
1) Seventh graders attend 8th grade in the same middle schools, which facilitates tracking.  
2) Compensation to families conveys respect for participant time and effort. Adolescents will receive $50 for 
completing assessment procedures at each time point (baseline, post-intervention/2-month, and 6-month fol-
low-up) to convey the importance and value of completing the project. Parents will receive $30 for each of their 
assessments (baseline and 6-month follow-up) which involve less time than adolescent assessments. 
3) The informed consent process is an opportunity to educate participants about the importance of data collec-
tion to research studies. To motivate collaboration, families will be educated during the consent process as to 
why completing the intervention (if randomized to iTRAC-SRP) and follow-up assessments is important to the 
research and problem solve any anticipated barriers to participation.  
4) Each school will have a dedicated research assistant who will assume primary responsibility for recruitment 
and tracking of participants at that school. In this way, participants have a primary project contact who will con-
duct project activities with the family over time to develop rapport with families. This encourages familiarity and 
comfort with the project and, in turn, retention.  
5) Follow-ups will be conducted at locations convenient to families (see Intervention Retention, item 4). The 
vast majority of adolescents prefer to complete study activities at school. However, alternative locations can 
include the local library, adolescent’s home, or the research office. All assessment procedures can be com-
pleted in any of these locations, due to the portability of study devices. 
6) Parent assessments will be brief (15-20 minutes) and follow-ups can be completed by email or phone. 
7) To limit barriers to families of diverse cultural backgrounds, all parent materials (consents, questionnaires) 
will be available in Spanish, and at least one Spanish-speaking RA will be a part of the staff.   
8) Other mailings, such as holiday cards are sent to enhance recall and familiarity with the project.  
9) Standard assessment procedures include research assistants providing appointment reminders by 
phone/text and providing scheduled appointments in writing. Standard lab procedure also includes inquiring 
about anticipated moves or phone number changes, to avoid challenges in reaching families for follow-ups. In 
the unusual circumstance in which a family moves out of the range in which a research assistant can conduct 
an in-person assessment, the questionnaire portion of the battery can be emailed to adolescents via REDCap.  

 
Version 1.2  9/27/22                                                                              16 
 
10) Families complete locator forms that provide contacts who can help staff locate the family if their contact 
information changes. These forms are completed at each assessment point and include a check box and sig-
nature procedure for parents to permit schools to provide updated contact information as well. 
 
 
HUMAN SUBJECTS RESEARCH 
This study is nonexempt research involving human subjects. Rhode Island Hospital (FWA#00001230) will pro-
vide the oversight IRB for the study. Approval for the project and for consent forms will be obtained from the 
Rhode Island Hospital Institutional Review Board.   
 
PROTECTION OF HUMAN SUBJECTS 
 
1. RISKS TO THE SUBJECTS 
 
a. Human Subjects Involvement and Characteristics 
 
In Phase 1, two Adolescent Advisory Boards (AABs) composed of 8 members each will be convened in Rhode 
Island. A Health Education Professionals Panel (HEPP) of 8 school professionals will also be formed to advise 
on the translation.  
In Phase 2, sixteen participants will take part in the Acceptability Testing Phase to complete the intervention 
and assessment to test procedures for the RCT. One hundred twenty adolescents (ages 12-14) and their par-
ents will be recruited from public middle schools in Rhode Island for the Randomized Controlled Trial Phase.  
 
For both phases, Inclusion criteria: enrolled at a participating school, in the 7th grade, between the ages of 12 
and 14 years, and has a parent/legal guardian who speaks English or Spanish to provide consent. Exclusion 
criteria: unable to read English at a 4th grade level, has a sibling who has participated in the project, or has ob-
servable cognitive or developmental delays that would preclude participation. In the RCT, eligible adolescents 
will be randomized to participation in a web-based, tablet-delivered emotion regulation intervention (iTRAC-
SRP) or a waitlist control until the goal of 120 participants is reached. Adolescents will be recruited from four 
schools in the Cranston School District: Hope Highlands Middle School, Bain Middle School, Western Hills 
Middle School, and Park View Middle School. (See Resources for more information on sites).  
 
Students will be introduced via a brief description of the program emphasizing the content (sexual and emo-
tional health) and approach (gaming), both of which are of interest to this age group. The introductions will be 
conducted by project staff in small groups (e.g., homerooms), who will distribute flyers with “consent to contact” 
forms for parents to sign and return. Principals will introduce the project via emails sent to all parents with 
school announcements. A REDCap link to the consent to contact form is attached for parents to complete elec-
tronically, to avoid students having to remember to return the form. Project staff will also recruit at school 
events attended by parents (open houses, sporting events, etc.).   
 
Once permission to contact the family has been received, school staff involvement ends (to avoid per-
ceived pressure to participate). Trained research staff will contact parents to arrange an individual 
meeting to describe the project activities they are being asked to complete and obtain parental con-
sent. Participants will be compensated for their time and effort. Staff will separately explain the project 
to adolescents and obtain assent away from their parent’s presence.  
 
These procedures will be used for both Phase 1 and Phase 2. Fewer homerooms will need to be ap-
proached for the Translation and Acceptability stages, as fewer participants are needed.  
 
A Health Education Professionals Panel of school personnel key to student health will also be formed to advise 
on the adaptation of the intervention to digital format. Identified from throughout the school system, this will in-
clude individuals with involvement in student health and risk behaviors (e.g., health teachers, administrators, 
school counselors, school nurses). Once each pool of professionals from Cranston Public Schools is identified, 
individuals within each profession will be chosen at random, to ensure a mix of perspectives are represented in 
the HEPP. They will be contacted using their publicly available school email addresses and offered participa-
tion. Additionally, members of the Rhode Island Department of Health (RIDOH) school nurses listserv, as well 

 
Version 1.2  9/27/22                                                                              17 
 
as representatives from the Planned Parenthood Federation of America, will be contacted and encouraged to 
outreach study staff if interested in participating. 
 
Adolescent participation for all activities will require parent consent and adolescent assent. Adolescents and 
their parents will be agreeing to let us use adolescent feedback as data for program adaptation and develop-
ment in the Translation and Acceptability Testing stages. In the RCT stage, adolescents and their parents will 
be agreeing to the adolescent’s participation in an 8-module tablet-delivered health intervention and completion 
of computerized questionnaires at three points (baseline, 2-, and 6-month follow-ups). Similarly, parents will 
provide responses to questionnaires. They will be agreeing to let their responses, collapsed with those of other 
participants, be used for reporting scientific results. Adolescents and their parents will be assured that they are 
free to withdraw from the study at any time.  
 
For all phases, once permission to contact the family has been received, trained research staff will contact par-
ents to introduce and describe the program and what it involves for parents and teens. At the conclusion of this 
conversation, parents will be asked whether they would be interested in meeting to learn more about the pro-
gram. Research staff will then arrange an individual meeting at a location convenient to them (e.g., the re-
search office, the school, home, etc.) to describe the project in detail, obtain parental consent, and (for the 
RCT Phase) complete measures, for which they will be compensated for their time and effort. Separate meet-
ings for adolescents will be arranged (typically at the school) to explain the project and obtain assent away 
from their parent’s presence. Adolescents will be reassured that they do not have to participate in the program 
despite their parent’s consent. They will be told that if they do not wish to participate, the research staff will not 
disclose to parents the reason that the student is considered ineligible, in order to protect their privacy and 
avoid coercion to participate. These methods have been successful in recruiting and consenting eligible ado-
lescents in our previous projects. 
 
b. Sources of Materials 
Participants in the Translation Stage (Phase 1) will provide feedback through discussions (AAB focus groups). 
Study staff will take notes on feedback provided by the Adolescent Advisory Boards. Data collected during this 
stage will include de-identified written notes taken by the research team during the meetings. During focus 
groups, discussions will be audiorecorded.  
 
Participants in the Acceptability Testing Stage (Phase 2) will complete brief questionnaires assessing accepta-
bility and usability after each module and participate in interviews about their experience with the digital mod-
ules (e.g., learning process, etc.). These discussions will be audiorecorded as well. They will also complete 
self-report questionnaires and computerized tasks to test assessment procedures for the RCT. 
 
Participants in the Randomized Trial (Phase 2) will complete audio computer-assisted self-interviews (ACASI) 
on laptops at baseline, 2-, and 6-months after baseline. Adolescents will complete several questionnaires as-
sessing demographic information, sexual attitudes and cognitions, risk behaviors, and emotional functioning. 
Parents will complete a locator form to assist in retention and tracking, as well as measures of demographic 
information and their adolescent’s emotion regulation. Teens will also complete performance measures related 
to emotion regulation (Behavioral Indicator of Resilience to Distress; Diagnostic Analysis of Nonverbal Accu-
racy-2).  
 
c. Potential Risks 
The risks in this study are considered minimal. During the Translation phase, participation requires only attend-
ance at Advisory Board meetings where opinions will be shared, though participants will be advised that they 
may decline to answer any questions at any time. During Acceptability Testing, participation requires comple-
tion of the web-based program across several sessions, and completion of measures and interviews that as-
sess the acceptability and clarity of the web-based program. Similarly, during the RCT, participation requires 
completion of measures as described above and participation in the finalized web-based program.  
 
Some adolescents and parents may feel uncomfortable with the topics regarding sexual health. Families are 
notified of this content during the informed consent process. These topics will be discussed sensitively and 
openly, in a developmentally appropriate way during all phases of the project. Moderators during feedback 

 
Version 1.2  9/27/22                                                                              18 
 
sessions will be trained to help participants with topics of sex and sexuality. In addition, they will heavily en-
courage confidentiality and respect. 
 
With respect to use of the ACASI for data collection, the computer-assisted format is expected to reduce partic-
ipants’ discomfort answering questions about sensitive topics such as sexuality because the computer assess-
ment format will prevent others from becoming aware of adolescents’ answers. However, it will also be empha-
sized to participants that they may skip anxiety-provoking questions on any survey and that they can terminate 
their participation at any time during the course of data collection or interventions. If at any point during data 
collection or during the intervention, a parent or adolescent wishes to discontinue participation, they will be al-
lowed to withdraw from the study without negative consequences.  
 
While not directly assessed during the intervention or assessment procedures, if a subject discloses an abu-
sive experience to project staff, parents will be notified and the matter resolved in accordance with standard 
hospital clinical practice (see Protection Against Risks). A similar protocol (described below) will be followed 
if adolescents disclose suicidal or homicidal ideation during the study.  
 
2. ADEQUACY OF PROTECTION AGAINST RISKS 
 
a. Informed Consent and Assent 
Only research staff at Rhode Island Hospital will consent participants. Given that our study involves children, 
we have carefully designed our consent and assent procedures to meet HHS regulatory requirements for pa-
rental permission and child assent (45 CFR 46.408). Specifically, trained project staff will gather informed con-
sent/assent from all participants in person. Parents will provide informed consent for their adolescent and their 
own participation in the project. Adolescents will provide assent for participation. Staff members will inform the 
parent of the research study procedures, discuss the inherent risks, explain the rules of confidentiality and ob-
tain consent. Separate from parents, staff members will individually speak with each adolescent to describe the 
study procedures, review the inherent risks, discuss the rules of confidentiality and obtain assent. Research 
staff will be available to answer any questions about the study. All signed consent forms will be locked away 
securely and kept separate from any study data (see below). Participants will be provided with copies of the 
Informed Consent/Assent Forms. 
 
b. Protection Against Risk 
 
As noted above, participation in all phases of the project is entirely voluntary and participants are free to with-
draw from the project at any time, without negative consequences to their affiliation with the referring school.  
 
Confidentiality. The study will maintain confidentiality of stored records according to the following guidelines:  
• 
All records will be kept by assigned study numbers;  
• 
RIH will maintain locked files of all gathered data and permit access only to authorized individuals. (The 
schools and any associated staff members will not have any access to these records.);  
• 
Completed, signed consent forms will be stored in a separate, locked cabinet that holds no other study 
information;  
• 
Data collected on laptops will be downloaded after each assessment onto a flash drive as separate pass-
word-protected files prior to transfer to RIH’s secure servers; the flash drives will be stored in a locked 
cabinet at RIH;  
• 
Audio files will be transferred from recording equipment to RIH encrypted servers and deleted from the 
original source. Any backup drives used will also be encrypted. Files will be deleted at the conclusion of 
the study; 
• 
RIH will maintain a separate, secured digital file that holds the list linking study identification numbers to 
names. Only the Principal Investigator and the research assistants will have access to this list, which will 
be destroyed at the completion of the project; 
• 
Notes and transcripts from focus groups will be deidentified and will be kept in locked cabinets;  
• 
Conclusions and final results will be presented for aggregate data only, with no disclosure of specific 
cases;  

 
Version 1.2  9/27/22                                                                              19 
 
• 
Collected information will not be released without the explicit consent of the participant, according to the 
Privacy Act;  
• 
No personal identifying information will be released to NIH.   
• 
All personal identifying information will be removed from data prior to sharing with Klein Buendel, Inc staff. 
 
Backend user data during the Acceptability and RCT stages will be collected and stored via internet to Klein 
Buendel’s server farm. KB’s server farm has five Dell PowerEdge servers with two 3.2 Ghz Xeon processors, 
two with 32 gigabytes of RAM and two with 128 gigabytes, and three 1 Terabyte hard drives that operate off a 
hardware RAID5 system, connected to a local area network (LAN) running the Windows 2008 or 2012 operat-
ing system, and to the Internet through a 50 megabyte fiber connection. KB programmers monitor and maintain 
all programs and databases, working with data management staff. Sensitive information is protected by a hard-
ware firewall (SonicWall TZ600) and each server has its own native Windows security software. All KB servers 
are connected via 1 Gigabit high-speed switched network, ensuring high-speed transfer between machines. All 
networked computers are protected from viruses by Sunbelt Vipre Business. A nightly backup of each com-
puter provides protection against the loss of data.  
 
Trauma-Informed Approach. Consistent with SAMHSA’s Trauma-Informed Approach and guidelines published 
by CARDEA (Guide to Trauma Informed Sex Education) and the Responsible Sex Education Institute, iTRAC-
SRP will be sensitive to the experiences of participants with a history of trauma. These focus on principles fun-
damental to a trauma-informed approach. While some recommendations apply to in-person, group-based edu-
cation programs (collaboration and mutuality, peer support), many can be incorporated into technology-deliv-
ered platforms.  
 
Safety, both physical and emotional, is the foundation for a trauma-informed approach to sexual health educa-
tion. iTRAC-SRP has advantages in creating a safe environment that may not exist in some group-based pro-
grams. The content is standardized and avoids shaming language (“you shouldn’t have sex…”). Participants 
may proceed at their own pace, maintaining control throughout the program. In addition, other participants are 
not present to introduce content that may lead to strong emotions based on their personal experiences. In this 
way, technology-delivered sex education may be a more effective strategy than classroom-based learning 
where some students may have undesired responses. Trustworthiness and transparency will be achieved by 
alerting participants to upcoming sexual health content before it is presented (e.g, drawings of sexual anat-
omy). These activities are also used as practice with emotion regulation, as they are emotionally arousing for 
most adolescents regardless of trauma history because of the taboo around sexual health in US culture. Thus, 
coaching will be provided for use of ER skills to manage feelings while learning this important content. These 
skills are expected to increase the likelihood that participants will recall the information provided and will in-
crease self-efficacy for ER strategy use. Empowerment and choice are facilitated through the teaching of emo-
tion regulation skills as a way to manage emotions that might otherwise influence decisions in ways that ado-
lescents do not intend. Other strategies for creating empowerment include reaffirming bodily autonomy during 
all sexual health content- that they have many choices over what happens to their body and when- and allow-
ing participants to skip questions that they prefer not to answer during assessment procedures. Cultural, histor-
ical, and gender issues will be considered in all elements of the program, to avoid stereotypes and biases. The 
use of a space theme allows for use of non-human characters that are not easily associated with cultural stere-
otypes. Of particular importance in sex education programs is consideration of heteronormative language that 
may exclude youth who identify as LGBTQ. Inclusive language (e.g., “partner” vs. “boyfriend/girlfriend”) creates 
a safe environment for all participants and reflects sensitivity to the traumas experienced by many who have 
historically had less power in our culture.    
 
The sexual health content of TRAC has been used in a previous large trial without adverse events. Staff will be 
provided education to improve awareness of how trauma can affect youth, families, and communities to inform 
their day-to-day work with participants. Descriptions of violence or abuse that might be particularly sensitive for 
those with trauma histories are not a part of the original TRAC program. Adolescents consistently see sexual 
content in media and social media; the content of iTRAC-SRP will be developmentally appropriate and provide 
a context in which adolescents can interpret the information that they receive in other forums. People with 
trauma histories can lead full lives and develop healthy sexual identities; sexual health education is critical to 
this development. Respectful, affirming programs are key to this process and can be achieved through 

 
Version 1.2  9/27/22                                                                              20 
 
technology. Those that incorporate emotion regulation skills may be particularly valuable to those with trauma 
histories as they navigate sexual content and experiences in their lives.   
 
If a subject discloses an abusive experience, suicidal, or homicidal thoughts and/or behavior during any phase 
of the project, a trained research staff member will interview the subject immediately and the PI will be con-
tacted. Next, the parent will be notified and the matter resolved in accordance with standard medical/psychiat-
ric care as follows. Abuse is a reportable condition. In the majority of cases, a history of abuse has already 
been reported to the appropriate authorities. If not previously disclosed, the situation will be reported to authori-
ties per state law and hospital standards. Suicidal or homicidal thoughts or behavior will be handled in the 
same fashion and the case will be reviewed immediately with one of the investigators. If the participant is in 
imminent danger, the participant will be taken by ambulance to the nearest hospital emergency room for imme-
diate intensive psychiatric evaluation. Participants are informed of the limits to confidentiality, including these 
reportable conditions, during the consent process.  
c. Vulnerable Subjects 
Our study is aimed at completing the development and testing the efficacy of a preventive intervention de-
signed for middle schoolers. As such, this study necessitates the involvement of children. Importantly, this 
study presents no greater than minimal risk to children. Furthermore, as noted above, we have made numer-
ous provisions for soliciting the assent of the children and the permission of their parents or guardians.  
 
3. POTENTIAL BENEFITS OF THE PROPOSED RESEARCH TO THE SUBJECTS AND OTHERS 
We hope that the emotion regulation training of the iTRAC intervention will be successful in reducing the rates 
of sexual onset and other health risks in our subject population. Waitlist control participants will be provided the 
opportunity to complete iTRAC-SRP after a 6-month delay. We believe that the clear examination of these 
questions outweighs the previously mentioned risks. Youth have an opportunity to learn emotion regulation 
strategies and consider safer sexual behavior to avoid future risks. The potential benefits to individual subjects 
outweigh potential risks, particularly in offering adolescents a chance to better understand the nature and con-
sequences of their risk behaviors and to learn skills related to emotion regulation. 
 
4. IMPORTANCE OF THE KNOWLEDGE TO BE GAINED 
This study will provide an important sex education tool and elucidate the role of emotion regulation in improving 
sexual self-efficacy, a critical factor in healthy sexual activity during development, and delaying early sexual 
debut, which is associated with significantly greater future sexual risk, including more unwanted pregnancies, 
more sexually transmitted infections, greater number of sexual partners, greater frequency of intercourse, and 
less condom use. Even if an adolescent or parent does not directly benefit from the project, the information we 
gain may be used to improve the effectiveness of sexual health programs for other early adolescents. 
 
5. HUMAN SUBJECTS PROTECTION EDUCATION 
To meet mandatory training certification in Human Subjects Protection, the Office of Research Administration 
offers an online training program through CITI, Collaborative IRB Training Initiative. This online program offers 
both the initial certificate and a three-year re-certification program required by Lifespan. Drs. Houck, Hadley, 
and Barker have completed required training through CITI. 
 
Klein Buendel, Inc. maintains compliance with mandatory training through the NIH Office of Extramural Re-
search’s training program. Ms. Berteletti and all required staff members have completed this training.  
 
Collaborating Sites  
This research involving human subjects will take place at both Rhode Island Hospital (FWA#00001230) and 
the participating school districts. School sites will be made aware by project staff of proper procedures for re-
porting any issues related to risks related to human subjects. Due to their limited role in the study (informing 
families about the availability of research and permitting use of their facilities for intervention) schools do not 
meet criteria for engagement in research according to OHRP guidance (dated October 16, 2008). Rhode Is-
land Hospital will provide the oversight review board for the study for all activities occurring in Rhode Island. 
Western Institutional Review Board will provide oversight for any study activities taking place at Klein Buendel 
(FWA#00003715).  

 
Version 1.2  9/27/22                                                                              21 
 
 
 
DATA AND SAFETY MONITORING PLAN 
The nature of the population and setting warrants the development of a Data Safety and Monitoring Plan. The 
current study is considered to present minimal risk to participants given that subjects will provide data via ques-
tionnaires and performance measures as well as participate in a psychosocial intervention. In this plan, the 
Principal Investigator with input from colleagues (Hadley, Barker, Berteletti) will provide oversight of all recruit-
ment and study procedures.  
 
Specifically, the Principal Investigator will take primary responsibility for: 
• 
Monitoring the safety of the participants  
It is possible, though it has never happened, that adolescents in the schools may become aggressive with 
each other. Each school has identified personnel who are present during after school activities, and session 
moderators will have cell phones to contact any necessary assistance.  
 
 
 
• 
Monitoring the safety of the researchers   
It is possible, though unlikely, that the physical safety of the PI and facilitators could be threatened, either 
through direct implementation of the study (e.g., a participant becomes agitated and assaults a facilitator) or 
while in the school environment more generally (e.g., an adolescent uninvolved in the study assaults a facilita-
tor at the school for the project). To help prevent and/or respond to such occurrences, school staff will be on-
site for assistance. Such events are unlikely to occur, but in the case that they do, the event will be reported to 
the PI and school staff immediately. If any project staff is injured as a result of such an event, they will seek 
care, as warranted, by Rhode Island Hospital’s Employee Health Services. 
 
Home visits will be conducted in pairs, to ensure staff safety. Lab procedures include providing supervisors 
with time and location of all home appointments and texting a supervisor prior to entering the home as well as 
when the pair has left. 
 
• 
Maintaining the confidentiality and integrity of the data 
Data will be confidentially maintained under the stringent guidelines put forth in the aforementioned Protection 
Against Risk (see Human Subjects Protection). RIH will maintain locked files, stored off-site, and permit 
electronic access only to authorized individuals. ID numbers are used as identifiers throughout databases. The 
collaborating schools and any associated staff members will under no circumstances have any access to rec-
ords.  
 
• 
Receiving/eliciting reports of adverse events from research assistants (RAs) and school staff   
The PI will receive these reports on an event-by-event basis. They will also be elicited in an open-ended man-
ner through regular contact between the PI and school staff. 
 
• 
Reporting adverse events to: 
 
Co-investigators 
School  
The Rhode Island Hospital IRB 
The Program Officer at NIH 
 
The major adverse event that could occur as a result of study participation includes psychological distress from 
study procedures resulting in hospitalization. Project staff will be instructed to immediately contact the PIs 
when they first become aware of any event that could potentially be considered adverse. The PIs will immedi-
ately investigate the event and determine the appropriate manner in which to proceed. It is the responsibility of 
the PI to review serious adverse event reports, provide commentary, and provide oversight to ensure that re-
ports are relayed to the co-investigators, Rhode Island Hospital IRB, and to the OHRP, as indicated. In the 
event of adverse events or unexpected problems posing risks to subjects, the PI is empowered to modify pro-
tocols and/or request interim data analyses.  
 

 
Version 1.2  9/27/22                                                                              22 
 
Department of Health and Human Services (DHHS) Required Education in the Protection of Human Re-
search Participants  
 
To meet mandatory training certification in Human Subjects Protection, the Office of Research Administration 
offers an online training program through CITI, Collaborative IRB Training Initiative. This online program offers 
both the initial certificate and a three-year re-certification program required by Lifespan. Drs. Houck, Barker, 
Hadley, and all research staff in their laboratory have completed required training. Klein Buendel, Inc. main-
tains compliance with mandatory training through the NIH Office of Extramural Research’s training program. 
Ms. Berteletti and all required staff members have completed this training.  
 
REFERENCES 
1. Kann L, McManus T, Harris WA. Youth Risk Behavior Surveillance - United States, 2017. MMWR. 
2018;67:SS-9.  
2. CDC. Youth risk behavior surveillanace-United States, 2017. Morbidity and Mortality Weekly Report. 2017; 
3. National Research Council. Adolescent health services: missing opportunities. Washington, DC: The 
National Academies Press; 2009. 
4. Valois R, Bryant E, Rivard J, Hinkle K. Sexual risk-taking behaviors among adolescents with severe 
emotional disturbance. J Child Fam Stud. 1997;21:789-803.  
5. Capaldi D, Stoolmiller, M., Clark, S., & Owen, L. Heterosexual risk behaviors in at-risk young men from 
early adolescence to young adulthood: Prevalence, prediction, and association with STD contraction. Dev 
Psychol. 2002;38:394-406.  
6. Sandfort T, Orr M, Hirsch J, Sentelli J. Long-term health correlates of timing of sexual debut: Results from 
a national U.S. study. Am Journal of Public Health. 2008;98:155-161.  
7. Magnusson BM, Masho SW, Lapane KL. Adolescent and sexual history factors influencing reproductive 
control among women aged 18-44. Sexual Health. 2011;8(1):95-101. doi:10.1071/SH100007 
8. Kaestle C, Halpern C, Miller W, Ford C. Young age at first sexual intercourse and sexually transmitted 
infections in adolescents and young adults. American journal of epidemiology. 04/15/05 2005;161(8):774-780. 
doi:10.1093/aje/kwi095 
9. Finer L, Philbin J. Trends in ages at key reproductive transitions in the United States, 1951-2010. Women's 
Health Issues. May-Jun 2014;24(3):e271-e279. doi:10.1016/j.whi.2014.02.002 
10. Boisvert I, Boislard M, Poulin F. Early sexual onset and alcohol use and misuse from adolescence into 
young adulthood. Journal of Adol Health. 2017;61:4.  
11. O'Sullivan L, & Brooks-Gunn, J. The timing of changes in girls' sexual cognitions and behaviors in early 
adolescence: A prospective, cohort study. Journal of Adolescent Health. 2005;37:211-219.  
12. Kirby D. Mathtech questionnaires: Sexuality questionnaires for adolescents. In: Davis CM, Yarber WL, eds. 
Handbook of Sexuality-Related Measures. Sage Publications; 2004:35-46. 
13. Romer D, Stanton B. Feelings about risk and the epidemic diffusion of adolescent sexual behavior. 
Prevention science : the official journal of the Society for Prevention Research. 2003;4:39-53.  
14. Udry J, Billy J, Morris N, Groff T, Raj M. Serum androgenic hormones motivate sexual behavior in 
adolescent boys. Fertility and sterility. 1985;43:90-94.  
15. Udry J, Talbert, L., & Morris, N. . Biosocial foundations for adolescent female sexuality. Demography. 
1986;23:217-228.  
16. Giedd J. The teen brain: Insights from neuroimaging. Journal of Adolescent Health. 1986:335-43.  
17. Sowell ER, Thompson PM, Holmes CJ, Jernigan TL, Toga AW. In vivo evidence for post-adolescent brain 
maturation in frontal and striatal regions. Nature neuroscience. 1999;2(10):859-861.  
18. Baird AA, Gruber SA, Fein DA, et al. Functional magnetic resonance imaging of facial affect recognition in 
children and adolescents. Journal of the American Academy of Child & Adolescent Psychiatry. 1999;38(2):195-
199.  
19. Bryan D, Magnan R, Gillman A, et al. Effect of including alcohol and cannabis content in a sexual risk-
reduction intervention on the incidence of sexually transmitted infections in adolescents.  A cluster randomized 
clinical trial. JAMA pediatrics. 2018;172(4):e175621.  
20. Lowry R, Dunville R, Robin L, Kahn L. Early sexual debut and associated risk behaviors among sexual 
minority youth. American journal of preventive medicine. 2017;52:379-384.  
21. Ritchwood T, Ford H, DeCoster J, Sutton M, Lochman JE. Risky sexual behavior and substance use 
among adolescents: a meta-analysis. Children Youth Serv Review. 2015;52:74-88.  

 
Version 1.2  9/27/22                                                                              23 
 
22. Hessler D, Katz L. Associations between emotional competence and adolescent risky behavior. J Adolesc. 
2010;33:241-46.  
23. Sussman S, Dent, C., & Galaif, E. The correlates of substance abuse and dependence among adolescents 
at high risk for drug abuse. Journal of Substance Abuse. 1997;9:241-255.  
24. Tarter R. Etiology of adolescent substance abuse: A developmental perspective. American Journal of 
Addiction. 2002;11:171-191.  
25. Wills TA, Walker C, Mendoza D, Ainette MG. Behavioral and emotional self-control: relations to substance 
use in samples of middle and high school students. Psychology of Addictive Behaviors. 2006;20(3):265.  
26. Webb T, Miles E, Sheeran P. Dealing with feeling: A meta-analysis of the effectiveness of strategies 
derived from the process model of emotion regulation. Psychological Bulletin. 2012;138:775-808.  
27. Gross JJ. Emotion regulation: Conceptual and empirical foundations. In: Gross JJ, ed. Handbook of 
emotion regulation, second edition. Guilford Press; 2014:3-20. 
28. Tice D, Bratslavsky, E., & Baumeister, R. Emotional distress regulation takes precedence over impulse 
control: If you feel bad, do it! Journal of personality and social psychology. 2001;80:53-67.  
29. Santelli J, Kaiser J, Hirsch L, Radosh A, Simkin L, Middlesteadt S. Initiation of sexual intercourse among 
middle school adolescents: The influence of psychosocial factors. Journal of Adolescent Health. 2004;34:200-
208.  
30. Brown L, Barone V, Fritz G, Cebollero P, Nassau J. AIDS education: the Rhode Island experience. Health 
Education & Behavior. 1991;18(2):195-206.  
31. Pritchard C, Cotton A, Cox M. Truancy and illegal drug use, and knowledge of HIV infection in 932 14-16-
year-old adolescents. Journal of Adolescence. 1992;15 
32. Wills T, Walker C, Mendoza D, Ainette M. Behavioral and emotional self-control: relations to substance use 
in samples of middle and high school students. Psychology of Addictive Behaviors. 2006;20:265-278.  
33. Holmes C, Brieant A, King-Casas B, Sim-Spoon J. How is religiousness associated with adolescent risk-
taking? The roles of emotion regulation and executive functioning. Journal of Res Adolescence. 2019;29:334-
344.  
34. Raffaelli M, Crockett L. Sexual risk in adolescence: The role of self-regulation and attraction to risk. 
Developmental Psychology. 2003;39:1036-46.  
35. Robin L, Dittus P, Whitaker D, et al. Behavioral interventions to reduce incidence of HIV, STD, and 
pregnancy among adolescents: A decade in review. Journal of Adolescent Health. 2004;34(1):3-26.  
36. Fiellin L, Hieftje K, Pendergrass T, et al. Video game intervention for sexual risk reduction in minority 
adolescents: randomized controlled trial. J Med Internet Research. 2017;19(9):e314.  
37. Haruna H, Hu X, Chu S, Mellecker R, Gabriel G, Ndekao P. Improving sexual health education programs 
for adolescent students through game-based learning and gamification. Int J Environmental Res and Public 
Health. 2018;15(9):2027.  
38. Peskin MF, Shegog R, Markham CM, et al. Efficacy of It's your game-tech: a computer based sexual health 
education program for middle school youth. J Adolesc Health. 2015;56(5):515-521.  
39. Guilamo-Ramos V, Benzekri A, Thimm-Kaiser M, et al. A triadic intervention for adolescent sexual health: a 
randomized clinical trail. Pediatrics. 2020;145(5):e20192808. doi:10.1542/peds.2019-2808 
40. Ford C, Mirman J, Garcia-Espana J, et al. Effect of primary care parent-targeted interventions on parent-
adolescent interventions on parent-adolescent communication about sexual behavior and alcohol use: a 
randomized clinical trail. JAMA Network Open. 2019;2(8) 
41. DiClemente RJ, Wingood GM, Sales JM, et al. Efficacy of a telephone-delivered sexually transmitted 
infection/human immunodeficiency virus prevention maintenance intervention for adolescents: a randomized 
clinical trial. JAMA pediatrics. 2014;168(10):938-946.  
42. Metz S, Frank J, Riebel D, Cantrell T, Sanders R, Broderick P. The effectiveness of the learning to 
BREATHE Program on adolescent emotion regulation. Res in Human Dev. 2013;10(3):252-272.  
43. Burke C. Mindfullness-based approaches with children and adolescents: a preliminary review of current 
research in an emergent field. J Child Fam Stud. 2010;19:133-144.  
44. Holmqvist Larsson K, Andersson G, Stern H, Zetterqvist M. Emotion regulations group skills training for 
adolescents and parents: a pilot study of an add-on treatment in a clinical setting. Clinical child psychology and 
psychiatry. 2020;25:141-155.  
45. Ford C, Jaccard J. New skills to reduce sexual risk behaviors among young adolescents. Pediatrics. 
2018;141(6) 

 
Version 1.2  9/27/22                                                                              24 
 
46. Lansing A, Guthrie K, Hadley W, Stewart A, Peters A, Houck C. Qualitative assessment of emotion 
regulation strategies for prevention of health risk behaviors in early adolescents. J Child Fam Stud. 
2019;28:765-775.  
47. Houck C, Barker D, Hadley W, Menefee M, Brown L. Sexual risk outcomes of an emotion regulation 
intervention for at-risk early adolescents. Pediatrics. 2018;141(6):2017.  
48. Houck C, Hadley W, Barker D, Brown L, Hancock E, Almy B. An emotion regulation intervention to reduce 
risk behaviors among at-risk early adolescents. Prevention Science. 2016;17(1):71-82.  
49. Houck CD, Barker DH, Hadley W, et al. The 1-year impact of an emotion regulation intervention on early 
adolescent health risk behaviors. Health Psychology. 2016;35(9):1036.  
50. Nowicki S, & Carton, J. The measurement of emotional intensity from facial expressions: The DANVA 
FACES 2. Journal of Social Psychology. 1993;133:749-750.  
51. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service 
organizations: systematic review and recommendations. Milbank Quarterly. 2004;82(4):581-629.  
52. Grimshaw JM, Shirran L, Thomas R, et al. Changing provider behavior: an overview of systematic reviews 
of interventions. Medical Care. 2001:II2-II45.  
53. Yuan CT. Blueprint for the dissemination of evidence-based practices in health care. Commonwealth Fund; 
2010. 
54. Chou WS, Prestin A, Lyons C, Wen K. Web 2.0 for health promotion: reviewing the current evidence. 
American journal of public health. 2013;103(1):e9-e18.  
55. Griffiths F, Lindenmeyer A, Powell J, Lowe P, Thorogood M. Why are health care interventions delivered 
over the internet? A systematic review of the published literature. Journal of medical Internet research. 
2006;8(2) 
56. Holt CL, Tagai EK, Scheirer MA, et al. Translating evidence-based interventions for implementation: 
Experiences from Project HEAL in African American churches. Implement Sci. 2014;9:66.  
57. Lightfoot M, Comulada W, Stover G. Computerized HIV preventive intervention for adolescents: Indications 
of efficacy. American journal of public health. 2007;97(6):1027-1030.  
58. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult 
depression: a meta-analysis. Cognitive behaviour therapy. 2009;38(4):196-205.  
59. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive 
disorders is effective, acceptable and practical health care: a meta-analysis. PloS one. 2010;5(10):e13196.  
60. Arnberg FK, Linton SJ, Hultcrantz M, Heintz E, Jonsson U. Internet-delivered psychological treatments for 
mood and anxiety disorders: A systematic review of their efficacy, safety, and cost-effectiveness. 2014; 
61. Hedman E, Furmark T, Carlbring P, et al. A 5-year follow-up of internet-based cognitive behavior therapy 
for social anxiety disorder. Journal of medical Internet research. 2011;13(2) 
62. Calear AL, Christensen H. Review of internet-based prevention and treatment programs for anxiety and 
depression in children and adolescents. Medical Journal of Australia. 2010;192(11):S12.  
63. Cuijpers P, Marks I, van Straten A, Cavanagh K, Gega L, Andersson G. Computer aided psychotherapy for 
anxiety disorders: A meta analytic review. Cognitive Behaviour Therapy. 2009/06/01 2009;38(2):66-82. 
doi:10.1080/16506070802694776 
64. Hedman E, Ljótsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of 
applications, clinical efficacy and cost-effectiveness. 2012; 
65. Titov N, Dear BF, Schwencke G, et al. Transdiagnostic internet treatment for anxiety and depression: a 
randomised controlled trial. Behaviour Research and Therapy. 2011;49(8):441-452.  
66. March S, Spence SH, Donovan CL. The efficacy of an internet-based cognitive-behavioral therapy 
intervention for child anxiety disorders. Journal of Pediatric Psychology. 2009;34(5):474-487.  
67. Clarke J, Proudfoot J, Birch MR, et al. Effects of mental health self-efficacy on outcomes of a mobile phone 
and web intervention for mild-to-moderate depression, anxiety and stress: secondary analysis of a randomised 
controlled trial. BMC psychiatry. 2014;14(1):272.  
68. Roberto AJ, Zimmerman RS, Carlyle KE, Abner EL. A computer-based approach to preventing pregnancy, 
STD, and HIV in rural adolescents. Journal of health communication. 2007;12(1):53-76.  
69. Davidson TM, Yuen EK, Felton JW, McCauley J, Gros KS, Ruggiero KJ. Feasibility assessment of a brief, 
web-based behavioral activation intervention for adolescents with depressed mood. The International Journal 
of Psychiatry in Medicine. 2014;48(1):69-82.  
70. Manicavasagar V, Horswood D, Burckhardt R, Lum A, Hadzi-Pavlovic D, Parker G. Feasibility and 
effectiveness of a web-based positive psychology program for youth mental health: randomized controlled trial. 
Journal of medical Internet research. 2014;16(6) 

 
Version 1.2  9/27/22                                                                              25 
 
71. Teens, social media & technology 2018. https://www.pewresearch.org/internet/2018/05/31/teens-social-
media-technology-2018/ 
72. Student Mobile Device Survey 2015. https://www.pearsoned.com/wp-content/uploads/2015-Pearson-
Student-Mobile-Device-Survey-Grades-4-12.pdf. Survey:  
73. Smyth J, Arigo D. Recent evidence support emotion-regulation interventions for improving health in at-risk 
and clinical populations. C Opin Psychiatr. 2009;22:205-210.  
74. Bell C, McBride D. Affect regulation and prevention of risky behaviors. JAMA. 2010;304(5):565-566. 
doi:10.1001/jama.2010.1058 
75. Jenkins R. Supplemental issue on does early intervention prevent health-risking sexual behaviors related 
to HIV/AIDS: Commentary on effects. Prev Science. 2014;15:S84-86. doi:10.1007/s11121-013-0422-6 
76. Brown L, Nugent N, Houck C, et al. Safe Thinking and Affect Regulation (STAR): human immunodeficiency 
virus prevention in alternative/therapeutic schools. J Amer Acad Child Adolesc Psych. 2011;50(10):1065-1074.  
77. Brown L, Houck C, Donenberg GR, Emerson E, Donahue K, Misbin J. Affect management for HIV 
prevention with adolescents in therapeutic schools: The immediate impact of project balance. AIDS Behav. 
2013;17(8):2773-2780. doi:10.1007/s10461-013-0599-5 
78. Burke CA. Mindfulness-Based Approaches with Children and Adolescents: A Preliminary Review of 
Current Research in an Emergent Field. Journal of Child and FAmily Studies. 2010;19:133-44.  
79. Erkut S, Grossman, J.M., ,Frye, A.A., Ceder, I., Charmaraman, I., Tracy, A. Can sex education delay early 
sexual debut? J Early Adolesc. 2012;33:482-497. 80. 
K-12 NSESccas. Future of sex education initiative. The 
Journal of school health. 2020; 
80. Future of Sex Education Initiative. (2020). National Sex Education Standards: Core Content and Skills, K-12 (Second Edition). 
81. Cortes, N., Eisler, A., & Desiderio, G. (2016). Tip sheet: Gender, sexuality, and inclusive sex education. 
Baltimore: Healthy Teen Network. 
82. Vermont Agency of Education (2018). Full Spectrum: Educator’s Guide to Implementing LGBTQ+ Inclusive 
Sex Ed. Barre, VT: Vermont Agency of Education. 
83. Brownson RC, Jacobs JA, Tabak RG, Hoehner CM, Stamatakis KA. Designing for dissemination among 
public health researchers: findings from a anational survey in the United States. American journal of public 
health. 2013;103:1693-1699.  
84. Kelders S, Kok R, Ossebaard H, Van Gemert-Pijnen J. Persuasive system design does matter: a 
systematic review of adherence to web-based interventions. J Medical Internet Res. 2012;14(6) 
85. Oinas-Kukkonen H, Harjumaa M. Persuasive systems design: Key issues, process model, and system 
features. Communications of the Association for Information Systems. 2009;24(1):28.  
86. Oinas-Kukkonen H, Harjumaa M. A systematic framework for designing and evaluating persuasive 
systems. Persuasive technology. Springer; 2008:164-176. 
87. Abras C, Maloney-Krichmar D, Preece J. User-centered design. In: Bainbridge W, editor. Encyclopedia of 
Human-Computer Interaction. Thousand Oaks: Sage Publications; 2004. p. 445-456. 
88. Norman DA, Draper SW. User Centered System Design: New Perspectives on Human-Computer 
Interaction. Lawrence Erlbaum Associates Inc.; 1986. 
89. Peterson C. Bringing ADDIE to life: instructional design at its best. Journal of Educational Multimedia and 
Hypermedia. Jul 2003;12(3):227-241.  
90. Gustafson KL, Branch RM. What is instructional design? In: Reiser RA, Dempsey JV, eds. Trends and 
Issues in Instructional Design and Technology. Merril/Prentice Hall; 2002. 
91. Molenda M. In search of the elusive ADDIE model. Performance Improvement. Feb 2015;54(2):40-42.  
92. Mazzara ML. Application of theory: minimalism and user centered design. 2001. 1891709097. Accessed 
December 20, 2017. http://www.gotomedia.com/goto/web2expo/workshop/resources/minimialism_ucd.pdf 
93. Pagulayan RJ, Keeker K, Wixon D, Romero RL, Fuller T. User-centered design in games. In: Sears A, 
Jacko JA, eds. Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging 
Applications. Lawrence Erlbaum Associaties, Inc.; 2003:883-906. 
94. Nicholson S. A user-centered theoretical framework for meaningful gamification. 2012: 
95. Cafazzo JA, Casselman M, Hamming N, Katzman DK, Palmert MR. Design of an mHealth app for the self-
management of adolescent type 1 diabetes: a pilot study. Journal of medical Internet research. 
2012;14(3):e70.  
96. Graafland M, Dankbaar M, Mert A, et al. How to systematically assess serious games applied to health 
care. JMIR serious games. 2014;2(2) 
97. Kulyk O, den Daas C, David S, van Gemert-Pijnen L. How Persuasive are Serious Games, Social Media 
and mHealth Technologies for Vulnerable Young Adults? 2015; 

 
Version 1.2  9/27/22                                                                              26 
 
98. Hieftje K, Duncan L, Sawyer B, Haskell S, Fiellin L. A tablet game for risk reduction and HIV prevention in 
adolescents. J Mobile Tech Med. 2012;1(4S):8-8.  
99. Shegog R, Brown K, Bull S, et al. Serious games for sexual health. Games for Health Journal. 
2015;4(2):69-77.  
100. 
Nah FFH, Telaprolu VR, Rallapalli S, Venkata PR. Gamification of education using computer games. 
Human Interface and the Management of Information Information and Interaction for Learning, Culture, 
Collaboration and Business. Springer; 2013:99-107. 
101. 
7 things you should know about BADGES. https://net.educause.edu/ir/library/pdf/ELI7085.pdf.  
102. 
Easley D, Ghosh A. Incentives, gamification, and game theory: an economic approach to badge design. 
ACM; 2013:359-376. 
103. 
Miller AS, Cafazzo JA, Seto E. A game plan: Gamification design principles in mHealth applications for 
chronic disease management. Health informatics journal. 2014:1460458214537511.  
104. 
Bangor A, Kortum P, Miller J. An Empirical Evaluation of the System Usability Scale. Int J Hum 
Computer Interaction 2006;24(6):574-594.  
105. 
Friedman L, Furberg C, DeMets D. Fundamentals of clinical trials. 3rd Edition ed. Springer; 1998. 
106. 
Armistead L, Clark, H., & Barber, C., Dorsey, S., Hughley, J., Favors, M., & Wyckoff, S. Participant 
retention in the Parents Matter! program: Strategies and outcome. J Child Fam Stud. 2004;13:67-80.  
107. 
Gwadz M, Rotheram-Borus M. Tracking high-risk adolescents longitudinally. AIDS Educ Prev. 1992:69-
82.  
108. 
Meyers K, Webb A, Frantz J, Randall M. What does it take to retain substance-abusing adolescents in 
research protocols? Delineation of effort required, strategies undertaken, costs incurred, and 6-month post-
treatment differences by retention difficulty. Drug and Alcohol Dependence. 2003;69(1):73-85.  
109. 
Scott C. A replicable model for achieving over 90% follow-up rates in longitudinal studies of substance 
abusers. Drug and Alcohol Dependence. 2004;24:21-36.  
110. 
Smith E, Gorman-Smith D, Quinn W, Rabiner D, Tolan P, Winn D. Community-based multiple family 
groups to prevent and reduce violent and aggressive behavior: The GREAT Families Program. American 
journal of preventive medicine. 2004;26:39-47.  
111. 
Yancey A, Ortega A, Kumanyika S. Effective recruitment and retention of minority research participants. 
Ann Rev Pub Health. 2006;27:1-28.  
112. 
Fisher J, Fisher W. Changing AIDS-risk behavior. Psychol Bull. 1992;111:455-474.  
113. 
Closson K, Dietrich, J., Lachowsky, N., Nkala, B., Palmer, A., Cui, Z., Beksinska, M., Smit, J., Hogg, R., 
Gray, G., Miller, C., Kaida, A. Sexual self-efficacy and gender: A review of condom use and sexual negotiation 
among young men and women in sub-saharan Africa. Journal of Sex Resesarch. 2018;55:1-18. 
doi:10.1080/00224499.2017.1421607 
114. 
Jones D, Richards, C. Correlation between self-efficacy in sexual negotiation and engagement in risky 
sexual behaviors: Pilot study of adolescents attending a secondary school in Grenada, West Indies. Int Public 
Health Journal. 2016;8(3):397.  
115. 
Rostosky SS, Dekhtyar, O., Cupp, P.K., Anderman, E.M. Sexual self-concept and sexual self-efficacy in 
adolescents: a possible clue to promoting sexual health. J Sex Res. 2008;45(3):277-286.  
116. 
Lawrence L, Levy S, Rubinson L. Self-efficacy and AIDS prevention for pregnant teens. J School 
Health. 1990;60:19-24.  
117. 
Miller B, Norton M, Fan X. Pubertal development, parental communication, and sexual values in 
relation to adolescent sexual behaviors. Journal of Early Adolescence. 1998;18:27-52.  
118. 
Donenberg G, Emerson E, Bryant FB, Wilson H, Weber-Shifrin E. Understanding AIDS-risk behavior 
among adolescents in psychiatric care: Links to psychopathology and peer relationships. J Am Acad Child 
Adolesc Psychiatry. Jun 2001;40(6):642-53. doi:10.1097/00004583-200106000-00008 
119. 
Meyer-Bahlburg H, Dugan T, Ehrhardt A. Psychosexual development interview: Child version for sexual 
risk behavior, female (PDI-RISK-CF). Unpublished manuscript. New York, NY: Columbia University; 1998. 
120. 
Gratz KL, Roemer L. Multidimensional assessment of emotion regulation and dysregulation: 
Development, factor structure, and initial validation of the Difficulties in Emotion Regulation Scale. Journal of 
Psychopathology and Behavioral Assessment. 2004;36:41-54.  
121. 
Linehan M. Cognitive-behavioral treatment of borderline personality disorder. Guilford Press; 1993. 
122. 
Brown L, Houck C, Lescano C, Donenberg G, Tolou-Shams M, Mello J. Affect regulation and HIV risk 
among youth in therapeutic schools. AIDS Behav. 2012;doi:10.1007/s10461-012-0220-3 
123. 
Houck C, Hadley W, Barker D, Brown L, Hancock E, Almy B. An emotion regulation intervention to 
reduce risk behaviors among at-risk early adolescents. under review; 

 
Version 1.2  9/27/22                                                                              27 
 
124. 
Valois RF, Zullig KJ. Psychometrics of a brief measure of emotional self-efficacy among adolescents 
from the United States. The Journal of school health. 2013;83(10):704-711.  
125. 
Michel B. The association between children's beliefs about emotions and symptoms of depression. 
Cambridge, MA: Harvard University; 2017. 
126. 
Dweck C, Chiu Cy, Hong Y-y. Implicit theories and their role in judgements and reactions: a world from 
two perspectives. Psychological Inquiry. 1995a;6(4):267-285. doi:doi:10.1207/s14327955pli0604_1 
127. 
Tamir M, John O, Silvastava S, Gross J. Implicit theories of emotion: Affective and social outcomes 
across a major life transition. J Personality Social Psychol. 2007;92(4):731-744. doi:doi:10.1037/0022-
3514.92.4.731 
128. 
Cummings JR, Bornovalova MA, Ojanen T, Hunt E, MacPherson L, Lejuez C. Time doesn’t change 
everything: The longitudinal course of distress tolerance and its relationship with externalizing and internalizing 
symptoms during early adolescence. J Abnorm Child Psychol. 2013;41(5):735-748. 
129. 
Bancroft J, Janssen E, Strong D, Carnes L, Vukadinovic Z, Long J. Sexual risk-taking in gay men:  The 
relevance of sexual arousability, mood, and sensation seeking. Arch Sex Behav. 2003;32(6):555-572.  
130. 
Schloss H, Haaga D. Interrelating behavioral measures of distress tolerance with self-reported 
experiential avoidance. Journal of Rational Emotive Cognitive Behavioral Therapy. 2011;29:53-63.  
131. 
Moilanen K. The adolescent self-regulatory inventory: the development and validation of a 
questionnaire of short-term and long-term self-regulation. Journal of youth and adolescence. 2007;36:835-848.  
132. 
Carskadon M, Acebo C. A self-administered rating scale for pubertal development. J Adolesc Health. 
1993;14(3):190-195.  
133. 
Bird H, Shaffer D, Fisher P, et al. The olumbia impairment scale (CIS): pilot findings on a measure of 
global impairment for children and adolescents. Int J Methods in Psych Res. 1993;3:167-176.  
134. 
Buhi E, Goodson P. Predictors of adolescent sexual behavior and intention: A theory-guided systematic 
review. Journal of Adolescent Health. 2007;40:4-21.  
135. 
Cubbin C, Brindis CD, Jain S, Santelli J, Braveman P. Neighborhood poverty, aspirations and 
expectations, and initiation of sex. Journal of Adolescent Health. 2010;47(4):399-406.  
136. 
de Graaf H, van de Schoot R, Woertman L, Hawk ST, Meeus W. Family cohesion and romantic and 
sexual initiation: a three wave longitudinal study. Journal of youth and adolescence. 2012;41(5):583-592.  
137. 
Kerr JC, Valois RF, Siddiqi A, Vanable P, Carey MP. Neighborhood condition and geographic locale in 
assessing HIV/STI risk among African American adolescents. AIDS and Behavior. 2014:1-9.  
138. 
Markham CM, Lormand D, Gloppen KM, et al. Connectedness as a predictor of sexual and 
reproductive health outcomes for youth. Journal of Adolescent Health. 2010;46(3):S23-S41.  
139. 
Secor-Turner M, McMorris B, Sieving R, Bearinger LH. Life Experiences of Instability and Sexual Risk 
Behaviors Among High-Risk Adolescent Females. Perspectives on sexual and reproductive health. 
2013;45(2):101-107.  
140. 
Hadley W, Barker D, Thamotharan S, Houck C. Relationship between unsupervised time and 
participation in an emotion regulation intervention and risk outcomes. Journal of developmental and behavioral 
pediatrics : JDBP. 2017;38(9):714-722.  
141. 
Treboux D, Busch-Rossnagel N. Social network influences on adolescent sexual attitudes and 
behavior. J Adolesc Res. 1990;5:175-189.  
142. 
Coyle K, Kirby D, Robin L, Banspach S, Baumler E, Glassman J. All4You! A randomized trial of an HIV, 
other STDS, and pregnancy prevention intervention for alternative school students. AIDS Education and 
Prevention. 2006;18(3):187-203.  
143. 
Crum R, Lillie-Blanton M, Anthony J. Neighborhood environment and opportunity to use cocaine and 
other drugs in late childhood and early adolescence. Drug and Alcohol Dependence. 1996;43(3):155-161. 
doi:10.1016/s0376-8716(96)01298-7 
144.  Office of Disease Prevention and Health Promotion. Healthy People 2030. https://health.gov/healthypeo-
ple/objectives-and-data/data-sources-and-methods/data-sources/national-survey-family-growth-nsfg 
 
 

STATISTICAL DESIGN AND POWER 
 
Data preparation: Relevant tests of model assumptions will be evaluated with all analyses, and appropriate 
corrections for problems in distributions will be implemented. Because even small and statistically non-
significant between condition differences in baseline characteristics can adversely affect treatment effect 
estimates, we will use inverse probability of treatment weights (IPTW) to adjust for potential imbalance 
between conditions1,2. Covariate balancing propensity scores3 will be used to generate the weights and the 
model will include demographics and baseline measures of primary outcomes, secondary outcomes, 
exploratory outcomes, moderators, and mechanisms. The effectiveness of the IPTWs will be evaluated by 
looking at the average and largest effect sizes for between condition differences prior-to and following 
weighting. We will also examine the positivity or common support assumption required for IPTW by visually 
examining the overlap in propensity score distributions for each condition. This assumption is likely to hold in 
the context of a randomized trial.  
Missing data: Based on previous projects, we expect 10% attrition by the end of the 6-month follow-up. We 
will use fully conditional multiple imputation (a.k.a., multiple imputation using chained equations [MICE]4) to 
address bias introduced by missing data. MICE was selected because it can handle outcomes with different 
statistical distributions (i.e., binomial, Poisson, normal), it can be applied to a large number of variables, and it 
has solid performance in simulation studies4. This approach assumes data are missing at random (MAR).  
Primary Outcome:  The primary outcome of the RCT will be Self-efficacy for Sexual Risk Prevention at the 6-
month follow-up. All participants who were randomized will be included in all analyses (i.e., intent-to-treat). This 
outcome will be assessed using an analysis of covariance with baseline included as a covariate.  
Secondary Outcomes: Secondary outcomes include sexual health knowledge, abstinence attitudes, sexual 
behaviors, and condomless sex acts. Sexual health knowledge will be analyzed like the primary outcome. 
Sexual behaviors will be summed across the follow-up assessments. Multiple imputation will be used to 
address missing assessments and the summation will occur within each imputation to ensure that missing 
assessment are appropriately counted. These data are expected to be zero-altered and will thus be analyzed 
using generalized linear models with a negative binomial distribution and a log link function. Condomless sex 
acts will be analyzed like sexual behaviors.   
Mechanisms: Measures of mechanisms include theoretically important emotional competencies (e.g., emotion 
regulation, emotion recognition, distress tolerance) that mediate risk as measured by self-report, performance 
measures, and parent report. These outcomes will be assessed at the 2-month and 6-month follow-ups using 
analysis of covariance like for the primary outcome. 
Power and Sample Size: This trial was powered to the continuously scaled outcomes (self-efficacy, 
knowledge, and emotional competencies). With 120 participants, this study is powered at .80 to detect 
moderate effect sizes (standardized mean difference d ≥ .47). The power analysis was run using the R 
package simstudy v0.2.1 and assumed a two tailed type-1 error rate of .05 and a correlation between baseline 
and the 6-month outcome of r=.6. Effect sizes from our previous work showed effect sizes ranging from .20 to 
.70.   
 
Exploratory analyses of mediation and treatment modifiers: Mediation: Causal mediation will be used to 
evaluate whether treatment-related change in self-efficacy was attributable to change in mechanisms. Causal 
mediation was selected because the framework can handle mediators and outcomes with non-normal 
distributions such as a binary outcome8. Mediation analyses will be run for each of the hypothesized 
mechanisms. Moderation: To better understand who is benefiting most from the intervention we will use causal 
random forests, a non-parametric data-mining procedure9. Although this principled exploratory approach 
cannot definitively define subgroups, it has better statistical properties than looking at individual moderators 
and will help suggest subgroups that differ in terms of their response to the intervention. Power was not 
calculated for these exploratory analyses.  
 
1. Austin, P. C. The use of propensity score methods with survival or time-to-event outcomes: reporting 
measures of effect similar to those used in randomized experiments. Stat. Med. 33, 1242–1258 (2014). 
2. Austin, P. C. & Stuart, E. A. Moving towards best practice when using inverse probability of treatment 
weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. 
Stat. Med. 34, 3661–3679 (2015). 

3. Imai, K. & Ratkovic, M. Covariate balancing propensity score. J. R. Stat. Soc. Ser. B Stat. Methodol. 76, 
243–263 (2014). 
4. Azur, M. J., Stuart, E. A., Frangakis, C. & Leaf, P. J. Multiple imputation by chained equations: what is it and 
how does it work?: Multiple imputation by chained equations. Int. J. Methods Psychiatr. Res. 20, 40–49 
(2011). 
5. Applied Survival Analysis. in Series in Probability and Statistics (eds. Hosmer, D. et al.) (Wiley, 2008). 
6. Grambsch, P. M. & Therneau, T. M. Proportional hazards tests and diagnostics based on weighted 
residuals. Biometrika 81, (1994). 
7. Johnson BT, S.-S. L. Interventions to reduce sexual risk for human immunodeficiency virus in adolescents: 
A meta-analysis of trials, 1985-2008. Arch. Pediatr. Adolesc. Med. 165, 77–84 (2011). 
8. VanderWeele, T. J. Mediation Analysis: A Practitioner’s Guide. Annu. Rev. Public Health 37, 17–32 (2016). 
9. Athey, S., Tibshirani, J. & Wager, S. Generalized random forests. Ann. Stat. 47, 1148–1178 (2019).