Delivering Patient-Centered Adolescent Preventative Care With Training and Technology
Phase: NA
Conditions: Adolescent Behavior
Sponsor: Seattle Children's Hospital

Official Title: Delivering Patient-Centered Adolescent Preventative Care with Training 
and Technology 
NCT: 02764190 
Unique Protocol ID: IHS-1402-10592 
Date: April 28, 2016 

 
 
 
Page 1 
1. 
Background and Rationale  
Adolescents have some of the highest rates of risk behaviors of all age groups and health 
behaviors developed in adolescence can persist into adulthood.  Health risk behaviors, such as 
alcohol and other drug use, smoking, unsafe sexual activity, and poor nutrition and physical 
inactivity are among the most common causes of adolescent illness and premature death.  
These behaviors carry significant risks for subsequent disease, disability, and healthcare 
burden.  Despite these risks, health risk screening in primary care is infrequently performed and 
results are rarely followed by targeted intervention. 
Studies have shown that electronic health risk screening is feasible and efficient in clinical 
practice, increases adolescents’ comfort with disclosure of behavior and encourages utilization 
of preventive health services.  Based on this evidence, electronic multi-risk behavior screening 
tools are being developed for adolescent care settings.  However, to reduce risk behaviors, 
screening needs to be linked to interventions.  In response to the need for screening-linked 
interventions, our study team has developed a web-based, electronic screening and 
Personalized Motivational Feedback tool which we refer to as the “Check Yourself” app.  Based 
on motivational interviewing, a technique to mobilize personal change, Check Yourself is 
designed to promote healthy choices for the multiple behaviors relevant to adolescents.  
Building on electronic health interventions, primary care providers (PCPs) can play an essential 
role in helping adolescents to make healthy behavior choices.  
Adolescents list PCPs among the first people with whom they would consider discussing risk 
behaviors, and report greater satisfaction with care when PCPs discuss these sensitive topics 
with them.  Thus, primary care visits present a key opportunity for improving the health of 
adolescents. However, to take advantage of this opportunity, health systems strategies are 
needed that can practically be implemented in the time-pressured environment of primary care.  
Emerging evidence suggests that the consistency of preventive counseling can be increased 
through provider training and the provision of screening tools; yet, we know very little about the 
quality of such counseling, and if it impacts outcomes that are important to adolescent patients 
themselves. 
In order to address these gaps, this proposal aims to test an interactive adolescent-centered 
PCP training (I-ACT) to increase PCPs’ ability to engage and empower adolescents to make 
healthy behavioral choices and to stimulate patient-provider discussions around health behavior 
choices. We will examine the impact of our training in the presence of Check Yourself, which 
systematizes screening, delivers motivational feedback directly to adolescents, and generates a 
targeted report to PCPs summarizing youth risk behaviors and recommendations for next steps 
in care.  
 
1.1. Purpose  
The purpose of this study is to evaluate the perceived impact and effectiveness of our system of 
interventions (I-ACT with Check Yourself) among a sample of adolescents aged 13-18 drawn 

 
 
 
Page 2 
from the targeted primary care clinics using a group who receives usual care as a comparison.  
This study has two aims: 
Aim 1. To evaluate I-ACT with Check Yourself on intermediary targets: improving 
adolescent motivation for health, increased provision of appropriate preventive counseling, 
and follow-up care. We will examine the impact of I-ACT with Check Yourself on adolescent 
self-report of motivation for health at baseline and 3, 6 and 12 months following the primary 
care visit. We will also examine the impact of I-ACT with Check Yourself on the adolescent 
self-report of receipt of PCP-delivered risk reduction counseling at one day following the 
primary care visit and follow-up care at 3, 6 and 12 months following the primary care visit.  
Aim 2. To evaluate the effects of I-ACT with Check Yourself on adolescent satisfaction with 
the provider visit, perception of patient-centeredness of care, and health risk behaviors. We 
will explore the impact of I-ACT with Check Yourself on health care satisfaction among 
adolescents and caregivers at one day following the primary care visit.  We will also explore 
the impact of I-ACT with Check Yourself on adolescent self-report of their perception of 
patient-centeredness of care at one day following the primary care visit.   We will examine 
the effect of I-ACT with Check Yourself on producing significantly lower rates of alcohol 
use, marijuana use, unprotected sexual activity, and depression compared to a usual care 
group at baseline and 3, 6, and 12 months after the primary care visit. We will also examine 
the effect of I-ACT with Check Yourself on producing significantly higher levels of physical 
activity, fruit and vegetable consumption, and seatbelt and helmet use when compared to a 
control group at baseline and 3, 6 and 12 months after the primary care visit. 
2. 
Criteria for Subject Selection 
2.1. Sample Size 
We plan to enroll approximately 300 adolescent (age 13-18) and parent dyads who have an 
appointment with a participating provider affiliated with one of six participating practices in the 
Puget Sound Pediatric Research Network (PSPRN).  We also plan to enroll approximately 30 
PSPRN providers (about 5 providers per clinic).   
2. 
Gender of Subjects 
This study does not have any gender-based restrictions.  However, given that women are more 
frequently primary caregivers and PCPs, we estimate the following gender distribution in our 
study population: 
Adolescents: 150 Females and 150 Males 
Caregivers: 240 Females and 60 Males 
PCPs: 18 Females and 12 Males 
 
 
3. 
Age of Subjects  
 

 
 
 
Page 3 
Participants will include adult caregivers and PCPs (over the age of 18 years) and adolescents 
(age 13-18).  As health risk behaviors typically begin in adolescence and persist into adulthood, 
we are targeting adolescents rather than children less than 13 years of age.   
 
4. 
Racial and Ethnic Distribution 
The proposed study population will include a wide cross section of adolescent patients and their 
caregivers, including many families of ethnoracial minority status. We will monitor diversity in 
our sample throughout the course of recruitment and adapt recruitment efforts as needed to 
assure a representative sample.  The Central Puget Sound geographic area, where our study 
population is situated, consists of the following approximate ethnic and racial breakdown:   
73% White  
5% African American 
12% Asian, Hawaiian or Other Pacific Islander  
1% American Indian or Alaska Native  
9% some other race (alone) or two or more races 
 
In addition, 9% of individuals in this geographic area are of Hispanic or Latino ethnic 
background.   
5. 
Inclusion Criteria 
Eligible caregivers will have a child 13 to 18 years old who has an appointment with a 
participating provider at a PSPRN clinic and will be able to understand English. 
Eligible adolescents will be 13 to 18 years old, have an appointment with a participating provider 
at a PSPRN clinic and will be able to understand English.   
Eligible PCPs will include providers (i.e., pediatricians, family practitioners, physician assistants 
and nurse practitioners) affiliated with a PSPRN clinic who are following a panel of adolescent 
patients and will be able to understand English.  
6. 
Exclusion Criteria 
Caregivers will be excluded from the study if they do not speak English; or if their child is not 
eligible or declines to participate in the study. 
Adolescents will be excluded from the study if they do not meet age requirements, do not have 
an appointment with a participating provider at a PSPRN clinic, lack the means to complete 
follow-up interviews (i.e., have neither telephone nor internet access), have a sibling who has 
been/is being enrolled in the study or have previously participated in our previous trial 
comparing Check Yourself to usual care, and/or are not able to understand English. Our 
experience in the Seattle region is that due to excellent English Language Learning school 
supports, inability to read or understand English is a barrier for <0.5% of adolescents.  Thus, we 
do not anticipate that language restrictions will significantly limit the demographic distribution of 
our participants.   

 
 
 
Page 4 
PCPs will be excluded from the study if they do not speak English, are not following a panel of 
adolescent patients, and are not affiliated with a participating PSPRN clinic.  
7. Vulnerable Subjects 
This study’s focus is on adolescent health care, and adolescents are study participants. We plan 
to enroll 300 adolescent subjects aged 13-18. Most will be minors. Study activities for minor 
adolescents will be no different from study activities for 18-year-old adolescents. The risk/benefit 
ratio is the same for all adolescent participants regardless of age.  Research staff will obtain oral 
consent from 18-year-old participants directly, without contacting a caregiver.  For younger 
adolescents, research staff will first obtain oral parental consent before obtaining oral assent. 
3. 
Methods and Procedures 
3.1. Study Design 
This study will evaluate the effect of I-ACT with Check Yourself on adolescent-perceived quality 
of care (i.e., satisfaction and perception of PCP communication), motivation for health, and 
health risk behaviors using a stepped wedge study design.  The study will take place in 6 
participating PSPRN affiliated clinics, where the system of interventions (I-ACT with Check 
Yourself) will be rolled-out sequentially to the clinics, chosen in random order, over spaced time 
intervals (see Figure 1).  We refer to pre-implementation of I-ACT with Check Yourself, as the 
control period and post-implementation as the intervention (i.e., case) period.  
Figure 1: Stepped Wedge Study Design and Timeline 
 
 
We anticipate that approximately 30% of youth approached will participate in the study.  Thus, 
we will invite approximately 1000 adolescent/caregiver dyads to participate in the study, in order 
to yield 300 adolescent/caregiver dyad participants in our total sample.  Upon enrollment, 
caregiver participants will be asked to complete a brief (10-minute) web-based questionnaire 
about their education/occupation, their adolescent’s health history, and an open-ended question 
about what they would like to get from PCPs for themselves and their adolescent.  Adolescents 
who enroll in the study will receive either electronic health screening alone (i.e., control group) if 

 
 
 
Page 5 
their clinic appointment occurs in the pre-I-ACT implementation period or Check Yourself (i.e., 
intervention group) if their clinic appointment occurs in the post-implementation period.   
 
At baseline (T1), adolescents will be asked to complete a web-based screening survey that 
includes questions about guideline-recommended health risk behavior screening areas, 
including substance use, nutrition, physical activity, sexual behavior, depressive symptoms, and 
safety.  The screener questions themselves are current standard of care.  Adolescents will also 
be asked to report their age, gender, race/ethnicity, interest in discussing health risks with the 
PCP, and self-efficacy.   
 
Adolescents in the control group will complete the web-based screening survey for research 
purposes, but neither they nor their PCP will receive feedback about their health behaviors.  
PCPs will screen and counsel the patient as they would normally do under usual care 
conditions, consistent with the current standard of care. 
 
During the intervention period, participating PCPs will receive adolescent-centered 
communication training, based on the tenets of motivational interviewing, to reinforce healthy 
behavior choices and to engage youth and parents to reduce risk behaviors as appropriate.  
PCPs will also receive three feedback reports (baseline, 2 months and 4 months) on 
adolescent-reported receipt of counseling on each of the target behaviors and adolescent-
reported visit satisfaction and follow-up care.  PCPs will not be asked to complete any study 
questionnaires; all of the outcome data will be gathered directly from adolescents and 
caregivers.  Outcome data for the three feedback reports will be given to providers in aggregate 
form and will be void of individual identifiers.    
 
Adolescents in the intervention group will receive full personalized motivational feedback 
following completion of the screening survey, both with web-based delivery.  Their PCPs will 
receive a summary report which will include specific information on the adolescent’s responses 
that will allow them to reinforce healthy behavior choices and implement further brief, in-person 
interventions for moderate to high risk adolescents.  The personalized feedback to the 
participant and provider is consistent with current standard of care.  However, the electronic 
method of delivery is unique to this project.   
 
In order to administer follow-up surveys, we will ask adolescents to provide an email address, 
cell phone number, or phone number.  One day (T2) following their PCP appointment, 
adolescent participants will be asked via email, text message or phone to complete a survey 
about their motivation for health, perception of PCP communication, and experiences.  Both 
adolescents and their caregivers will be asked about their satisfaction with treatment during the 
appointment with their PCP.  In addition, caregivers will be asked open-ended questions to 
assess the provider’s delivery of caregiver education regarding strategies to reduce health risks 
for their child during the visit.   
 
Three (T3), six (T4) and twelve (T5) months following their PCP appointment, adolescent 
participants will be asked via email, text message or phone to complete all assessments 

 
 
 
Page 6 
administered at T1 and T2 except the questionnaires related to delivery of preventive 
counseling, demographics, interest in discussing health risks with PCP, and self-efficacy.  
Additionally, adolescents will be asked about follow-up care they may have received.  At T2 and 
T3, adolescent participants will be asked about their general satisfaction with their provider and 
will be asked 3-4 follow-up questions to ascertain satisfaction with the Check Yourself app and 
study.  These surveys are not standard of care, because they are not collected as part of an 
appointment; however, the probability and magnitude of discomfort associated with the 
completion of these surveys is no greater than ordinarily encountered in daily life or during the 
performance of routine psychological examinations/tests or medical care.  
 
3.2. Measures by Construct and Time point 
Tables 1 and 2 describe the study measures by construct. 
 
Table 1: Measures used in the study, organized by construct and method.   
Measures and Constructs 
Time point 
Moderators  
Gender, age, race, household composition, school, truancy, support, goals, 
interest in discussing health risks with provider  
T1(A) 
Number of visits with provider  
T2(A) 
Health Self-Efficacy  
T1(A),T2 (A), T3(A), T4(A), 
T5(A) 
Caregiver Concerns  
Guidelines for Adolescent Preventative Services Caregiver Questionnaire: 
adolescent health risk behavior concerns.   
T1(C)  
Perception of provider engagement with the caregiver  
T2 (C) 
Intermediary Targets 
Health Motivation (Readiness to Change Rulers):  Self-report used to assess 
adolescent motivation for change among specific health behaviors. 
T1(A),T2 (A), T3(A), T4(A), 
T5(A) 
Adolescent Report of the Visit: adolescent-report of the PCPs’ delivery of risk 
behavior screening and counseling based on risk level.   
T2 (A) 
Interval Receipt of Care: adolescent self-report of extent of follow up care. 
T3 (A), T4(A), T5(A) 
Main Outcomes: Satisfaction & Perception of PCP Communication 

 
 
 
Page 7 
Satisfaction Questionnaire: self-report of satisfaction with care based on 
items adapted from the Consumer Assessment of Healthcare Providers 
and Systems (CAHPS) measure.  
T2(A/C)  
Adolescent Perception of Patient-Centeredness: adolescent self-report of 
how adolescents feel about visits with their primary caregiver. 
T2(A) 
Main Outcomes: Health Risk Dimensions* 
Alcohol Consumption and Marijuana Use: self-report based on items about 
typical quantity and frequency of drinking, smoking/using tobacco, and 
marijuana use (each calculated separately) over the past 30 days. 
T1(A), T3(A), T4(A), T5(A) 
Fruit and Vegetable Screener: self-report of fruit and vegetable consumption.  T1(A), T3(A), T4(A), T5(A) 
Physical Activity Screener: self-report of physical activity.  
T1(A), T3 (A), T4(A),T5(A) 
Sexual Risk: self-report of sexual activity, birth control, and condom use.   
T1(A), T3(A), T4(A), T5(A) 
Patient Health Questionnaire-9 (PHQ-9): self-report of DSM-V symptoms for 
depression. 
T1(A), T3(A), T4(A), T5(A) 
Safety: self-report to assess behaviors that contribute to unintentional 
injuries (e.g., seat belt use, helmet use). 
T1(A), T3(A), T4(A), T5(A) 
Key: 
T1 = Timepoint 1 (Baseline)    T4 = Timepoint 4 (6 month)     
 
T2 = Timepoint 2 (1-day)         T5 = Timepoint 5 (1 year)          
 
T3 = Timepoint 3 (3-month) 
 
A = Adolescent survey 
 
C = Caregiver survey  *See Table 2 for Individual Health Risk Outcomes and Summary Risk Variable 
 
Table 2: Summary Risk Variable and Individual Health Risk Outcomes 
TOPIC AREA 
Original 
variable 
Summary Health Risk 
Variable 
Dichotomized Individualized Health 
Risk Variable (Risk vs. no risk) 
EATING 
Sweetened 
beverage 
consumption 
0, 1, 2, 3+ 
# of sweetened beverages 
consumed in a typical day 
2-3+ vs 0-1 
Fruit and 
vegetable 
consumption 
0-5+ 
 # of fruits and vegetables 
consumed in a typical day 
0-3 vs. 4+/day   
ACTIVITIES 

 
 
 
Page 8 
Physical activity 
(60+ minutes) 
0-7 
# of days with >60 minutes 
of physical activity in an 
average week 
0-3 days/week vs.  
4-7 days/week  
Sleep 
0-12+ 
# of hours of sleep on a 
typical night 
0-7 hours vs. 8+ hours 
SAFETY 
Seatbelt Use 
4 pt. scale 
Frequency of seatbelt use 
in a car  
Not always vs. always 
Helmet Use 
4 pt. scale 
Frequency of helmet use 
when bicycling  
Not always vs. always 
DUI/Riding with a 
DUI driver 
Dichotomous 
Yes vs. no 
Yes vs. no 
Texting while 
driving 
4 pt. scale 
Frequency of texting while 
driving (4-point scale) 
NOT INCLUDED IN SUMMARY 
RISK VARIABLE  
ALCOHOL AND DRUGS 
Drinking 
frequency 
0-30 
Decrease in # of days on 
continuous scale 
>1 (Ages 13-15)  vs. 0 
>2 (Ages 16-17) vs. 0-1 
>3 (Age 18) vs. 0-2 
Drinking quantity 
(typical or 
maximum) 
0-15+ 
# of alcoholic drinks 
typically consumed per 
occasion 
>3 (Girls 13-17 and Boys 13) vs. 1-2  
>4 (Girls 18 and Boys 14-15) vs. 0-3 
>5 (Boys 16-18) vs. 0-4 
Marijuana use 
frequency 
0-30 days 
and/or 
0-365 days 
#of days  used marijuana in 
the past 30 days and or 
year  
≥1 days/month (Ages 13-15)  vs. 0 
>2 days/month (Ages 16-18) vs. 1  
>1 days/year(Ages 13-15)  vs. 0 
>6 days/year (Ages 16-18) vs. 0-5 
 
 
SEXUAL ACTIVITY 

 
 
 
Page 9 
Condom Use 
4 point scale 
Frequency of condom use 
with sexual intercourse  
Not always vs. always  
(or not sexually active) 
Birth control use 
Dichotomous 
Use of any  birth control at 
last sexual intercourse 
(yes/no) 
Not always vs. always  
(or not sexually active) 
EMOTIONS 
Depression (PHQ-
9) 
0-27 
Score on PHQ-9 
Total score on PHQ 10 or higher vs. 
0-9 
 
 
3.3. Data Analysis and Data Monitoring 
 
3.3.1 Strategies to Minimize Missing Data.  
At the time of enrollment, we will gather the preferred and all available email addresses, 
phone numbers (including mobile for text messaging), and social networking inbox 
addresses for eligible adolescents. To confirm accuracy of the email address, participant 
payments will be sent electronically, via email, and participants will be instructed to contact 
study staff if payment is not received. We will also collect phone numbers for 3 friends or 
family members who would know how to reach the participant if we do not.  Our experience 
is that we can maintain >85% of our enrollees over the course of the study.  Despite our 
best efforts, we expect some attrition over the course of follow up, which has been factored 
into our sample size calculations.  In the case that an adolescent does not have an email 
address, surveys will be completed by phone and payments will be sent via mail.  
 
Maintaining statistical power is a key consideration. We will implement various 
recommended strategies for maximizing power within the current design, including use of 
covariates in data analyses when appropriate, preventing attrition and missing data, 
implementing advanced missing data analyses such as multiple imputation techniques 
using reliable measures, and maintaining integrity of the intervention throughout the study. 
For all analyses, we will start with models using all available data, not just complete cases. 
In addition we will apply inverse probability weighted methods to adjust for missing data 
and conduct sensitivity analyses looking at how other estimates are impacted by including 
the variables with missing values. 
 
3.3.2 Data Analysis.   
Data from the web-based surveys will be uploaded into STATA analytic software.  Prior to 
analysis, all variables will be checked for validity, missingness, and distributional 
assumptions.  Preliminary analyses required for variable construction, any required 
imputation of missing data, assessment of psychometric properties of scale scores, and 
assessment of the validity of study variables will be completed before analyses assessing 

 
 
 
Page 10 
intervention effects. Data set documentation will then be developed and distributed to the 
study team. Before carrying out analyses addressing each of the research questions, an 
analysis plan will be formally presented to the study team and Project Advisory Board so a 
wide range of expertise and experience is brought to bear on each work product of the 
team. 
 
Some key features of the stepped wedge design require special consideration in the 
planned analyses.  First, the design involves unidirectional cross-over from the control arm 
to the intervention arm for all PCPs.  This has to be captured by an indicator specific to 
PCP and time point. Second, there are natural hierarchical structures in the data with PCPs 
nested within clinics, patients nested within PCPs, and repeated assessments made within 
patients for some outcomes, such as health risk behaviors.  Because different patient 
samples will be surveyed at each time point, within patient correlation over time should not 
be strong.  Still, clustering at the clinic and PCP-levels needs to be examined in all 
analyses.  Third, secular time trends need to be carefully examined and separated from the 
intervention effect.  Based on these considerations, we will focus on model- based 
approaches (linear or generalized linear mixed effects models) for the main analyses, in 
addition to standard hypothesis testing techniques. 
 
Aim 1 will compare intermediary targets at all clinics before and after the intervention using 
an analytic approach that both accounts for design features, repeated assessments within 
adolescents, time trends in these target behaviors during study period and adjust for 
potential confounders. For this aim we will use patients as the units of analysis. We will 
consider outcomes at the level of the patient (adolescent reported motivation for health and 
provision of appropriate preventive counseling).  
 
 
Our primary analysis will be based on a set of mixed effects regression models. As an 
illustration, let’s consider the adolescent self-reported health motivation (readiness to 
change, as other adolescent reported outcomes can be modelled in a similar fashion.  
Specifically, we will consider the following regression model: 
Yijk =  +  Xijk+ k Tk + γk Xijk* Tk  + bi + bij +φ Covariates + ijk, 
 
in which Yijk denotes the health motivation reported by jth patient within ith provider at kth 
time interval, Xijk is a binary indicator of exposure to the intervention for the provider 
and the predictor of interest, Tk is a binary indicator of the kth assessment, bi and bij are 
PCP and adolescent specific random effects that account for the within PCP and within 
adolescent correlations, and ijk is the error term. Additional patient-level or PCP-level 
confounding variables can be adjusted in the model as well.  
 
 
With this model, the fixed effects α captures the treatment effects on mean responses 
and time-by-intervention interaction coefficients γk’s captures the treatment effects across 
time. Fixed effects k ‘s aim to capture any temporal trend or learning effects. Using this 
approach, both within-adolescents and between adolescents information will be used to 
estimate the treatment effects. All coefficients will be tested using Wald test. Also note this 

 
 
 
Page 11 
model framework is flexible enough that a patient can be enrolled at any time during the 
study and the model can accommodate different numbers of post-intervention 
assessments. For example, for an adolescent enrolled after his/her PCP has received 
intervention, all Xijk’s would take value 1 for that subject.         
 
We will apply linear mixed effects models for Gaussian outcome Yijk,  and generalized 
linear mixed effects models for other types of outcomes (e.g., logistic model for binary 
outcome). 
 
For Aim 2, analyses will be based on patient survey responses to the 8-item satisfaction 
scale, patient perception of PCP communication, and the summary health risk variable, 
which will all be treated as continuous response variables.  The summary health risk 
variable constructed from a count of dichotomized key risk behaviors (as shown in Table 2).  
First, satisfaction levels and risk behaviors will be summarized for each period (pre- and 
post-intervention) and will be compared using two sample t-tests.  For the main analysis, 
we will apply mixed effects models similar to those proposed in Aim 1 to examine the 
intervention effect. For patient satisfaction, PCP satisfaction, adolescent’s perception of 
PCP communication and health risk behaviors, the analysis will be cross-sectional 
comparisons at one day following the primary care visit. Let Yij denotes the satisfaction at 
one-day post visit of adolescent j within PCP I, let Xij denote whether the PCP has received 
intervention at the time of visit, then the following mixed effects model can be applied 
   
Yij =  +  Xij + bi +φ Covariates + ij, 
 
where bi accounts for within-PCP correlations for adolescents seen by the same PCP. The 
fixed effect α captures the intervention effect on the patient satisfaction. Similar model can 
be applied to PCP outcomes and we can account for within-clinic correlations as well. 
 
When examining the intervention effects on adolescents individual risky behaviors (alcohol 
use, marijuana use, unprotected sexual activity, depression, physical activity, healthy food 
consumption, seatbelt and helmet use) at baseline, 3, 6, and 12 months after the primary 
care visit, same modeling framework as outlined for Aim 1 can be applied. 
 
3.3.3 Heterogeneity of Treatment Effects. 
The goal of any heterogeneity of treatment effects analyses will be to ascertain whether 
treatment effects vary significantly across subgroups of patients and to identify subgroups 
of patients that achieve the greatest benefit from part or entire treatment regimen. (PCORI 
Methodology Standard HT-1) Based on our current knowledge, we hypothesize that 
treatment effects may vary substantially across age groups (13-15 yrs vs. 16-19 yrs), 
gender groups (male vs. female), racial groups (White, African American, Asian/American 
Indian/Pacific Islander, and multi-racial), ethnic groups (Hispanic vs. non-Hispanic), 
adolescents with differential baseline motivation status (highly motivated adolescents vs. 
less motivated), and adolescents with differential self-efficacy (high vs. low) (PCORI 
Methodology Standard HT-2). We recognize the possibility that additional subgroups may 

 
 
 
Page 12 
arise in the course of the analyses described above as we identify which factors modify the 
associations. (PCORI Methodology Standard HT-4) Our stepped wedge design allows us to 
employ both within participants and between participants information to estimate and 
compare treatment effects.  
As we have outlined in the Statistical Analysis section, the primary analysis will be based 
on mixed-effects regression models in the following form, for example, when examining 
PCP outcomes (similar models will be applied to adolescent outcomes): 
Yijk = µ+ αXijk+ βk Tk + bi + bij + γcovariates + εijk, 
in which Yijk  denotes the appropriate preventive counseling rate for jth PCP within ith clinic  
at time interval k, Xijk is a binary indicator of exposure to the intervention (predictor of 
interest), Tk is a binary indicator of the kth assessment time interval, bi and bij are clinic 
and PCP specific random effects that account for the within clinic and within PCP 
correlations, εijk is the error term. Adjustment for additional clinic-level or PCP-level 
confounding variables can be incorporated into the model as well. With this model, the fixed 
effect α captures the treatment effect, while fixed effects βk’s aim to capture any temporal 
trend. Using this approach, both within-PCP and between PCP information will be used to 
estimate the treatment effect.  
Our treatment heterogeneity (TH) analysis will consist of two steps. The first step will be 
subgroup analysis, in which the above outlined regression models will be applied to each of 
the subgroups. We will then inspect the treatment effect sizes and significances across 
subgroup analyses to get a sense of the direction and magnitude of treatment 
heterogeneity. In second step we will formally test the treatment heterogeneity by including 
a treatment-by-group interaction term in the model: 
    Yijk = µ+ α1Xijk+α2Group+α3 Xijk*Group+βk Tk + bi + bij + γcovariates + εijk, 
A wald t-test on the coefficient α3 formally tests the significance of treatment heterogeneity 
across groups. (PCORI Methodology Standard HT-3) We may have to apply multiple 
comparisons adjustments to minimize false discovery rates due to large number of 
subgroup analyses.  
Treatment heterogeneity can also be interpreted as effect modification, i.e., the treatment 
effects are being modified by group characteristics. However, such modifications may work 
through different mechanisms. For example, same intervention may be received 
differentially by different groups with one group embrace every aspect of the intervention 
and the other group does not comply at all. Or, different groups may respond differentially 
to the different treatment components, with one group respond better to one part of the 
intervention and another group to another part of intervention. Therefore, when examining 
treatment heterogeneity, we will have to apply careful scientific reasoning and scrutinizing, 
in addition to statistical analysis. 
 
4.0 Data Monitoring and Management  

 
 
 
Page 13 
 
The proposed research involves no greater than minimal risk to participants.   
 
4.1 Adverse Events 
Project staff will be trained to identify potential adverse events and instructed to report them 
immediately to Dr. Richardson or Dr. McCarty.  Should any study participant or caregiver 
express concerns about the study or their participation or appear distressed during any study 
activities, the witnessing research team member will bring the matter to the attention of Dr. 
Richardson or McCarty, who will distinguish serious adverse events from non-serious adverse 
events.  Serious adverse events will be documented and discussed with the IRB as soon as 
possible, and reported to the IRB within 48 hours.  An annual report will be submitted to PCORI 
and the IRB summarizing all adverse events. 
 
4.2 Data Quality Assurance and Confidentiality  
Data quality assurance will be monitored on an ongoing basis by the Research Manager, 
Heather Spielvogle.  She will each conduct routine protocol compliance checks to ensure that 
safety procedures, such as ensuring participant confidentiality and maintaining approved 
standards for data transport, are strictly adhered to at each site.  All study data will be stored in 
password-protected computer files and identified with study IDs.  Analytic data files will contain 
no identifying information.  Other confidentiality issues are discussed in the prior section 
“Protection Against Risks.” 
 
4.3 HIPAA Compliance   
All participants will be recruited from clinical sites, and as such their participation is subject to 
the Health Insurance Portability and Accountability Act of 1996 (HIPAA) privacy and security 
standards.  All relevant staff has completed required HIPAA training and all research activities 
will be conducted in compliance with HIPAA standards.  All study data will be securely stored 
and labeled as discussed on the “Data storage and confidentiality” section of this protocol, 
below.  PCPs of participants assigned to the Check Yourself intervention will receive feedback 
including specific information on adolescents’ responses that will allow them to reinforce healthy 
behavior choices and implement further brief in-person interventions for moderate to high risk 
adolescents.  Although a participant’s individual responses to the health screener constitute 
protected health information generated from the research study, since the results are shared 
with the participant’s PCP for treatment and/or health monitoring purposes, this disclosure 
appears to fall within an exception of the minimally necessary requirement of the Privacy Rule 
(Section 164.502).  PCPs will receive three feedback reports on adolescent-reported receipt of 
counseling on each of the target behaviors and adolescent-reported visit satisfaction and follow-
up care gathered directly from adolescents and caregivers.  Although the data from the 
feedback reports will come from adolescent and caregiver participants, the data will be 
distributed in aggregate form and will be void of individual identifiers.    
 
 
 
4.4 Data storage and confidentiality 

 
 
 
Page 14 
One month prior the adolescent’s scheduled appointment with their PCP, participating PSPRN 
clinics will generate reports with potential participants’ contact information from electronic 
scheduling tools which will be provided to study staff.  We will request a full waiver of 
authorization under HIPAA to obtain contact information for individuals who have scheduled an 
appointment at the participating clinics.  We will collect the minimum amount of information 
necessary to identify potentially eligible participants from the participating clinics’ electronic 
scheduling tools.  Data used include: caregiver/guardian name, adolescent name, age, date of 
appointment with PCP, address, and telephone number.  Contact and identifying information will 
be used solely for study recruitment purposes and will only be accessible to study staff with a 
direct need to access this information (e.g., research staff mailing letters and conducting phone 
screening).  We will assign a unique study number to individuals for recruitment purposes.  This 
study number is then used in study files, rather than subjects’ names.  We will then maintain 
and protect a linking file which links study number to participants’ names and other identifying 
information including participants’ e-mail addresses and cell phone numbers.  This linking file 
will be stored in Seattle Children’s Research Institute (SCRI) on a secure database on the SCRI 
network.  Access to the SCRI network is controlled by valid, networked user accounts which 
include study researchers and staff.  Identifying information used to recruit participants will be 
destroyed within 5 years of the end of the study.  SCRI is responsible for storage of data 
collected from participants.  Study data for enrolled participants will be identified by study IDs 
and will also be retained in a SCRI secure database on the SCRI network.  Analytic data files 
will contain no identifying information.  
 
Subject demographic data will be recorded in a password-protected database.  Completed 
questionnaire data will be collected through DatStat Illume, a platform used to develop secure, 
web-based surveys, and then exported from DatStat Illume into SPSS, Stata/SE, and SAS for 
analysis purposes.  Data are stored on a secure SQL server, and are available to study 
investigators for queries, reports, and download for analysis. 
 
Web-based surveys will be accessed by logging on to a secure server with security and data 
integrity violations obviated by requiring participants to log in with a password unrelated to any 
identifying information on any on-line page or database.  All information transferred between 
client and server machines will be secured using 128-bit encrypted Secure Sockets Layer.  All 
data stored in the online repository will be encrypted using the official Advanced Encryption 
Standard algorithm.  Protocols have been informed by prior internet-based studies conducted by 
study investigators. 
 
For participants enrolled in the post-implementation period, their baseline report will be printed 
by study or participating clinic staff and given to the participant’s PCP prior to their appointment.  
Participant reports will be shared with PSPRN staff via a protected, designated fax machine 
available only to clinic staff and providers or through a secure server with security and data 
integrity violations obviated by requiring them to log in with a password unrelated to any 
identifying information related to the research participant. All information transferred between 
client and server machines will be secured using 128-bit encrypted Secure Sockets Layer. All 
data stored in the online repository will be encrypted using the official Advanced Encryption 

 
 
 
Page 15 
Standard algorithm.  Protocols have been informed by prior internet-based studies conducted by 
study investigators.   
 
For purposes of analysis and manuscript preparation, paper data files will be maintained for up 
to 3 years after the project ends, after which all paper data will be disposed of via shredding. 
Secure electronic data files will be kept for up to 7 years after the project ends on SCRI’s secure 
network. 
 
5. 
Risk/Benefit Assessment 
 
5.1. Risk Category  
This is a minimal risk study, with the probability and magnitude of discomfort no greater than 
ordinarily encountered in daily life or during the performance of routine psychological 
examinations/tests or medical care.  We will request a Full Waiver of Authorization under HIPAA 
for waiver of documentation of consent and waiver of the requirement for signature on the 
authorization for use and disclosure of health information.  
 
5.2. Potential Risks and Protection Against Them 
5.2.1 Risks. 
We foresee two potential risks to study participation.  The first risk is that we may discover 
a harmful behavior requiring urgent intervention such as suicidality.  Youth with concerns 
for suicidal ideation on the PHQ-9 (reporting “thoughts of being better off dead or of hurting 
yourself in some way” several days or more) will be asked two additional questions from 
the Teen Screen (www.teenscreen.org) version of this questionnaire asking if: a) in the past 
month the youth has had “serious thoughts about ending your life” and b) they have ever 
made a suicide attempt.  Further details on addressing suicidality are provided below.  The 
second risk to participants is that the information provided on the Check Yourself app may 
not remain confidential.  The PCP may choose to put risk behavior information in the 
youth’s electronic medical record.  It is also possible that youth may leave their computer 
screen with information from the Check Yourself app open in their web-browser and 
someone else might see it.  Although we will emphasize the importance of completing the 
survey in a private setting, participants may still complete the Check Yourself app in an 
open setting (i.e., computer in the family room) and other family members might observe 
their responses.  
 
5.2.2 Protections against Risks. 
All research activities will be reviewed and approved by Seattle Children’s IRB as well as 
clinic-specific IRBs (when required) to ensure participants are adequately protected against 
risk and all research activities are HIPAA compliant.  Participants will be fully informed of 
the potential risks of participation, alternate treatment options, and their right to discontinue 
study participation at any time.  Specific steps we will take to reduce known risks are 
described below: 
 
5.2.2.1 Confidentiality and Protections. 

 
 
 
Page 16 
As part of the consent procedures, adolescents and caregivers will be informed that 
the adolescent should complete the assessment privately and adolescents will be 
instructed to exit the web browser at the completion of the survey.  Data collected 
from the web-based screening survey at baseline may be shared with treating PCPs.  
Data collected during all follow-up assessments, including receipt of counseling on 
each of the targeted behaviors, visit satisfaction, and receipt of follow-up will be given 
to providers in aggregate form, segregated by clinic and will be void of individual 
identifiers.  All other data will be held confidential and will not be shared.  At each 
contact, study materials will clarify the nature of data collected and whether data will 
be shared with treating PCPs.  Study materials will also outline situations in which 
results may need to be shared with caregivers (i.e. suicidal thoughts).  All materials 
included in the chart by PCPs will be considered part of confidential patient data and 
will be bound by customary restrictions on access to patient records.  For example, 
Washington State law requires consent of the youth to release information regarding 
their mental health, sexual health, and alcohol and drug use.  It should be noted that 
screening of health risks including depression, sexual activity, and alcohol use is a 
routine and recommended aspect of adolescent clinical care.  All study data will be 
securely stored in either password-protected computer files or in locked file cabinets 
and identified with study IDs (see section 4.4 for specific data security measures).  
 
5.2.2.2 Management of suicidality. 
Regardless of intervention status, participants with a positive response on the items 
assessing suicidality will be flagged for further evaluation during their appointment 
with the PCP.  Additionally, research staff will track all flagged youth to ensure that an 
assessment is completed by one of the study clinicians.  A study clinician (Dr. 
Richardson, a board certified Adolescent Medicine Provider, Dr. Spielvogle, a 
licensed mental health counselor, or Dr. McCarty, a licensed clinical psychologist) will 
call to evaluate any youth with a positive response on items assessing suicidality 
using a protocol that we have employed in our prior depression studies and will assist 
in connecting the youth with care based on the level of assessed risk.  The 
assessment consists of a semi-structured interview, including questions about 
pervasiveness of thoughts, impulsivity, presence of a plan and current supports, with 
guidance for assessing risk.  All youth judged to be at higher than minimal risk will be 
encouraged to seek care and assisted with identifying an appropriate resource (PCP, 
mental health specialty care, emergency services) based on severity.  Youth who are 
found to be actively suicidal will be assisted with receiving resources within 24 hours.  
Youth who are at low risk will be assisted with connecting with the PCP within 1-2 
weeks.  For youth <18 years old who are found to be at higher than minimal risk, the 
investigator will also speak with a parent/guardian to share recommendations and 
offer assistance in accessing care.  Consent procedures will make clear the situations 
in which a caregiver would be notified using standard clinical language regarding 
danger to self or others. This protocol has functioned smoothly in each of our prior 
studies, and youth and caregivers have expressed gratitude for the information 
provided and for additional assistance in receiving care. 

 
 
 
Page 17 
 
5.3. Potential Benefits 
5.3.1 Individual Benefit (other than remuneration). 
All participants may receive some satisfaction or indirect benefit from contributing to this 
research.  PCPs may benefit from the training they will receive on adolescent-centered 
preventive counseling, which may increase their motivation, self-efficacy, and skills to 
address challenging behaviors and provide MOC 4 and CME credits. Adolescents may 
benefit from the additional training provided to their PCPs in addressing preventive and 
high-risk health behaviors.  Post-implementation youth participants will receive 
personalized feedback aimed at increasing their readiness for the visit and motivation for 
healthy behaviors, which may have a positive impact on their health. 
 
5.3.2 Societal Benefit. 
Potentially preventable health-compromising behaviors are among the leading causes of 
morbidity and mortality in the adolescent age group.  Screening and preventive 
interventions are recommended by multiple professional organizations, but are often not 
performed.  This project takes an innovative strategy of training PCPs to deliver preventive 
counseling in adolescent-centered ways, testing this approach in tandem with a tool that 
provides personalized feedback for adolescents and decision support for PCPs.  
 
This project will be an important contribution towards developing intervention strategies that 
are adolescent-centered and can be broadly disseminated to reduce health compromising 
behaviors in primary care settings.  Society and healthcare settings will benefit from the 
knowledge gleaned regarding these innovative tools to enhance adolescent preventive 
counseling quality and delivery.  In this regard, the minor risks incurred are outweighed by 
the anticipated benefits. 
 
 
5.4. Alternatives to Participation 
Participation is voluntary and discontinuation is an option at any time during the study.  
 
6. 
Recruitment and Consent 
 
6.1. Recruitment Method  
 
We plan to recruit approximately 5 providers from each of the participating clinics using the 
following methods (a total of 30 providers). First, we will develop a study overview sheet 
outlining the I-ACT intervention and study protocols and expectations. We will present 
information on the study with this study sheet at a provider meeting in each clinic. We will ask 
providers to indicate their interest in the study and will set up appointments to review and obtain 
consent. 
 
Staff from Seattle Children’s Research Institute (SCRI) will recruit and enroll adolescent and 
caregiver participants through a rolling recruitment process, organized by clinic.  

 
 
 
Page 18 
 
Potential participants will be identified up to six weeks before their scheduled appointment via a 
data pull from the clinic scheduling software.  Research staff will mail an Introductory Packet to 
the youth’s home containing: 
 Introductory letter signed by the clinic with an opt-out phone number 
 Introductory flyer 
 Study Information sheet  
 
We will request a full waiver of authorization under HIPAA to obtain contact information for 
potential participants.  Our procedure will be: 
1. Contact the caregiver by phone  
2. Screen for eligibility, explain the study, and answer questions.  If the caregiver wishes to 
participate, study staff will arrange a meeting with the caregiver and adolescent via 
telephone to review the study information sheet and obtain oral consent/assent.  
 
“Adult adolescent” (i.e. 18 year old) recruitment: “Adult adolescents” will be approached before 
their caregiver.  Study staff will attempt to contact the “adult adolescent” by phone.  Once the 
“adult adolescent” is contacted, study staff will explain the study, give detailed information, and 
answer questions.  If the “adult adolescent” wishes to participate, staff will arrange a meeting 
with the adult adolescent via telephone to obtain oral consent.  Then the caregiver will be 
approached and asked to participate in the parent survey portion of the study if the teen 
consents to us contacting and inviting the parent. “Adult adolescents” will not be excluded from 
the study if their parents do not participate. 
 
6.2. Consent and Assent   
We are requesting a waiver of documentation of consent/assent and waiver of Authorization 
under HIPAA.  This research study poses no more than minimal risk to research participants 
and the screening questions are standard of care.  We are requesting this waiver because, 
whenever possible, we intend to deliver the online screening survey and follow-up visits via the 
internet, rather than in-person.  Thus it would not be practicable to obtain written parental 
consent and assent if study visits are completed online.   
 
 
 
6.2.1 PCP oral consent will be obtained  
Oral consent will be obtained prior to study initiation. The study information sheet will 
include the required elements of authorization under HIPAA. 
 
6.2.1 Parent oral consent and permission and oral assent. 
Oral parent consent and permission will be obtained over the telephone, prior to the youth’s 
appointment at a mutually agreed upon time.  The study information sheet will include the 
required elements of authorization under HIPAA. 
 
 
6.2.2 Minor adolescent oral assent.  

 
 
 
Page 19 
Study staff will reach the minor adolescent by phone to explain the study.  Assent will be 
obtained in conjunction with the oral parental consent and permission. 
 
6.3.1“Adult adolescent” oral consent will be obtained via telephone.  
“Adult adolescents” may participate in the study even if their parent does not participate. 
 
6.3. Subject Capability  
All potential subjects will have the capacity to give consent. 
 
6.4. Subject Comprehension  
Interested caregivers and PCPs will have the opportunity to discuss the study with SCRI staff 
prior to enrollment.  Throughout this conversation, SCRI staff will assess parental and PCP 
understanding of the project and of participation expectations by asking parents and PCPs to 
use their own words to describe the project and their role in it.  Efforts to clarify and simplify the 
research and parent’s/PCP’s role will be prioritized.  Additionally, staff will provide reminders 
that parents and PCPs retain the right to withdraw from the study at any time.  SCRI staff will 
invite questions from the parents and PCPs, reiterate their right to refuse participation, and ask 
whether they are comfortable with participation. 
 
Interested adolescents will also have the opportunity to discuss the study with SCRI staff prior to 
enrollment.  Throughout this second conversation, SCRI staff will assess teens’ understanding 
of the project and of participation expectations by asking them to describe the project in their 
own words.  Staff will remind youth that they can withdraw from the study at any time without 
adverse consequences.  SCRI staff will encourage questions, reiterate the right to refuse 
participation, and ask whether they are interested in participating. 
 
6.5. Consent Forms 
Consent forms for parents and adolescents will be included with the submission to the SCRI 
IRB.   
 
6.6. Documentation of Consent 
Caregivers, adult adolescents, and PCPs may consent orally and adolescents may provide oral 
assent.  Documentation of oral consent and assent will be noted by research staff and kept in 
study files.  
 
6.7. Costs to Subjects 
There are no study-related costs for any participants. 
 
6.8. Payment for participation 
Adolescents will be given $20 for completion of the baseline screener and $20 for each follow-
up assessment (up to $100). Caregivers will be given $10 for each completed brief caregiver 
survey ($10 baseline + $10 1-day follow up =$20 total). 
 
6.9. HIPAA 

 
 
 
Page 20 
The SCRI IRB requires that research participants provide authorization to use their protected 
health information in connection with research.  We have identified the information that will be 
collected from participating clinics.  We will provide participant protected health information to 
PCPs including specific information on the adolescent’s responses, which will allow PCPs to 
reinforce healthy behavior choices and implement further brief in-person interventions for 
moderate to high risk adolescents.  Although a participant’s responses to the health screening 
survey constitute protected health information generated from the research study, since the 
results are shared with the participant’s PCP for treatment and/or health monitoring purposes, 
this disclosure appears to fall within an exception of the minimally necessary requirement of the 
Privacy Rule (Section 164.502).  Thus, we will request a waiver of documentation of consent 
and waiver of the requirement of signature on the Authorization.  As detailed above in the data 
storage section, all information will be collected through fully secure sites and all participants are 
coded to prevent identification of individuals.