Improving Student Mental Health: Adaptive School-based Implementation of CBT
Phase: NA
Conditions: Depression, Anxiety
Sponsor: University of Michigan

 
Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-
based CBT 
NCT03541317 
Last Date of IRB Approval: October 21, 2020 
PI: Amy Kilbourne, PhD, MPH

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
2 
 
HUM00132239 
Objective 
The overarching goal of this study is to improve the delivery of an established, evidence-based 
intervention (cognitive-behavioral therapy-CBT) in Michigan schools through different 
implementation strategies designed to better educate school professionals. Specifically, the 
study will assist the ongoing Transforming Research into Action to Improve the Lives of 
Students (TRAILS) Program by evaluating different ways to educate school professionals (SPs) 
to improve their delivery of CBT to high school students and ultimately improve student mental 
health outcomes in the state of Michigan.  The three educational approaches are Replicating 
Effective Programs (REP), Coaching, and Facilitation. Through the TRAILS program, schools 
will be recruited to receive training and support through REP to help SPs deliver CBT for 
students with mood disorders. Through the TRAILS program, schools in need of additional 
assistance in delivering CBT will be subsequently randomized by TRAILS staff to receive 
additional coaching or coaching and facilitation for SPs. We are seeking IRB oversight to assess 
school administrators, School Professionals (SPs), and students involved in this study. SPs at 
participating schools will be required to deliver CBT as part of their employment and will be 
offered the opportunity to participate in these assessments, including collecting data from 
students they identify.  
 
Specific Aims/Hypotheses 
Depressive and anxiety disorders affect 20-30% of school age youth, contributing to poor 
developmental and academic outcomes, substance abuse, and adult psychopathology, with 
immense social and economic costs. Psychosocial evidence-based practices (EBPs), such as 
cognitive behavioral therapy (CBT), can improve outcomes in these disorders, but less than 
20% of youth in need access EBPs due to limited availability, stigma, and financial barriers. 
Further, even when CBT is delivered, most recipients do not get adequate doses of CBT core 
components. Providing CBT in schools, through existing school professionals (SPs), including 
school social workers, counselors and psychologists, could substantially improve student 
access to CBT. However, we do not yet know how to cultivate needed CBT expertise in SPs 
and how to foster successful uptake. Efficient and successful implementation of CBT in schools 
requires scientific determination of optimal implementation strategies that educate SPs in 
improved delivery of CBT in the context of school settings, while addressing individual, 
organizational, and community barriers to uptake.  
This project involves a novel application of three theory-based implementation strategies—
Replicating Effective Programs (REP), Coaching, and Facilitation. REP customizes content to 
local needs and uses didactic training and brief technical support to improve uptake of EBPs in 
community organizations. REP is efficacious and low burden, but has not been tested for 
school-based implementation of CBT. Its primary modality is didactic training, and considerable 
data show that in the absence of ongoing supervision, such training is less likely to lead to 
adoption of EBPs. Coaching is an implementation approach that addresses this clinician barrier 
by extending training via live supervision of CBT delivery. It shows promise in facilitating CBT 
adoption and fidelity in schools. Facilitation is an implementation approach that mitigates 
organizational, community, and leadership barriers by mentoring providers to become EBP 
champions at their sites when additional institutional support is needed. It has also been shown 
to enhance EBP uptake. Neither Coaching nor Facilitation has been tested to determine the 
most cost-effective way to implement CBT in schools. They both add cost and intensity but also 
are likely to enhance uptake, thus offsetting potential later expense and sustainability. 
Optimal CBT uptake in schools likely requires a “stepped up” type of adaptive implementation 
intervention, whereby more intensive and expensive implementation strategies (e.g., 
Facilitation) are only provided to SPs within schools that do not respond to a simpler approach. 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
3 
 
Our team has extensive experience with novel study designs such as sequential multiple 
assignment randomized trials (SMART) to answer critical questions related to adaptive 
implementation interventions (ADEPT, R01MH099898; PI: Kilbourne). Augmentation of REP 
with Coaching may be essential to overcome SP barriers. Facilitation may help when 
institutional barriers stymie uptake. Comparative research is needed to best combine these 
approaches to create and optimize an adaptive implementation intervention that maximizes 
uptake, cost-effectiveness and sustainability of an established EBP like CBT, to ultimately 
improve student mental health. 
 
This study responds to this need via an implementation trial designed to identify an optimal 
and most cost-efficient approach for adapting strategies to enhance CBT uptake and delivery by 
SPs within their own schools, and reduce impairing anxiety and depression among their 
students. This five-year project will pursue this goal via a SMART implementation trial, which 
has not previously been used in schools, across a network of high schools throughout Michigan, 
to address the following specific aims:  
 
Primary Study Aim:  
The Primary Aim of this study is to compare the effectiveness of an adaptive 
implementation intervention on CBT delivery among schools versus REP alone (the control). 
The adaptive intervention provides schools with REP + Coaching from the start and 
subsequently augments with Facilitation for schools needing additional assistance. The primary 
outcome is the number of CBT sessions delivered to students by SPs over a 15-month period. 
Specific CBT component delivery and whether delivered sessions were brief (<15 minutes) or 
full-length (≥15 minutes) will also be tracked and examined as secondary outcomes. As an 
exploratory outcome for this primary aim, we will also examine change in student mental health 
symptoms over the study period, among students identified by SPs (prior to randomization) to 
need CBT.  
 
Exploratory Aims: 
1. To estimate the costs of different implementation interventions and determine the incremental 
cost-effectiveness of added Coaching and/or Facilitation. 
2. To assess whether (a) the effect of augmenting REP with Coaching is moderated by school-
aggregated SP training or baseline perceptions of CBT, school size or percent free/reduced 
lunch eligible; and (b) amongst schools that show a potential need for further support, whether 
the effect of augmentation with Facilitation is moderated by school-level CBT delivery during 
first 8 weeks, number of barriers to CBT reported at 8 weeks, or school administrator support for 
adoption of innovation.  
3. To determine whether Coaching and Facilitation improve CBT knowledge, 
perceptions/comfort, skills, or championing skills among SPs, and which of these account for 
increases in frequency of CBT delivery and improvement in student clinical symptoms. 
 
Methodology 
Overview: This study is an evaluation of the adaptive implementation intervention involving 
educational strategies (REP, Coaching, and Facilitation) already deployed by the TRAILS 
program designed to improve frequency of CBT delivery to students by SPs and to reduce 
student mental health symptoms. This specific study protocol involves assessments of school 
administrators, school professionals, and students and will take place in high schools identified 
in the State of Michigan by the TRAILS program. 
 
Site study population: Site inclusion criteria include the following:  Interested schools that 
serve high school students (up to 100 out of Michigan’s 900) are identified as part of the 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
4 
 
ongoing TRAILS program. Schools are contacted by TRAILS staff regarding the TRAILS 
training program via a variety of methods including: (a) email to school administrators; (b) email 
to school mental health professionals (e.g., social workers, counselors); (c) presentations by 
TRAILS staff to the district or building; (d) phone or email response to inquiries submitted via the 
TRAILS website; (e) social media campaigns to increase TRAILS program awareness (e.g., 
Facebook). Eligible schools that wish to participate in the TRAILS program will then be given the 
opportunity to participate in this research study.  
Schools will be eligible if they: 
 
1. Are a school from a school district serving high school students (grades 9-12) in one of 
the 83 counties in Michigan 
2. Are within a 2-hour driving distance of a TRAILS coach (who are mental health 
professionals primarily working in community mental health clinics and currently being 
trained in all 83 counties) 
3. Agree to participate throughout the study duration 
4. Have at least 2 SPs who are eligible and agree to participate in study assessments 
throughout the study duration 
5. Have space for SPs to deliver individual and/or group mental health support services on 
school grounds, yet outside of the general education classroom environment 
 
The opportunity to participate in the study will be shared by a TRAILS staff person via email or 
phone using IRB-approved phone scripts or email text following their initial identification for 
involvement with the TRAILS training program. The phone scripts or email text sent to 
schools will include specific information about SPs being asked to submit academic data 
for up to 10 students via the web-based tool developed by the honest broker for use in 
the study. This data will be automatically de-identified upon entry into the web-based tool 
before it is sent to UM. If a school does not agree to have their SPs send this data, then 
the school can still enroll in the study but the SPs at that school will be informed upon 
consent that reporting of the academic indicators data is not required.   
 
Site Selection and representation: Up to 100 schools from across the state of Michigan will be 
selected by the TRAILS coordinator to participate in the study. Every attempt will be made to 
select at least one eligible school from each county. A diverse range of schools will be recruited 
to maximize generalizability of this study. Schools will vary in terms of geographic location (i.e. 
upper and lower peninsula, rural and urban) as well as overall demographics. 
 
Eligible SPs are full- or part-time SPs who are school social workers, counselors, or school 
psychologists, or have a similar background in counseling. All eligible SPs from participating 
schools will receive REP and training from the TRAILS program to deliver CBT in their schools. 
They will also be given the opportunity to participate in the study assessments, however this 
participation is voluntary and is not required to receive the TRAILS training in delivering school-
based CBT. School professionals will have a waiver of documentation of informed consent and 
will be consented electronically prior to receiving their first survey.  
 
SP Inclusion Criteria: 
1. Employed at a Michigan high school full or part time 
2. Have a background in clinical school social work, counseling, psychology, or similar area   
3. Able to read and understand English and comprehend study assessments 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
5 
 
 
SP Exclusion Criteria: 
1. Has a significant illness or condition that precludes their participation in the 
implementation strategies including the REP training and student identification process, 
Coaching, or Facilitation 
2. Unable to provide informed consent for participation in the study activities 
 
 
Study Design flow The study has four phases over 18 months coinciding with the different 
phases of offering the REP, Coaching, and Facilitation implementation interventions: (1) a Run-
In Phase: 3 months; (2) Phase 1: 2 months; (3) Phase 2: 10 months; and (4) Phase 3: 3 
months. In the Run-In Phase (October 2018-January 2019), all eligible SPs will receive 
Replicating Effective Programs (REP) components, including a 1-day didactic training in CBT in 
January of 2019. REP will be provided to all schools for the duration of the study.  
 
Randomization of Schools 
 
All randomization will be performed by TRAILS program staff as part of their statewide adaptive 
implementation process, and is not part of the research study protocol. In Phase 1 (mid-January 
to mid-March 2019), following the didactic training, schools will be randomized by the TRAILS 
program staff with equal probability to either continue REP or to REP+Coaching. At the end of 
Phase 1 (mid-March), schools will be assessed for whether they would potentially benefit from 
facilitation. In Phase 2 (March 2019 to January 2020), schools deemed unlikely to benefit from 
facilitation will continue with their Phase 1 strategy (either REP or REP+Coaching). Schools 
eligible for facilitation will be further randomized by the TRAILS Program staff with equal 
probability to continue their Phase 1 strategy (REP or REP+Coaching) or to have their current 
strategy augmented with Facilitation (REP+ Facilitation or REP+Coaching+Facilitation). Note 
that implementation strategies will be reduced during summer months (June-August 2019). In 
Phase 3 (February to April 2020), no implementation support is provided but outcomes are 
ascertained. Primary and exploratory outcomes (Table 1) will be collected longitudinally from 
SPs up to 18 months after the baseline assessment and from students up to 15 months after 
baseline assessment. We anticipate that 50% of schools (N=25) in the Phase 1 Coaching arm 
and 90% of schools (N=45) in the Phase 1 REP arm will be eligible for Phase 2 randomization.  
 
Randomization: All randomization occurs at the school level. All schools are randomized in 
Phase 1 with equal probability, and schools eligible for facilitation are again randomized with 
equal probability in Phase 2. The first randomization will be stratified based on school size, 
location of school (e.g., rural vs. urban), % of students on free/reduced lunch program (≥50%, 
<50%), and school-aggregated baseline SP CBT fidelity score.   
 
Evidence-based Practice to be implemented: Cognitive-behavioral Therapy (CBT) for 
Youth 
    The EBP to be implemented is a modular CBT program for youth with depression and 
anxiety, previously found in several studies to be associated with reduced depressive and 
anxiety symptoms when compared to usual care. Core components of the modular version of 
CBT include psycho-education, relaxation, instruction in identification and replacement of 
anxious or depressive thoughts, behavioral activation, creation of fear hierarchies, gradual 
exposure, and active intervention techniques, e.g., agenda setting, modeling of skills, active 
practice with feedback, and assignment of take-home practice activities. 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
6 
 
 
Implementation Strategies  
 
REP: REP includes a daylong didactic training covering core elements of CBT and proper 
screening and identification of students; training to help SPs identify eligible students; a 
package that includes tools to deploy CBT; and ongoing technical assistance in CBT 
implementation. All sites will receive REP materials starting in October 2018 one-day didactic 
trainings will be provided to SPs in early to mid-January 
Package: SPs will receive access to the CBT intervention package through the TRAILS 
website starting in October 2018. The package includes a CBT implementation manual and all 
materials needed to implement CBT including an overview of core components (e.g., cognitive 
restructuring, exposure), group session agendas, sample student screening forms, talking 
points for students, and associated resources. It also provides suggestions for school-based 
delivery.  
     Technical Assistance (TA):  TA will also begin in October 2018. TA consists of biweekly 
conference calls with an interactive website that provides additional resources (e.g., video, case 
simulations) and a forum for asking questions led by a CBT expert to address questions 
regarding clinical content, help with identifying students, manualized materials, and school-
based implementation; and will encourage SPs to organize student groups for CBT delivery. 
Student identification: Identification of appropriate students is a crucial first step for SPs in 
delivering CBT. Most schools lack universal screening for anxiety or depression and the inability 
of schools to appropriately identify students who would benefit from mental health services is a 
frequently cited barrier to successful implementation of EBPs. To address this barrier, REP 
(including pre-training technical support and printed information; provided to all schools) will 
provide specific information and materials to support student identification and the utilization of 
standardized tools to identify student symptoms. Furthermore, REP engages with the SPs in the 
building who are best-positioned to recognize signs and symptoms of mental health difficulties 
in students and to provide appropriate support. These include school social workers, 
Figure 1. Study Flow Diagram 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
7 
 
counselors, psychologists, and nurses, all of whom have established relationships with 
students; particular knowledge of social and emotional health; and familiarity with policies 
regarding student privacy and protections. Because these SPs will be trained to deliver CBT 
groups, training them to identify students who might benefit from CBT is essential. TRAILS pilot 
work has shown that with proper support and training, these SPs are able to accurately identify 
students and refer them to appropriate levels of care available. All SPs will be encouraged to 
identify at least 10 students that would benefit from CBT prior to attending the didactic training 
but student identification can occur up until 1 week following the training (see more info in the 
Student identification, recruitment, outcomes and measures section below). 
 
     Training: In January 2019, the TRAILS team will provide a 1-day training on the evidence 
behind CBT and a step-by-step walk-through of core components. Training will also cover 
common signs of depression and anxiety in students and utilization of public domain screens. 
Regional trainings delivered in early- to mid-January (prior to Phase 1 randomization) will 
enable simultaneous start-up for all participating SPs.  
      
Coaching: All SPs from schools receiving Coaching in Phase 1 will receive weekly visits from a 
CBT expert or “Coach”, for a minimum of 12 weeks. Coaches are recruited, trained and 
assigned to schools as part of their involvement with the TRAILS program. Coaches will meet 
with SPs before and/or after each session to review selected REP materials and address any 
concerns, questions, or challenges to delivery. Coaching consists of the following components 
provided on site in SPs’ schools:  
1. Weekly pre-session planning by phone or email, direction to appropriate materials and 
resources within REP, and role-play practice of specific treatment elements; 
2. In-vivo modeling of treatment skills during modular CBT group treatment sessions with 
referred students, observation of SPs’ treatment delivery, post-session discussion of 
strengths and areas for improvement, and practice of skills with feedback; 
3. Formal didactic instruction and guided practice of specific skills as needed.  
4. Follow-up phone calls and email consultations once adequate SP skill set has been 
observed and on-site visits are no longer necessary. 
 
Facilitation: A full-time Facilitator who is a member of the study team and has expertise in 
CBT, implementation methods, and use of EBPs in schools will support SPs in strategic thinking 
and leadership skills to address organizational barriers. The Facilitator will have extensive 
experience in implementing CBT and EBPs in community settings. Sites receiving Facilitation 
will receive weekly calls for up to 10 weeks from the Facilitator who will cover the following:  
1. Initiation and benchmarking (week 1): Facilitator contacts each SP and holds a call with 
SP to give background on CBT, review potential barriers and facilitators to CBT uptake, 
and set measurable goals for CBT uptake.  
2. Mentoring (weeks 2-9): Facilitator and SP hold regular weekly calls to develop rapport; 
Facilitator provides guidance to SP on overcoming specific barriers to CBT uptake by 
aligning SP’s strengths with SP’s available influence at the school and needs of local 
staff.  If needed, Facilitator refers SP to REP support assistant. 
3. Leveraging (weeks 2-10): Facilitator continues calls with SP and reaches out to school 
administrators, identifies school/community priorities per administration input, and helps 
SP align CBT use/goals with these existing priorities.  The Facilitator helps SP 
summarize and describe added value of CBT to administrators and other school 
employees (e.g., consistency with other initiatives). 
4. Ongoing marketing (week 10 and onwards): Facilitator, leadership, & SP summarize 
progress and develop sustainability plans. 
 
Assessments and Measures 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
8 
 
     Key measures for this study (Table 1) will collect data on frequency of CBT session delivery 
by SPs through month 18 (primary outcome); CBT fidelity, student mental health symptoms; 
student and SP knowledge of CBT; school-level factors; and REP, Coaching and Facilitation 
fidelity. Independent study research associates (RA) will collect all assessments from SPs and 
schools. In order to protect student anonymity over the course of the study, SPs will administer 
student surveys. Assessments include weekly SP CBT delivery surveys; SP surveys; student 
surveys; administrator surveys; state records; and activity logs from coaches, facilitators, and 
other staff involved in implementation efforts. Forty randomly selected SPs will also be asked to 
complete time-motion surveys for two weeks during the study.  
 
AIM 1 Primary Outcome (CBT Delivery): The primary outcome is the total number of CBT 
sessions delivered by each SP over the course of 18 months. Each week, SPs will receive an 
email or text message reminder to fill out a short survey recording the number of student 
sessions of each of five CBT components they have delivered that week (noting either group or 
individual sessions as well as brief (<15 minute) or full-length sessions). No student names will 
be recorded in the study survey. SPs will also be provided with a tracking template for personal 
use that they can use to track individual students and which will also simplify their weekly data 
entry. Study staff will follow up with SPs on erroneous or duplicative reports, as well as with SPs 
who fail to report CBT delivery. Sensitivity analyses will consider the following outcomes: 
different types of CBT delivery (individual vs. group; full sessions vs. brief) and delivery of 
specific CBT components.  
 
Assessing Potential to Benefit from Facilitation After Phase 1: Following their 7th week post 
Phase 1 randomization, SPs will also receive a short survey from the REP technical assistant 
querying their potential for benefiting from facilitation in Phase 2. This survey includes three 
questions: (1) whether they have delivered 4 or more CBT components to any students, either 
in groups or individual sessions, over the past 8 weeks, and if yes, (2) the number of students 
that have received 4 or more CBT components over the past 8 weeks; and (3) if they have 
experienced any of a number of barriers to delivering CBT to students. Schools where one or 
more participating SPs delivered ≤3 CBT components to ≤10 students (in either individual or 
group format) OR reported 4 or more barriers will be classified as eligible for 
coaching/facilitation and will be re-randomized by TRAILS staff to continue with their current 
strategy (REP or REP + Coaching) or to augment with Facilitation (REP + Facilitation or REP + 
Coaching + Facilitation). This threshold is based on our previous studies (e.g., R01MH099898) 
in which providers who have not offered multiple components of CBT to >10 students over 8 
weeks are unlikely to offer an adequate dose of CBT over the course of the semester, thus 
prompting potential use of additional implementation support. From past studies and our 
preliminary data, we anticipate that 90% of schools receiving REP and 50% of schools receiving 
REP + Coaching will have potential to benefit from Facilitation after Phase 1. As this survey is 
provided as part of the REP implementation strategy and is used to determine further 
implementation support, it is not incentivized. SPs who fail to fill out the survey will be deemed 
as having potential for benefiting from facilitation.  
 
     Justification for facilitation benefit measures: Identifying SPs who deliver ≥3 core CBT 
components to ≥10 students allows for two distinct forms of non-response: (1) failure to identify 
students and (2) failure to deliver CBT. Failure to identify appropriate students is a key measure 
of non-response to CBT implementation because identification of appropriate students is directly 
taught as a component of the Stage 1 implementation strategy. Failure to deliver CBT 
components to students is also part of the non-response definition because it directly reflects a 
failure of the CBT training provided by both REP and Coaching. We chose these variables in 
our non-response definition because they directly impact fidelity, and are measurable in the 
short time frame (8 weeks) needed for determining facilitation eligibility prior to randomization to 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
9 
 
higher-level implementation strategies. Although fidelity will be measured and is a critical 
variable, it is not the primary target of our Stage 2 implementation strategy (Facilitation), which 
targets organizational barriers to implementation. 
 
School and administrative-level measures: A longitudinal, web-based school assessment will 
be given to school administrators (school principal/dean) to record percentage of students 
eligible for free/reduced lunch, average classroom size, attendance rate, number of students 
referred to psychiatric emergency services, administrator tenure, and perceived school-wide 
support for evidence-based practices. A one-page consent form will be included as the first 
page of the Qualtrics administrator survey, which will outline study objectives, anticipated 
participation activities, and the voluntary nature of the research opportunity. Completion of the 
consent page will be required before the rest of the survey items may be accessed. They will 
check a box to confirm they agree to participate and will then be taken to the survey. This 
survey will be provided during the run-in phase, and again at 18 months. Support for evidence-
based practices will be assessed using 2 previously-established tools for evaluating 
organizational capacity and support for adopting EBP: the Implementation Leadership Scale 
(ILS), which assesses leadership (e.g., district) support for EBP and the Implementation Climate 
Scale. No identifying information will be collected as part of these assessments and no 
compensation will be provided.  
 
SP measures:  In addition to weekly assessment of our primary outcome of CBT delivery, SPs 
will be surveyed prior to Phase 1 (baseline) and again at 3, 6, 12, and 18 months. These 
surveys will all be delivered through Qualtrics with a link provided via text and/or email. SPs will 
be electronically consented prior to their first assessment and will have a waiver of 
documentation of consent for this portion of the project. SPs will be compensated for all 
completed assessments. Baseline surveys will collect information about SP background, 
including level of education, job tenure, and prior experience administering CBT. Baseline and 
follow-up surveys also assess SPs on four dimensions of CBT competence: knowledge of and 
comfort using a core set of CBT skills; self-reported frequency of CBT skill use; general 
perceptions of CBT; and barriers to CBT use in school. In the absence of existing, validated 
tools specifically for this setting, measures were developed based on several existing tools and 
tailored for the school setting, including the CBT Knowledge Questionnaire; Provider Attitude 
Survey; Treatment Manuals Survey; and the Psychotherapy Practice Scale, which inquire about 
the frequency of use of specific EBP techniques and self-efficacy in using them.  These 
measures have been deployed successfully in pilot data collection, and appropriate 
psychometric analyses will be performed to ensure reliability and validity as part of study 
analyses. SP surveys at baseline, 12 and 18 months will also include the ILS, ICS, and 
Evidence-Based Attitude Practice Scale (EBPAS). Forty randomly selected SPs will also be 
asked to complete time-motion surveys for two weeks (weeks 12-14 of the study) that ask about 
time allotted to providing CBT versus other forms of student counseling, care or crisis 
management. In response to the COVID-19 pandemic, the final SP survey (month 18) will be 
opened early and will remain available for a two-month period to give SPs the flexibility to 
complete the SP survey while adjusting to working in a virtual learning environment due to 
school closures. Separate from the SP survey, the TRAILS program is distributing a COVID-19 
related survey. This survey asks about SPs’ use of TRAILS materials and their emerging needs 
in their position due to COVID-19. SPs are not required to complete this as part of the ASIC 
study, but will have the opportunity to do so as participants of TRAILS. ASIC SPs who choose to 
take this survey will have the option to link their survey responses to their ASIC Study ID. 
 
SP Compensation: All SPs will receive $3 per weekly assessment (x 60 weeks), $10 for taking 
the SP survey (x 5 time points), $25 for reporting academic outcomes for students (x 2; end of 
each school year) for a potential total of $280 over the 18 month study. SPs randomly selected 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
10 
 
to complete the time-motion surveys will receive an additional $50; and SPs randomly selected 
to complete the qualitative interview will receive an additional $25 for a total of $355. 
Compensation will be delivered to SPs in form of a reloadable gift card at eight points during the 
study period (approximate payment dates: February 2019; March 2019; May 2019; October 
2019; November 2019; February 2020; May 2020; Summer 2020). For each payment, the 
research associates will calculate compensation amounts each individual SP has earned during 
that time point, and have the appropriate amount loaded on to the gift card.  
 
Dropped SPs: SPs who choose to withdraw from the study, or are dropped from the study due 
to not meeting eligibility requirements (e.g. not attending the one-day didactic training), will have 
their access to the web based tool used for data collection disabled. There are two plans in 
place for how the dropped/withdrawn SPs’ access will be disabled:  
 
1) Temporary plan: The web developer/honest broker who created the web tool has 
already been notified of the dropped SPs. The Research Associates will notify the web 
developer if additional SPs are dropped. Until the permanent plan (described below) is 
implemented, the web developer will “disable” the dropped SPs from the access to their 
web tool account. When disabled, these SPs will not be able to log in or reset their 
password for the web tool. No reminders will be sent. The web developer is currently 
working on disabling all dropped/ineligible SP accounts. 
 
2) Permanent plan: The web developer will create a new feature on the admin dashboard 
of the web tool, where the Research Associate can upload a file (indicating which SPs 
are dropped), and then it will automatically disable their access to the web tool. These 
SPs will be noted as “Dropped SPs” on the admin dashboard and no data will be 
collected afterwards. The Research Associate will be able to indicate SPs to be dropped, 
but will not see any of the confidential student information (i.e. initials) that the web 
developer (honest broker) has access to. 
  
 
 
 Table 1: Data Sources and Measures 
Aim 1: 
Measures 
Measure frequency 
Data Sources 
Primary outcome 
and endpoint 
Total number of sessions of CBT delivered 
over the course of 18 months 
Weekly, Months 1-
18 (no collection 
during summer 
months) 
SP weekly survey  
Exploratory 
student outcomes 
Full sessions of CBT delivered; non-group 
CBT sessions delivered; brief sessions of 
CBT delivered 
Weekly, Months 1-
18 (no collection 
during summer 
months) 
SP weekly survey 
 
Student mental health outcomes (PHQ-9 
modified for teens; GAD-7) 
Months 3, 6, 12, 18 
Student survey 
administered by SP 
 
Student knowledge of CBT; reported CBT 
receipt 
Months 3, 6, 12, 18 
Student survey 
administered by SP 
Exploratory Aim 
1: Cost 
Effectiveness 
Cost of REP, Coaching, and Facilitation 
Weekly, Months 1-
15; Daily during 
two-week time and 
motion survey 
Coach & facilitator 
logs; REP TA 
database; SP time & 
motion survey 
 
School outcomes (absences, graduation, 
GPA); services (referrals to care; ED 
admission)  
Months 3, 6, 12,18; 
end of each 
academic year  
Student survey 
administered by SP; 
school records 
reported by SPs with 
student consent 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
11 
 
Exploratory Aim 
2: Moderators 
School factors: Baseline (Phase 1 & 2): 
Size, % of students eligible for free/reduced 
lunch; school administrator support 
(turnover; EBPAS); barriers to CBT 
delivery;  
Baseline, Month 18 
State records; school 
administrator survey 
 
SP factors (aggregated): Baseline (Phase 
1 and 2): Tenure, perceptions of CBT, job 
title, prior training; Time-varying (Phase 2): 
change in number of CBT sessions 
delivered 
Baseline, Month 3 
SP survey 
Exploratory Aim 
3: Mechanisms 
CBT fidelity, knowledge, perception, skills, 
barriers to use; contextual factors; EBPAS 
Baseline, Months 3, 
6, 12, 18 
SP survey 
 
SP support for implementation; ILS, ICS, 
EBPAS 
Baseline, Month 12 
and 18 
SP survey; school 
administrator survey 
Covariates 
across all Aims 
Student demographics; access to mental 
health services; GPA; school attendance 
Months 3, 6, 12,18 
Student survey 
administered by SP 
     
Student identification, recruitment, outcomes and measures: For student mental health 
outcomes (secondary outcomes), SPs will be asked to provide study staff with a list of 10 
students whom they believe could benefit from CBT and who could potentially complete study 
data collection between the period of January 2019 and April 2020 (i.e., not high school seniors 
who would be graduating prior to April 2020). Identified students may or may not be the same 
students SPs intend to treat or are currently treating in their CBT groups. SPs receive training 
through REP (printed materials, technical support, and didactic training) regarding the 
identification of students that could benefit from CBT based on the presence of at least one or 
more symptoms of depression or anxiety that potentially impact their day to day well-being and 
functioning.  
 
Once SPs have identified 10 students, they will use an honest broker-developed, web-based 
tool to enter the first two letters of the students’ first and last names only of the 10 students they 
have identified as being able to possibly benefit from CBT and who could potentially complete 
study data collection between January 2019 and April 2020. The tool will automatically translate 
each student’s first two letters of their first and last name into a unique ID and populate a study-
managed secure database. Student identifying information will only be seen by the web 
administrator, who is an honest broker unaffiliated with the research team. The honest broker 
will be the only person to have access to both student IDs and student names, though he should 
not need to access this information, he would have no benefit from accessing this information, 
and is not required to access it at any point during the study. Additionally, as a staff member at 
the University of Michigan, the honest broker is compliant with all rules related to confidentiality 
of records. Study staff will not see any identifying information for students and school 
professionals will not see the student’s study ID number.  
 
SPs will not be compensated on student identification or recruitment into the study, as we want 
to ensure that student recruitment is voluntary and the presence of the SP in the recruitment 
process is not seen as coercive. The study team will provide all information and materials to the 
SPs to enable them to recruit identified students into the study. This process will include the 
following: 
(1) SPs will receive from the study team a document – ‘SP instruction sheet’ – which will 
outline all instructions for identifying students and inviting them to participate in the study 
activities. 
(2) SPs will arrange for in-person meetings with each student individually any time between 
Sept 1, 2018 and January 23, 2019. During this time, the study condition REP will be 
under way for all SPs and the TRAILS team will be providing training and materials on 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
12 
 
proper identification of students who may exhibit signs or symptoms of depression or 
anxiety and who could potentially benefit from school-delivered mental health services.  
(3) During the SP/student meetings, the SP will read the ‘SP Oral Script’ to the student 
(4) As instructed in the script, the SP will hand the student the ‘ASIC study student info 
sheet’ if the student wants to learn about the study opportunity. This information sheet 
will outline the opportunity to participate in research, student eligibility criteria, study 
assessments, student compensation, and potential risks and benefits. This information 
sheet will also explicitly state that the decision to participate in the research study is 
entirely voluntary and unrelated to the student’s academic record and performance, that 
student participation in the study is not linked to their ability to seek or receive CBT 
treatment or other services provided by the SP, that all student data will be completely 
de-identified, and that the SP (or any other school staff) will not have access to any 
assessment data. 
(5) The student will be asked to read along on the study info sheet, while the SP follows 
their oral script 
(6) The SP will send home a ‘ASIC study family Info Sheet” to the parent/guardian via email 
(first choice) or with the student (back up choice for families that do not have access to 
email) for the parent/guardian to review. There will be an exception in place for students 
who are receiving behavioral health services confidentially.  
(7) Students will not be required to opt in or out of the study during the initial  meeting with 
the SP, and they may take time to think about participating in the study before letting the 
SP know of their decision. If a student decides to participate, they will be told that they 
will be notified by the SP when the first study survey is available for them to complete.  
(8) Parents/guardians will have a minimum of 1 week to opt their child out of the research 
study before the student is presented with the first opportunity to complete a survey. 
Parents may also opt their child out at any time during the study. 
(9) All student surveys will be administered online via Qualtrics. The SP will arrange for the 
student to access a private location in the school with a computer. When the student is 
ready to start the survey, the SP will open the link on to the survey and then leave the 
room. Students may complete the survey independently without the SP’s help. 
(10) On every student survey, the first page will be an assent paragraph, reminding the 
student of the purpose of the study, the voluntary nature of the study, and the student’s 
ability to decline to complete the study or to leave blank any item on the survey that they 
do not want to answer without risk of any penalty. Students will be required to check a 
box indicating that they understand this paragraph and agree to complete the survey 
items, before the survey will open. The final screen of the survey will again ask the 
student to confirm that they are voluntarily submitting their responses to the study.  
 
Students that decline to participate in the study at any point will not be approached by SPs any 
further in relation to study participation. SPs will not be expected to provide replacement names 
for students that decline to participate in the study or for whom participation is not possible (e.g., 
due to changing schools or dropout).    
 
SPs have been identified as the best school personnel to assist with student recruitment and 
data collection for the following reasons:  
 
First, SPs are well-positioned to perform these functions because of their professional 
expertise and school role; flexible schedule during the school day (i.e., not teaching in 
classrooms); their established relationships with students; their knowledge of social and 
emotional health; their perception among students as supportive, non-disciplinary adults; 
and their familiarity with policies regarding student privacy and protections. 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
13 
 
 
Second, introducing a third party (e.g., study staff or alternate school personnel) poses 
the potential for a breach of student confidentiality and/or protected health information, 
as students are identified by their SPs based on the presence of mental health 
symptoms perceived by the SP. 
 
Third, mental illness continues to carry a significant social stigma and school staff 
uninvolved in student behavioral health may be unaware of the critical need for 
sensitivity when working with students for study recruitment or data collection purposes, 
thus jeopardizing the well-being and safety of those students; 
 
Fourth, in many cases SPs will have a pre-existing relationship with the student 
regarding their behavioral health, whereas other school staff members (e.g., teachers, 
school administrators) are unlikely to have this relationship with students and therefore 
students may feel pressure to participate in the study or complete assessments due to 
the authoritative and/or evaluative role of the individual presenting them with the 
opportunity. 
 
This process of identifying and recruiting students into the study ensures that student identities 
and personal health information are protected. However, this process of student recruitment 
may also introduce biases as to which students are recruited. In particular, SPs may include 
students on the study list that they believe would be more likely to fill out study assessments 
and/or easier to recruit to the study. Further, SPs that are more invested in the TRAILS program 
and mission may put more effort into recruiting students. The study team will make efforts to 
reduce these sources of bias by: (1) providing all SPs with adequate training during the REP 
run-in phase and didactic training regarding the identification of students that might benefit from 
CBT; (2) providing all SPs with identical recruitment materials to provide to students; and (3) 
ensuring that all student identification and recruitment is done prior to the first randomization, so 
that any differences in SP efforts to recruit are distributed across study arms.  
 
Student outcomes will be assessed at months 3 (their baseline), 6, 12 and 18. Student 
outcomes will be collected in schools by their SP. SPs will be instructed to ensure that students 
do not miss classes or other school activities in order to complete assessments. All responses 
will be recorded and de-identified through a proprietary web-based platform. This platform is 
being designed and maintained by a programmer that is not a part of the study team and who 
will serve as an “honest broker” that alone has access to the crosswalk between student names 
(seen by SPs) and their study IDs (seen by study staff), though the honest broker does not have 
any reason to access this crosswalk during the study. To reduce SP burden in collecting student 
outcomes data and to ensure that student outcomes can be assessed without worry that 
responses will be shared with their parents, a waiver of documentation of student consent and a 
waiver of parental consent will be obtained. Collection of student outcomes data carries no more 
than minimal risk to students and collection via SPs may offer some benefit as continued 
contact may facilitate connections to needed services. SPs will be required to provide students 
with a private location for completing all assessments.*  
 
*Note: the only exception to this privacy is if a student endorses suicidality in their assessment 
in which case the study team will follow our data and safety monitoring plan stipulating that the 
web-based platform will automatically generate a notification to the SP of the student so that 
they can follow up with the student for safety purposes. The notification will be sent to the SP 
study dashboard. This privacy exception will also be explicitly stated in the study information 
sheet provided to students prior to study consent. The study team will not be notified if a student 
endorses suicidality in their assessment. In rare circumstances, the study team may be notified 
by the school, SP, or another outside party should a serious adverse event occur. In those 
situations, the study staff will report AEs to the IRB.  

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
14 
 
 
At the beginning of every student survey, the first screen of each Qualtrics assessment will 
remind students that their participation is voluntary, that SPs or other school staff will not have 
access to their answers, that all answers will be de-identified, and that their participation in the 
study is not linked to their ability to seek or receive CBT or other treatment from the SP. At the 
end of each survey, students will be reminded again that they are not required to submit their 
survey responses but instead may choose to close their survey without submitting it. 
 
All assessments will be delivered via personalized Qualtrics links to the SPs study dashboard. 
Assessments will include mental health symptoms (PHQ-9 modified for teens, GAD-7); CBT 
receipt; school attendance; access to mental health services and other healthcare use (e.g., ED 
referrals or admission). Original measures piloted during TRAILS were acceptable to students 
and sensitive to individual change. Psychometric data about student measures will also be 
collected and interpreted. As part of their waiver of documentation of consent, students will also 
be informed by SPs that at the end of each school year (Spring 2019, Spring 2020), SPs will 
report their de-identified academic indicators (attendance, expulsion, graduation) from student 
records for purposes of assessing long-term cost effectiveness, post-study through student 
graduation. Because this data will be de-identified, sharing if this information with the study 
team is compliant with FERPA (Family Educational Rights and Privacy Act of 1974). Students 
who withdraw from data collection or from the school prior to completion of assessments will be 
dropped from follow-up assessments and no longer contacted by the SP for data collection. No 
efforts will be made to recruit replacements for students who drop out of the study.  
 
COVID-19 Modifications for Final Student Survey: In response to the COVID-19 pandemic, 
the final student survey (month 18) will be distributed electronically, will be opened early, and 
will remain available for a two-month period to give students the flexibility to complete the 
student survey while adjusting to a virtual learning environment due to school closures. To 
adapt this protocol for a virtual learning environment while still protecting student anonymity, 
SPs will now receive a personalized survey link as well as a message template for distributing 
this link to their enrolled students via the method their school has established for their virtual 
learning community (e.g., email). This message indicates that students can either complete the 
survey via the link at their convenience, or can request to complete the survey with a member of 
the ASIC study team over the phone. For both options, all current survey features will remain 
intact, including the required consent form, the option for students to cancel their responses but 
still get paid for survey completion, and notification of the student’s SP if they indicate suicidality 
as part of their survey clinical assessments. No changes in current protocol are required for 
students who complete the survey online, beyond SP dissemination of the survey link via email 
rather than through in-school contact. 
 
In the event that a student does not have access to the technology and/or internet capability 
necessary to complete the final student survey online, they will be offered the opportunity to 
take the survey telephonically. If this is desired by a student, the study team will work with the 
SP to arrange a time for the student to contact the study to complete a telephonic survey. To 
minimize both the chance of perceived coercion as well as protect against unnecessary physical 
interactions, under no circumstances will SPs be asked to complete surveys with or for their 
ASIC-enrolled students. SPs will be asked to refer to their students using only initials to protect 
students’ identifying information. To further minimize risk to students, a member of the TRAILS 
clinical staff will be on call during all scheduled phone interviews with students; students that 
report suicidality will be connected with this clinical staff member immediately following their 
survey completion, and all students will be offered the opportunity to discuss any mental health 
concerns with a TRAILS clinician following their interview.   
 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
15 
 
At the start of the interview, the interviewee will read the consent form to the student and obtain 
oral consent. No changes have been made to the consent form. The student will be reminded 
that their answers are completely confidential and they don’t need to answer any question that 
they don’t want to and can stop at any time. Students will also be reminded that, should their 
answers indicate that they may need help, that their SP will be notified. At the end of the 
interview, students will be asked if they want to “submit” their answers. If yes, their telephonic 
answers will be submitted to the study database. If not, student answers will be destroyed.  
 
In line with current study protocol, should a student indicate suicidality while completing their 
PHQ-9 evaluation, a message with resources will pop up and the interviewer will read these 
resources to the student. As previously described, the SP for the student will also automatically 
receive a triggered email that their student with “Xx Xx” initials may need help and they should 
follow their schools’ protocols (in line with current study protocol).  A TRAILS clinical team 
member will also be on call during all interviews. Should a student indicate suicidality, they will 
be notified and will join the call to offer assistance to the student as soon as the interview is 
complete (or earlier if the study staff deems it appropriate).  
 
 
Students will be compensated $5 for each completed survey, for a total of $20 total over the 
study period. To ensure student anonymity, $5 gift cards will be sent from UM to school SPs for 
each student that completed the survey following data collection at study months 3, 6, 12, and 
18. SPs will receive a notification on their study dashboard alerting them to which student has 
completed their survey and is due for their gift card. The SP will then pass these gift cards along 
to students that the dashboard identifies as completing the surveys. In the event that a student 
does not submit their answers to the survey (i.e. checking the box at the end of the survey that 
says they want to close the survey without submitting their answers), the student will still get 
their $5 gift card. In the event that the student opts out of the study at the beginning of the 
survey (i.e. checking the box that says they no longer wish to participate in the study), the SP 
will not approach the student and the student will not receive compensation as the survey was 
not completed. In responses to the COVID-19 pandemic, students will be sent an electronic gift 
card for the submission of their final student survey (month 18) to minimize physical contact 
between SPs and students, as well as SP burden. 
 
REP, Coaching and Facilitation cost and utilization: For each implementation strategy, we 
will calculate the average costs and average outcomes per SP using methods described 
elsewhere. The primary implementation costs are the personnel time spent in REP activities 
(e.g., SP training, TA), Coaching (e.g., time to hire/train coaches, network maintenance, SP 
coaching time), and Facilitation by study personnel (including SP and school administrator 
time). Costs will be quantified as hours multiplied by wages and fringe benefits for each person. 
Wage rates will be obtained from school records, and in cases where this information is not 
available, average wages for each occupational level will be used from the Bureau of Labor 
Statistics.  Hours will be tracked through attendance logs for each implementation activity. In 
addition to differences in training and administrative time across implementation strategies, 
there may also be some changes in the distribution of how SPs spend their work time resulting 
from their newly acquired skills. To examine this possibility, we will conduct an exploratory 
analysis of how SPs allocate their time to various activities (e.g., group support for students; 
individual support for students, etc.), using a time-and-motion survey during a two-week period 
for a randomly-selected group of 40 SPs.  
     Student-level service costs will be estimated from study records of participation in CBT 
sessions, school records of participation in other school services, and self-reported utilization 
survey data on inpatient, ER, and outpatient use outside the school setting. School services will 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
16 
 
be translated to costs based on the wage rates of school providers. Health care costs will be 
assigned using Current Procedural Terminology (CPT) codes, and a relative value unit (RVU) 
weight in the Medicaid Fee Schedule will be used to calculate standardized costs in U.S. dollars 
for each service, adjusted for annual levels of inflation using the consumer price index. 
 
Study Sample Retention Protocol, Study Participation: Consistent with best practices for 
implementation study designs, we will aim to prevent study attrition by following a planned 
protocol for obtaining the primary research outcome (total CBT sessions delivered by each SP) 
regardless of engagement in or adherence to implementation strategies or even if an SP moves 
to another institution (occurring <2% in our previous studies). The study RA will monitor SP 
reports of CBT delivery monthly. SPs who fail to submit reports for four consecutive data 
collection weeks will receive two personalized emails from the study RA asking for their report. 
SPs who do not respond will be contacted by phone by the study coordinator. To ensure 
complete data collection, staff will maintain brief communication with all SPs through periods of 
vacation and will provide easy methods for reporting job transitions that could impact data 
collection. Study staff will work with school administrators, coaches and facilitators to ensure 
that SP current contact information is maintained throughout the course of the study. 
 
Statistical Analyses 
Intent to treat: All 100 schools, once randomized at Phase 1, will be included in the intent-to-
treat data analysis sample for all aims.  
 
Primary Aim Analysis: This study design includes within it four distinct adaptive 
implementation interventions (Table 2). The primary aim analysis will determine the effect of 
implementation intervention #4, an adaptive intervention which (i) provides all SPs within 
schools with REP+Coaching in Phase 1 and (ii) augments with Facilitation in Phase 2 for 
schools eligible for facilitation; versus implementation intervention #1, which provides REP 
alone to schools in both phases of intervention (‘control’) on frequency of SP CBT delivery. The 
primary aim analysis is a mean comparison of the total number of CBT sessions delivered by 
SPs over the course of 18 months between schools in experimental conditions D+F versus 
schools in experimental conditions A+B in Figure 1. A weighted comparison between 
experimental conditions D+F vs. A+B is required since, as part of the design, a school will 
contribute differentially to one or more of the 4 embedded adaptive implementation interventions 
depending on whether or not the school is eligible for randomization to facilitation after Phase 1. 
To facilitate this weighted comparison, an easy-to-use, marginal, weighted least squares 
regression approach will be used.  The regression analysis model includes an intercept, a 
contrast coded (+1/-1) indicator for Phase 1 intervention A1, a contrast-coded indicator for 
Phase 2 intervention A2, and the interaction between Phase 1 and Phase 2 interventions. 
Analyses will adjust for the following baseline school-level measures: school size, percent of 
students eligible for free/reduced lunch, SP level of education, and job tenure. The same 
strategy will be used to analyze the secondary and exploratory outcomes. 
 
 
 
Table 2. Adaptive Implementation Interventions Embedded in Study Design 
 
Implementation 
Intervention  
Phase 1 
Intervention 
Eligible for 
Facilitation After 
Phase 1 
Phase 2 
Intervention 
Experimental 
Conditions  
(Figure 1) 
(+1, -1) 
#1: REP only 
REP 
Ineligible 
REP 
A+B 
Eligible 
REP 
(+1, +1) 
#2: REP with Facilitation 
REP 
Ineligible 
REP 
A+C 
Eligible 
REP + Facilitation 
(-1, -1) 
#3: REP with Coaching 
REP + Coaching 
Ineligible 
REP + Coaching 
D+E 
Eligible 
REP + Coaching 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
17 
 
(-1, +1) 
#4: REP with Coaching and 
Facilitation 
REP + Coaching 
Ineligible 
REP + Coaching 
D+F 
Eligible 
REP + Coaching + 
Facilitation 
 
 
Analyses for Exploratory Aims:  
 
The goal of Exploratory Aim 1 analysis is to estimate the costs of different embedded adaptive 
interventions. Incremental cost effectiveness ratios (ICERs) will be calculated for each relevant 
comparison of adaptive implementation interventions, by dividing the incremental average costs 
by the incremental average outcomes. The outcomes will include the primary outcome 
(depression or anxiety-free days based on PHQ-9T or GAD-7 score changes between each 
time point). Confidence intervals and cost-effectiveness acceptability curves will be calculated 
using standard Monte Carlo methods for simulation/bootstrapping. Analyses will include 
economic costs associated with student behavioral outcomes, including absences, suspensions, 
and grades.  
 
Exploratory Aim 2 analyses will determine whether baseline or time-varying SP or school-level 
factors moderate adaptive implementation intervention effectiveness. From prior literature, we 
have identified we have identified several candidate moderators for testing moderation of the 
effects of both REP versus REP+Coaching (Phase 1) and Facilitation vs. no Facilitation (Phase 
2). Specifically, for the comparison between REP and REP+Coaching, we will examine school-
aggregated SP prior training and baseline perceptions of CBT, as well as school size and 
percent free/reduced lunch-eligible. For Phase 2, amongst schools that could benefit from 
Facilitation, we will examine whether the effect of augmenting with Facilitation is moderated by 
school-aggregated CBT delivery during first 8 weeks, and number of barriers to CBT reported, 
or school administrator support for adoption of innovation. Q-learning, a generalization of 
moderated regression analysis to multiple phases of treatment, will be used for these analyses. 
Q-learning regression uses a backward induction (dynamic programming) logic that 
incorporates effects of future treatment decisions in evaluation of tailoring variables (i.e., time-
varying moderators) to build optimal present treatment decisions.  
 
Exploratory Aim 3 analyses will test mechanisms through which the Coaching and Facilitation 
implementation strategies increase frequency of CBT delivery and improve student mental 
health outcomes. Coaching is hypothesized to improve CBT fidelity and uptake by increasing 
SP CBT knowledge, perception and skill; Facilitation is hypothesized to improve CBT uptake by 
improving SP CBT perception and school administrator support. Using mediator-analysis 
methodologies, we will examine how (a) changes in SP CBT knowledge, perception and skill 
strategies due to Coaching act as mechanisms by which CBT delivery by SPs increases; and 
(b) changes in SP CBT perception and school administrator support due to Facilitation acts as 
mechanisms by which CBT delivery by SPs increases, adjusting for baseline and time-varying 
factors jointly associated with hypothesized mediators and SP delivery of CBT to students (e.g., 
change in administrator ILS) via inverse-probability-of-treatment weighting.  
  
ASIC Qualitative Data Collection: To augment the analyses of the exploratory aims, we will 
conduct and analyze qualitative interviews with SPs. Interviews will cover three broad topics: (1) 
barriers/facilitators to implementing TRAILS in their schools; (2) satisfaction and/or engagement 
with their assigned implementation strategies, notably REP and/or Coaching and/or Facilitation; 
and (3) for SPs that were able to successfully implement at least some TRAILS delivery, the 
anticipated barriers/facilitators of sustained delivery of TRAILS programming, including 
resources they anticipate needing in order to sustain current delivery, or increase delivery to 
meet present or anticipated student need. Interviews will be carried out with approximately 30, 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
18 
 
but no more than 40, randomly selected SPs between mid-April and mid-June 2020.The 
interviews are expected to take 30-45 minutes to complete and SPs will receive an additional 
$25 for taking part in the interview opportunity.  
Forty SPs will be randomly selected from all consented SPs still participating in the study. 
Random selection will be stratified on two variables: (1) whether they achieved “implementer” 
status, defined as having delivered either 6+ group sessions in the past three months or 8+ 
individual sessions, utilizing 3 or more CBT components, in the past six weeks; and (2) whether 
they received Coaching or not. Post-hoc analyses will examine balance for our other treatment 
of interest, Facilitation. In April 2020, selected SPs will receive an email invitation to participate 
in the interview opportunity. The email will contain a Qualtrics link wherein SPs can either opt 
out of the opportunity (in which case, a replacement SP will be randomly selected and sent an 
interview invitation), or alert our study staff to schedule their interview to be carried out via the 
BlueJeans Video Conferencing system. At the time of the interview, the study staff conducting 
the interview will read the ‘ASIC Oral Consent Document’ to the SP and obtain their oral 
consent if SPs opt in to participating. Before SPs consent, they will be asked for permission to 
record the interview via BlueJeans. SPs will also be notified that all interviews not containing 
student information or personal health information will be transcribed using an outside 
transcription service that is an approved UM vendor. As the outside vendor is not HIPAA-
compliant, any interviews that include personal health information or student information will be 
indicated directly following the end of the interview, and will be transcribed in-house by study 
staff, with all identifying information excluded from the transcript. Upon completion of each 
interview, BlueJeans audio files will be immediately downloaded and stored locally on an 
encrypted server of the University of Michigan that can only be accessed by study staff. Once 
downloaded, the recordings will then be deleted immediately from BlueJeans. 
 
Following transcription, all interviews will be independently coded by two study staff members. 
Codes will be compared and discussed until consensus is reached by coders. Thematic 
analysis will be used to identify emergent recurring and/or salient themes in the interview data. 
The qualitative data management software system NVIVO will be used to facilitate data 
analysis. 
 
Coach Qualitative Interviews: 
To augment the analyses of the exploratory aims, we will conduct and analyze qualitative 
interviews with Coaches participating in ASIC. Interviews will cover three broad topics: (1) coach 
interactions with ASIC SPs throughout the ASIC study; (2) coach interactions with the TRAILS 
program throughout the ASIC study; and (3) mechanisms impacting the coaching 
implementation strategy in the context of the ASIC Study. Interviews will be carried out with 
approximately 20, randomly selected coaches between October and December 2020.The 
interviews are expected to take 45-60 minutes to complete and coaches will receive an 
additional $25 ($35 if they coached at multiple schools for the ASIC Study) for taking part in the 
interview opportunity. 
Twenty coaches will be randomly selected from all ASIC-participating coaches. Post-hoc 
analyses will examine balance for our variables of interest, coach engagement with SPs, coach 
engagement with TRAILS, and coach recommendation for continuation as a coach. In October 
2020, selected coaches will receive an email invitation to participate in the interview opportunity. 
The email will contain a Qualtrics link wherein coaches can either opt out of the opportunity (in 

Full NIH Title: Improving Student Mental Health: Adaptive Implementation for School-based CBT 
Protocol # HUM00132239 
Protocol version: v5 5-3-2018 
PI: Amy Kilbourne, PhD, MPH 
 
19 
 
which case, a replacement coach will be randomly selected and sent an interview invitation), or 
alert our study staff to schedule their interview to be carried out via the Zoom Video 
Conferencing system. At the time of the interview, the study staff conducting the interview will 
read the ‘ASIC Oral Consent Document’ to the coach and obtain their oral consent if coaches 
opt in to participating. Before coaches consent, they will be asked for permission to record the 
interview via Zoom. Coaches will also be notified that all interviews not containing student 
information or personal health information will be transcribed using an outside transcription 
service that is an approved UM vendor. As the outside vendor is not HIPAA-compliant, any 
interviews that include personal health information or student information will be indicated 
directly following the end of the interview, and will be transcribed in-house by study staff, with all 
identifying information excluded from the transcript. Upon completion of each interview, Zoom 
audio files will be immediately downloaded and stored locally on an encrypted server of the 
University of Michigan that can only be accessed by study staff. Once downloaded, the 
recordings will then be deleted immediately from Zoom. 
Following transcription, all interviews will be independently coded by two study staff members. 
Codes will be compared and discussed until consensus is reached by coders. Thematic 
analysis will be used to identify emergent recurring and/or salient themes in the interview data. 
The qualitative data management software system NVIVO will be used to facilitate data 
analysis. 
 
Missing data: Missing outcome data may occur due to school or SP dropout or loss of contact 
with SPs or students. Our sample retention protocol will ensure that all efforts are made to 
obtain primary outcome measures for all SPs in all 100 schools. For our primary SP-level 
outcome, based on preliminary data from TRAILS, we anticipate an attrition rate of <10%.  Prior 
to conducting all primary and exploratory data analyses, missing data will be dealt with explicitly 
using multiple imputation (MI) methods for SMART studies.