A Mobile Health Solution to Improve Between-Session Skills Practice in Youth Mental Health Treatment
Phase: NA
Conditions: Anxiety, Depression, Posttraumatic Stress Disorder
Sponsor: University of South Florida

 
 
 
 
 
PROTOCOL TITLE: 
A Mobile Health Solution to Improve Between-Session Skills Practice in Youth 
Mental Health Treatment  
 
PRINCIPAL INVESTIGATOR: 
Brian E. Bunnell 
Department of Psychiatry and Behavioral Neurosciences 
813-974-8607 
bbunnell@usf.edu 
 
VERSION NUMBER/DATE: 
V11. July 27, 2023 
 
REVISION HISTORY 
*This table should only be used during submission of a Modification application to the IRB.  
 
Revision 
# 
Version Date 
Summary of Changes 
Consent 
Change? 
1 
05/18/2021 
Added the following: Attitudes Toward 
Homework Questionnaire for providers to 
complete at baseline and post-trial; missing 
question from the first page of the CATS-Y 
and CATS-P; weekly audio recording upload 
form; weekly email about providers uploading 
audio files. 
Made minor edits to the following: weekly 
emails about provider referrals; families phone 
screen; families and providers demographics; 
HRS-II questions for simplicity; embedded 
informational video in the redcap referral form 
and edited the end of the video to reflect this. 
Revised the following consent forms to 
improve clarity: K23 Aim 1 Caregivers Trial 
Consent and Permission, Version #2; K23 Aim 
1 Providers Trial Consent, Version #2; K23 
Aim 1 Youth Trial Assent, Version #2 
Revised protocol to add the Attitudes Toward 
Homework Questionnaire. 
Y 
2 
6/22/2021 
Made minor edits to the following: HRS-II 
questions. 
Revised protocol to add ClinicalTrials.Gov 
NCT# and update abbreviations on screening 
questionnaires for consistency.  
N 
3 
7/7/2021 
Revised protocol to increase provider sample 
size to 30 and family sample size to 76 
Y 

 
 
 
Page 2 of 28 
Revised: 7/27/2023 
 
Revised the following consent forms to update 
sample sizes: K23 Aim 1 Caregivers Trial 
Consent and Permission, Version #3; K23 Aim 
1 Providers Trial Consent, Version #3; K23 
Aim 1 Youth Trial Assent, Version #3 
4 
12/1/2021 
Revised protocol to include the Assessment of 
Core CBT Skills (ACCS) to assess provider’s 
CBT delivery. Added the following document: 
Assessment of Core CBT Skills (ACCS) 
N 
5 
3/1/2022 
Revised protocol to include option for families 
to be able to opt out of audio recording their 
therapy sessions. Made minor edits to the K23 
Aim 1 Study Introduction Script about family’s 
option to audio-record therapy sessions and 
K23 Aim 1 Provider Family Referral Form 
with updated video link. Revised the following 
consent forms with updates: K23 Aim 1 
Caregivers Trial Consent and Permission; K23 
Aim 1 Providers Trial Consent; K23 Aim 1 
Youth Trial Assent. 
Y 
6 
3/21/2022 
Revised protocol to increase provider sample 
size to 40. 
Revised the following consent forms to update 
sample sizes: K23 Aim 1 Caregivers Trial 
Consent and Permission; K23 Aim 1 Providers 
Trial Consent. 
Y 
7 
5/10/2022 
Revised protocol to include the re-consent 
procedure if a study family changes therapists 
and the verbal re-consent if a youth turns 18-
years old during the study. 
Revised the following consent forms: K23 Aim 
1 Caregivers Trial Consent and Permission; 
K23 Aim 1 Youth Trial Assent; Added the 
following consent form: K23 Aim 1 Youth 18+ 
Trial Consent. 
Y 
8 
7/7/2022 
Revised protocol to increase provider sample 
size to 60 and update provider compensation to 
be sent after family enrollment. Currently 
enrolled providers will not be re-consented, but 
will be contacted via phone call and email to be 
updated with the changes in compensation. 
Revised the following consent forms to update 
sample sizes: K23 Aim 1 Caregivers Trial 
Consent and Permission; K23 Aim 1 Providers 
Trial Consent; K23 Aim 1 Youth 18+ Trial 
Consent. Uploaded the following script to 
provide current providers with updated 
compensation procedures: K23 Aim 1 Provider 
Y 

 
 
 
Page 3 of 28 
Revised: 7/27/2023 
 
Compensation Update Script 
9 
9/1/2022 
Revised protocol to clarify provider 
compensation. Revised the following consent 
form to clarify provider compensation: K23 
Aim 1 Providers Trial Consent 
Y 
10 
7/27/2023 
Revised protocol to remove the requirement of 
having been in the CBT+Adhere.ly condition 
to participate in the post-trial interviews. 
Added that we will be conducting post-trial 
interviews with up to ≤20 providers, ≤10 of 
whom that referred families and ≤10 of whom 
did not refer families to the study. Uploaded 
revised versions of K23 Aim 1 Trial Post-Trial 
Interview – Providers and K23 Aim 1 Trial 
Post-Trial Interview – Families to update 
interview questions. Uploaded K23 Aim 2 
Providers Impl Interview and K23 Aim 1 Post-
Trial Interviews Verbal Consent. Uploaded 
revised version of Aim 2 Interview 
Recruitment Email and K23 Aim 2 Providers 
Impl Interviews Verbal Consent. Currently 
enrolled providers will not be re-consented, but 
we will contact 10 providers who were not in 
the CBT+Adhere.ly group via phone call to be 
updated with the changes and will obtain 
verbal consent to participate in post-trial 
interviews. 
Y 
 
 

 
 
 
Page 4 of 28 
Revised: 7/27/2023 
 
Table of Contents 
1.0 
Study Summary....................................................................................................... 5 
2.0 
Objectives................................................................................................................ 6 
3.0 
Background ............................................................................................................. 6 
4.0 
Study Intervention................................................................................................... 7 
5.0 
Procedures Involved.............................................................................................. 11 
6.0 
Data and Specimen Storage for Future Research.................................................. 15 
7.0 
Sharing of Results with Subjects .......................................................................... 15 
8.0 
Study Timelines .................................................................................................... 15 
9.0 
Inclusion and Exclusion Criteria........................................................................... 15 
10.0 
Vulnerable Populations ......................................................................................... 16 
11.0 
Local Number of Subjects..................................................................................... 16 
12.0 
Recruitment Methods ............................................................................................ 16 
13.0 
Withdrawal of Subjects ......................................................................................... 17 
14.0 
Risks to Subjects ................................................................................................... 17 
15.0 
Potential Benefits to Subjects or Others ............................................................... 17 
16.0 
Data Management and Confidentiality ................................................................. 18 
17.0 
Provisions to Monitor the Data to Ensure the Safety of Subjects ......................... 20 
18.0 
Provisions to Protect the Privacy Interests of Subjects......................................... 20 
19.0 
Compensation for Research-Related Injury .......................................................... 20 
20.0 
Subject Costs and Compensation .......................................................................... 20 
21.0 
Consent Process .................................................................................................... 21 
22.0 
Setting ................................................................................................................... 22 
23.0 
References ............................................................................................................. 23 
 
 

 
 
 
Page 5 of 28 
Revised: 7/27/2023 
 
1.0 
Study Summary  
1.1 
Study Title 
A Mobile Health Solution to Improve Between-Session 
Skills Practice in Youth Mental Health Treatment 
Study Design 
Randomized trial methodology with assessments 
Primary 
Objective/Purpose 
The purpose of this study is to preliminarily evaluate 
Adhere.ly, a web-based platform to improve provider 
implementation and patient engagement in homework (i.e., 
between-session practice of skills learned during therapy) 
during child mental health treatment by conducting a small-
scale feasibility trial in community practice settings 
comparing Cognitive Behavioral Therapy (CBT) to 
CBT+Adhere.ly. 
Secondary 
Objective(s)/Purposes 
To assess strategies for implementing Adhere.ly in 
community practice settings.  
Research 
Intervention(s)  
Cognitive Behavioral Therapy 
ClinicalTrials.gov 
NCT # 
NCT04931199 
Study Population 
Masters-level mental health providers; Treatment seeking 
children ages 7-17 years and their caregivers 
Sample Size 
N=242 
n=60 providers; 76 youth patients; 76 caregivers;  
≤10 community leaders; ≤10 supervisors; ≤10 senior leaders 
Study Duration for 
individual subjects 
Up to 12 months for providers 
6 months for families 
Study Specific 
Abbreviations/ 
Definitions  
CBT=Cognitive Behavioral Therapy 
EBT=Evidence-Based Therapy 
IOM=Institute of Medicine 
 
 

 
 
 
Page 6 of 28 
Revised: 7/27/2023 
 
2.0 
Objectives 
2.1 The objectives of this project are to preliminarily evaluate Adhere.ly 
(https://adhere.ly/), a web-based platform to improve provider implementation and 
patient engagement in homework (i.e., between-session practice of skills learned during 
therapy) during child mental health treatment, conduct a small-scale feasibility trial in 
community practice settings comparing Cognitive Behavioral Therapy (CBT) to 
CBT+Adhere.ly, and begin to assess strategies for implementing Adhere.ly in community 
practice settings. 
 
Aim 1. Conduct a feasibility trial comparing CBT to CBT+Adhere.ly 
The trial will include 60 mental health providers randomized to deliver standard CBT 
(n=30) vs. CBT+Adhere.ly (n=30) and 76 treatment seeking youth (ages 7-17) with 
clinically elevated symptoms of anxiety, depression and/or PTSD and their caregivers, 
referred by providers; each provider will treat 3 study cases. Clinical outcomes (i.e., 
anxiety, depression, PTSD, disruptive behavior) will be assessed at baseline and 3- and 6-
months post-baseline. Study targets (i.e., homework use and adherence) will be assessed 
via session audio recordings and weekly patient report. Post-trial qualitative interviews 
will be conducted with ≤20 providers and ≤10 families. 
 
Aim 2. Assess strategies for implementation in preparation for a large-scale trial We 
will conduct remote, 30-minute individual qualitative interviews with ≤10 community-
clinic providers, ≤10 supervisors, and ≤10 senior leaders. 
 
2.2 This is a feasibility trial and is not powered for hypothesis testing. 
 
3.0 
Background 
3.1 Mental health disorders affect 1 in 4 youth in the US and are associated with costly 
physical and behavioral health problems. The quality of community mental health 
services these patients receive vs. should receive is highly variable and characterized as a 
“quality chasm” by the IOM.1 Homework is integral to high quality EBTs for many 
clinical conditions (e.g., MH, substance use, chronic pain, diabetes).2–5 Homework 
exercises (e.g., self-monitoring, relaxation, exposure, parent behavior management) are 
assigned by providers in-session and completed by patients between sessions with the 
goal of “practicing” therapeutic skills. There are numerous benefits to homework use 
(i.e., assignment and review by providers) and adherence (i.e., quantity and quality of 
completion by patients).6,7 Homework enables the generalization of skills and behaviors 
learned during therapy, provides continuity between sessions, allows providers to better 
grasp patients’ learning, and strengthens that learning, leading to improved maintenance 
of treatment gains.8,9 Meta-analytic and systematic reviews have shown that homework 
use and adherence predict and increased treatment engagement, decreased treatment 
dropout, and medium-to-large effects on improvements in clinical outcomes for use 
(d=.48-.77) and adherence to (d=.45-.54) homework.2,8–13 Simply explained, 68% vs. 
32% of patients can be expected to improve when therapy involves homework.2  
 
Despite its many benefits, homework is significantly underutilized in mental health 
treatment. Providers for child and adult patients use homework in an average of 57% of 
sessions, and only 25% of providers report using systematic procedures for 
recommending homework.14 A national survey revealed that 93% providers estimate 

 
 
 
Page 7 of 28 
Revised: 7/27/2023 
 
rates of patient adherence to homework to be low to moderate.15 Only ~55% of family 
providers report using homework “often” to “almost always,” and studies report low to 
moderate rates of youth/caregiver adherence (~39-63%).16–23 There are many barriers to 
successful homework implementation. Providers experience difficulty with remembering 
to assign and assess homework, communicating rationale and instructions, providing 
adequate reinforcement, deciding what to assign, and assigning exercises flexibly and 
creatively. Patients experience difficulty remembering when, how, and why assignments 
should be completed and lack resources to do assignments effectively.24  
 
Among adults aged 18-49 years (likely representative of primary caregivers in this study) 
88-92% own a smartphone and ~99% own a cellphone.25 Many technology-based mental 
health resources (e.g., mHealth) are effective, practical, desired by patients and providers, 
and available at low cost.26–31 Prior work by our team and others suggests that mHealth 
has tremendous potential to positively affect homework use and adherence and 
resultantly, treatment efficiency and effectiveness.32,33 Providers and patients strongly 
support the concept of using an mHealth app to improve homework implementation. 
Providers desire automated reminders, built-in rewards systems, data management, 
tracking, and summaries, and rationale and implementation instructions. Patients desire 
customizable avatars, incentives and rewards (e.g., levels, points), reminder notifications, 
instructions and rationale explanations, and data transfer between youth, caregiver, and 
provider accounts.24  
 
mHealth solutions to improve access and quality of care are widely investigated and 
effective in facilitating behavior change.27,34–36 As noted in recent reviews, some mHealth 
resources include features that can support homework (e.g., voice and SMS reminders 
and feedback, self-monitoring and assessment, modules and activities that can be used to 
facilitate between-session practice), but little emphasis has been placed on their role in 
affecting use and adherence.32,33 Some mHealth apps with homework -specific resources 
have been developed with positive preliminarily effects (e.g., “PE Coach” for PTSD; 
“Challenger” for social anxiety; “Get Happy” for depression).37–42 However, these were 
designed mainly for adult patients and data on their ability to explicitly target provider 
and use and patient adherence to homework is limited. It would be beneficial to develop 
and test an mHealth homework resource for youths and caregivers with a broad symptom 
focus. 
 
3.2 Adhere.ly’s initial conceptualization was informed by semi-structured qualitative 
interviews with 21 national trainers in Trauma Focused Cognitive Behavioral Therapy 
(TF-CBT) and 15 youth/caregiver TF-CBT patients. These interviews explored potential 
mHealth solutions to barriers to implementing homework during youth mental health 
treatment. Results suggested that many providers struggle to consistently develop, assign, 
and assess homework exercises with their patients, many of whom have difficulty 
remembering to practice skills in a correct and timely way that fosters adequate learning. 
Trainers and patients were generally positive about the potential for mHealth to improve 
the implementation of homework and provided suggestions for mHealth solutions in 
terms of functionality and user experience.24  
 
4.0 
Study Intervention  

 
 
 
Page 8 of 28 
Revised: 7/27/2023 
 
4.1 We designed Adhere.ly as a simple, HIPAA compliant, web-based application to help 
mental health providers implement homework during CBT with children with elevated 
PTSD, anxiety, and/or depression, and their caregivers. The three major components of 
Adhere.ly help providers to (1) Practice interactive, digitized CBT exercises with 
patients in-session; (2) Remind patients to practice CBT exercises for homework, and (3) 
Review homework during the next session. These functions are accessible via tabs on the 
left side of the site, as displayed in Figure 1. 
 
 
Figure 1 
 
The Practice component includes several brief interactive, digitized CBT exercises for 
providers to practice with patients in-session within each of the following areas: 
relaxation, affect and emotion, cognitive coping, exposure, parent-child activities, 
enhancing safety, and parenting videos (Figures 2-4). 
 
Figure 2 
 

 
 
 
Page 9 of 28 
Revised: 7/27/2023 
 
 
Figure 3 
 
 
Figure 4 
 
The Remind component enables providers to set automated text message/email 
reminders with editable default days/times, for patients to practice exercises between-
session. This includes all exercises found in the Practice component as well as automated 
customizable reminders, encouraging messages, self-monitoring, and parenting exercises 
(Figures 5-6). Patients receive automated text message (or email) reminders during the 
following week on the specified days and times that contain links to the digitized 
exercises, which are opened and completed in the patients’ smartphone or computer 
browser (i.e., no downloads or logins are required). 
 
 
Figure 5 

 
 
 
Page 10 of 28 
Revised: 7/27/2023 
 
 
 
Figure 6 
 
The Review component enables providers to view patients’ homework completion and 
relevant data for certain exercises (e.g., self-monitoring ratings, anxiety ratings during 
exposure exercises; Figure 7).  
 
 
Figure 7 
 
Adhere.ly is accessed by computer, tablet, and/or smartphone browser so it does not 
require any downloads by patients or providers. Patients do not create accounts or log in, 
and providers who choose not to create an account are still able to use the site with 
limited functionality. This functionality is limited in that it does not allow providers to 
save patient contact information (i.e., it must be entered every time a reminder is 
scheduled), does not allow providers to review homework results, and limits reminders to 
customizable reminders, encouraging messages, and self-monitoring and relaxation 
exercises (Figure 8). 
 

 
 
 
Page 11 of 28 
Revised: 7/27/2023 
 
 
Figure 8 
 
5.0 
Procedures Involved 
5.1 Aim 1 of this study will use a randomized controlled trial design. The trial will 
include 60 community mental health providers randomized to deliver standard CBT 
(n=30) vs. CBT+Adhere.ly (n=30) to a total of 76 treatment-seeking youth ages 7-17 
(n=38 per condition) with clinically elevated anxiety, depression, and/or PTSD, and their 
caregivers (n=38 per condition). Youth and caregivers will be referred and treated by 
providers. Study staff will facilitate web-based administration of baseline and 3- and 6-
month post-baseline questionnaires with children and caregivers via REDCap. Treatment 
sessions may be audio recorded and uploaded to REDCap by providers and coded by 
study staff to assess provider assignment and assessment of homework. Families will 
have the option to allow providers to audio-record the youth’s therapy sessions through 
the caregiver consent. Youth homework adherence will be assessed weekly on the day of 
their next session using a REDCap survey, a link to which will be sent via automated 
text-message and/or email reminders, or by telephone after 1 day of no response. Post-
trial qualitative interviews will be conducted with ≤20 providers and ≤10 families. 
 
Aim 2 of this study will use a mixed methods design and will include remote, 30-minute 
individual qualitative interviews with ≤10 community-clinic providers, ≤10 supervisors, 
and ≤10 senior leaders. 
 
5.2 
☒ Audio/Video Recording 
☐ Psychophysiological Recording 
☒ Behavioral Interventions 
☐ Record Review - Educational 
☒ Behavioral Observations and Experimentations 
☐ Record Review - Employee 
☐ Deception 
☐ Record Review- Medical  
☐ Focus Groups 
☐ Record Review - Other 
☒ Interviews 
☐ Specimen Collection or Analysis 
☐ Investigational Device – Non-Significant Risk 
(e.g. Mobile Applications) 
☒ Surveys and/or Questionnaires 
☐Psychometric Testing 
☐ Other Social-Behavioral Procedures 
 
 

 
 
 
Page 12 of 28 
Revised: 7/27/2023 
 
Recruitment of Providers and Orientation to Study Procedures  
We will recruit 60 master’s-level mental health providers by leveraging established 
research-practice partnerships with mental health clinics and centers in Florida (see 
attached Letters of Support). Providers will be contacted by study staff via telephone 
and/or email to inform them about the study and inquire about their interest in 
participating. Providers who express interest in participating in the study will be 
scheduled for a 30-60-minute, televideo-based consent and training process with study 
staff to discuss the study, referral procedures, audio recording sessions, and uploading 
recordings to REDCap. Signed informed consent from providers will be obtained by 
study staff via REDCap e-Consent. Afterwards providers will complete a demographics 
questionnaire and the Attitudes Toward Homework Questionnaire (ATHQ).15 
 
Recruitment of Youth and Caregiver Patients  
We will recruit 76 treatment-seeking youth ages 7-17 with clinically elevated anxiety, 
depression, and/or PTSD, and their primary caregivers (n=76). Families will be referred 
and by providers following their initial intake session with that provider. We will ask 
providers to refer 4-5 families to maximize the likelihood of treatment completion with at 
least 3 families. Providers will receive automated weekly emails with reminders to make 
referrals and links to a REDCap referral form. Providers will also be able to provide 
referral information via telephone or email if preferred. Providers will be given 
informational cards with a QR code to a brief video with information about the study to 
show to families. If families express interest in the study, providers will assist them in 
completing the referral or obtain verbal consent to submit their referral information.  
 
Upon receiving a referral, study staff will contact families within two working days to 
provide study information, assess eligibility, obtain signed informed consent from 
caregivers and assent from youth via REDCap e-Consent, and assist families in 
completing baseline assessments. Study staff will assess eligibility by: (1) administering a 
phone screen to the caregiver; and (2) emailing or texting families a link through the 
REDCap system to a survey that will include a brief demographic questionnaire, the 
Revised Children’s Anxiety and Depression Scale-25 Child Version (RCADS-25-Y)43, 
Child and Adolescent Trauma Screen - Youth Report (CATS-Y)44, Revised Children’s 
Anxiety and Depression Scale-25 Parent Version (RCADS-25-P)43, and Child and 
Adolescent Trauma Screen - Caregiver Report (CATS-P)44. Families who are eligible, as 
indicated by a T-score ≥ 65 on the RCADS-25-Y and/or RCADS-25-P or total score ≥ 15 
on the CATS-Y and/or CATS-P, will then provide informed consent and assent via 
REDCap e-Consent, and complete the remaining study baseline questionnaires. 
 
Treatment 
Providers will be randomly assigned to administer either CBT alone (n=30) or 
CBT+Adhere.ly (n=30). All providers will be asked to administer CBT as they usually 
would for each patient, either in-person or via telemedicine, with CBT+Adhere.ly 
providers integrating Adhere.ly into treatment with their families. To minimize risk of 
CBT alone (i.e., “control”) providers being less motivated to refer cases, providers will 
be informed of their assignment only after their first study-eligible family has been 
enrolled. Each provider will be asked to treat at least 3 families over the course of 3 
months. Providers will be asked to audio record their treatment sessions using either 
audio recorders provided by the study team, or audiorecording software on their 
computer, if preferred. Families will have the option to audio record the treatment 
sessions through the caregiver consent. Providers will be asked to upload those 
recordings to REDCap either weekly or bi-weekly. Providers will receive automated 

 
 
 
Page 13 of 28 
Revised: 7/27/2023 
 
weekly emails with a reminder to upload audio recordings and a link to a REDCap form 
for uploading recording. This form will ask the provider to enter their name, the name of 
the youth being treated, and the session number. Providers who have not uploaded a 
recording after 2 weeks will receive a follow-up phone call from study staff to provide 
reminders and assistance where needed. 
 
Assessment Strategy and Measures  
Trial questionnaires are shown in Table 1. Baseline and 3- and 6-month post-baseline 
questionnaires will be completed by families via REDCap surveys. Study staff will email 
or text families a link to the survey through the REDCap system and will be available to 
remotely assist families in completing questionnaires. Youth and caregiver questionnaires 
will be sent via separate links to allow older youth privacy. As stated previously, two 
youth (i.e., RCADS-25-Y and CATS-Y) and two caregiver questionnaires (i.e., RCADS- 
25-P and CATS-P) will be administered to screen potential participants for eligibility 
criteria and will be administered prior to obtaining informed consent. The informed 
consent document will specify that these data will be included with participants’ study 
data if they are eligible and agree to participate or will not be included if they are 
ineligible or decide not to participate. 
 
Youth Homework Adherence, Provider Homework Use, and Treatment Fidelity 
Youth homework adherence will be assessed weekly on the day of their next session using 
a REDCap survey, a link to which will be sent via automated text-message and/or email 
reminders, or via telephone by study staff after 1 day of no response. The first 4 items of 
the Homework Rating Scale II (HRS II) will be used to measure homework adherence.52 
Session audio recordings will be observationally coded by independent coders blinded to 
study aims. Provider adherence and competence in reviewing, designing, and assigning 
homework (i.e., homework use) will be assessed using the Homework Adherence and 
Competence Scale (HAACS).53 Provider fidelity to the CBT treatment model will be 
assessed by coding a randomly selected 20% of session audio recordings using the 
Cognitive Behavioural subscale of the Comparative Psychotherapy Process Scale (CPPS-
CB).54 Provider general therapeutic and CBT-specific skills to appropriately deliver CBT 
will be assessed by the Assessment of Core CBT Skills (ACCS).55 
 
Post-Trial Qualitative Interviews 
Table 1. Assessment Measures for Trial 
Domain 
Informant Measure 
Time Point 
B 3M 6M 
Demographics 
Provider 
Youth 
Caregiver 
Provider Demographics Questionnaire 
Families Demographics Questionnaire 
 
 
Attitudes Toward 
Homework 
Provider 
Attitudes Toward Homework Questionnaire (ATHQ)15 

Youth Anxiety/Depression Youth 
Caregiver 
Revised Children’s Anxiety and Depression Scale-25 Child Version (RCADS-25-Y)43 
Revised Children’s Anxiety and Depression Scale-25 Parent Version (RCADS-25-P)43   
 
Youth PTSD  
Youth 
Caregiver 
Child and Adolescent Trauma Screen - Youth Report (CATS-Y)44 
Child and Adolescent Trauma Screen - Caregiver Report (CATS-P)44 
  
 
Youth Disruptive Behavior Caregiver Child and Adolescent Disruptive Behavior Inventory (CADBI)45 
  
 
Youth Quality of Life 
Youth 
Caregiver 
KIDSCREEN-10 Index Child and Adolescent Version (KIDSCREEN-10-Y)46 
KIDSCREEN-10 Index Parent Version (KIDSCREEN-10-P)46 
  
 
Caregiver Anxiety 
Caregiver Generalized Anxiety Disorder-7 (GAD-7)47 
  
 
Caregiver Depression 
Caregiver Patient Health Questionnaire-8 (PHQ-8)48 

Therapeutic Alliance 
Youth 
Caregiver 
Working Alliance Inventory - Short Revised - Client Version (WAI-SR-Y)49 
Working Alliance Inventory - Short Revised - Client Version (WAI-SR-P)  
 
 
 
Treatment Satisfaction 
Youth 
Caregiver 
Child/Adolescent Satisfaction Questionnaire (CASQ-Y)50 
Caregiver Satisfaction Questionnaire (CSQ-P)51 
 
 
 
Note. B=Baseline; 3M=3-Month Follow-Up; 6M=6-Month Follow-Up. 

 
 
 
Page 14 of 28 
Revised: 7/27/2023 
 
Study staff will invite up to 20 providers (≤10 of whom that referred families and ≤10 of 
whom did not refer families to the study) and 10 randomly selected youth/caregiver 
patient dyads to participate in remote, ≤ 30-minute, individual qualitative interviews. 
Participants will be recruited via telephone by study staff and scheduled for an interview. 
Informed consent from providers and caregivers, and assent from youth to participate in 
these interviews will have been obtained by study staff during the initial consent process 
for the trial. Study staff will conduct semi-structured interviews during which 
interviewees will be asked to describe their experience with and/or reactions to the trial 
and each component of Adhere.ly, ranging the overall “look and feel” of the user 
interface to ease of navigation, with the goal of identifying barriers/facilitators relating to 
use. Interviewees will also be asked to complete the System Usability Scale56 via 
REDCap survey to provide quantitative data on Adhere.ly’s usability. 
 
Qualitative Interviews to Assess Implementation Strategies 
Study staff will invite ≤10 providers, ≤10 supervisors, and ≤10 senior leaders (e.g., 
Directors) from local community-based clinics to participate in remote, 30-minute, 
individual qualitative interviews to assess strategies for Adhere.ly’s implementation. 
Potential participants will be contacted by telephone and/or email, informed about the 
study, invited to participate, and scheduled for an interview. Study staff will obtain verbal 
informed consent and begin interviews with an overview of the benefits of homework, its 
underutilization, and barriers to use and adherence. Study staff will provide participants 
with an overview of Adhere.ly and access to the app to become familiar with it. Study 
staff will then administer a semi-structured interview guided by the Modified 
Consolidated Framework for Implementation Research (mCFIR) domains relevant to this 
implementation (i.e., intervention characteristics, outer settings, inner settings, end-user 
characteristics, and process of implementation).57,58 
 
Local Site Documents (See Uploads): 
A. Provider Demographics 
B. Patient Referral Form 
C. Phone Screen 
D. Screening Questionnaires 
E. Trial Measures 
F. Post-Trial Qualitative Interviews and SUS 
G. Implementation Strategies Interviews 
 
5.3 The standard of care procedures for children with clinically elevated anxiety, 
depression, and PTSD is traditional CBT from their mental health provider. 
 
5.4 There are no additional foreseeable risks to the above procedures in need of further 
mitigation beyond those ordinarily incurred in working with this population. Standard 
operational procedures of clinics specify responsibilities for handling dangers to self and 
others, safety planning in the event of domestic violence, and reporting responsibilities in 
the event of a new episode of child abuse. 
 
5.5 N/A 
5.6 N/A 
5.7 N/A 

 
 
 
Page 15 of 28 
Revised: 7/27/2023 
 
 
6.0 
Data and Specimen Storage for Future Research 
6.1 N/A 
6.2 N/A 
6.3 N/A 
 
7.0 
Sharing of Results with Subjects 
7.1 Total scores on study questionnaires will be shared verbally by study staff with youth 
and caregivers upon request. Total scores will also be shared with providers upon request 
from providers and when permission to do so is granted by families as indicated on their 
informed consent form. 
 
8.0 
Study Timelines 
8.1 Aim 1. Providers will participate in one 30-60-minute consent and training process 
and twelve ~60-minute therapy sessions per study case. Study referral procedures will 
take an additional 5-10 minutes per patient. An additional 5 minutes per patient, per 
session will be required to upload audio recordings. Providers who participate in post-
trial qualitative interviews will spend an additional 30 minutes. In all, the total amount of 
time spent in the study will be approximately 40.5 hours over the course of 12 months, 
only 4.5 hours of which will be spent engaging in study procedures beyond their 
everyday practice. 
 
Families will participate in a 15-minute eligibility screening and a 30-minute 
consent/assent process. Families will participate in twelve ~60-minute therapy sessions, 
engage in twelve ~10-minute homework assignments, and spend ~5 minutes each week 
completing homework assessments. Baseline, 3-, and 6- month follow-up assessments 
will take ~30 minutes. Families who participate in post-trial qualitative interviews will 
spend an additional 30 minutes in the study. In all, the total amount of time spent in the 
study will be approximately 16.5 hours over the course of 6 months, only about 2.5 hours 
of which will be spent engaging in study procedures beyond the time they would have 
otherwise been receiving therapy from their provider. 
 
Aim 3. Providers, supervisors, and senior leaders who participate in qualitative 
interviews to assess implementation strategies will spend 30 minutes in the study. 
 
9.0 
Inclusion and Exclusion Criteria 
9.1 Aim 1. 
Providers: English-speaking, mental health providers who have obtained at least a 
master’s degree in social work, counseling, clinical psychology, or related field; carry 
active child mental health treatment caseloads; and have a laptop, tablet, or smartphone 
with internet access.  
Youth: English-speaking youth ages 7-17 years with clinically elevated anxiety and 
depression as indicated by a T-score ≥ 65 on the RCADS-25-C and/or RCADS-25-P or 
total score ≥ 15 on the CATS-Y and/or CATS-P; and have a laptop, tablet, or smartphone 
with internet access. 
Caregivers: English-speaking, adult (≥18 years old) caregivers of a youth who meets 
eligibility criteria and have internet and smartphone access. 
 

 
 
 
Page 16 of 28 
Revised: 7/27/2023 
 
Aim 2. English-speaking, adult (≥18 years old) mental health providers, supervisors, and 
senior leaders in Florida. 
 
9.2 Youth with self- or caregiver-reported (1) active psychotic symptoms (e.g., 
hallucinations, delusions) or (2) significant cognitive disability, developmental delays, or 
pervasive developmental disorder. 
 
9.3 Families will be ineligible to continue if there is a discontinuation or interruption of 
CBT treatment with participating provider. If a family changes therapists to a provider 
enrolled in the study, they will be able to participate with the new provider but will be re-
consented to confirm that they still want to participate with the new provider and will be 
notified that their baseline questionnaire data will be maintained to avoid the unnecessary 
burden of collecting those data a second time. 
 
9.4 We will be including employees as participants. 
 
10.0 Vulnerable Populations 
10.1 This study will involve youth ages 7-17. Informed consent will be obtained from 
parents or legal guardians, who also will be participants in the study. Assent will be 
obtained from all youth. Youth and their caregivers will be informed that their 
information will remain confidential and their decision to participate in the study will not 
affect the quality of care that they would have otherwise received from their provider. 
 
11.0 Local Number of Subjects 
11.1 Aim 1. We will recruit 60 providers, 76 youth patients, and 76 caregivers. N=212. 
 
Aim 2. We will recruit ≤10 community-clinic providers, ≤10 supervisors, and ≤10 senior 
leaders. N=30. 
 
12.0 Recruitment Methods 
12.1 
 
12.2 Aim 1. Providers will be recruited by leveraging established research-practice 
partnerships with mental health clinics and centers in Florida. Providers will be contacted 
by study staff via telephone and/or email to inform them about the study and inquire 
about their interest in participating. Families will be referred and by providers following 
their initial intake session with that provider. We will ask providers to refer 4-5 families 
to maximize the likelihood of treatment completion with at least 3 families. Providers 
will receive automated weekly emails with reminders to make referrals and links to a 
REDCap referral form. Providers will also be able to provide referral information via 
telephone or email if preferred. Providers will be given informational cards with a QR 
code to a brief video with information about the study to show to families. If families 
express interest in the study, providers will assist them in completing the referral or 
☒ Email 
☐ Online/Social Media Advertisement 
☒ Flyer 
☐ Record Review 
☐ Letter 
☐ SONA 
☐ News Advertisement 
☒ Other  

 
 
 
Page 17 of 28 
Revised: 7/27/2023 
 
obtain verbal consent to submit their referral information. Upon receiving a referral, 
study staff will contact families within two working days to provide study information, 
assess eligibility, and obtain informed consent/assent. 
 
Aim 2. We will leverage established research-practice partnerships Potential participants 
will be contacted by telephone and/or email, informed about the study, invited to 
participate, and scheduled for an interview. 
 
12.3 We will emphasize and remind families that their decision to participate in the study 
will not affect the quality of care that they would have otherwise received from their 
provider. 
 
13.0 
Withdrawal of Subjects 
13.1 If a participating provider decides to withdraw participation from the study, families 
being treated by that provider will also be withdrawn from the research without their 
consent and will revert to standard treatment. If a family changes therapists to a provider 
enrolled in the study, they will be able to participate with the new provider but will be re-
consented to confirm that they still want to participate with the new provider and will be 
notified that their baseline questionnaire data will be maintained to avoid the unnecessary 
burden of collecting those data a second time. 
 
13.2 Subjects who withdraw will continue to receive the same standard of care from their 
mental health provider. 
 
14.0 Risks to Subjects 
14.1 Physical, psychological, social, cultural, financial, and legal risks, and risks to 
privacy and/or confidentiality associated with this research are minimal because the 
probability and magnitude of harm or discomfort anticipated in the proposed research are 
not greater, in and of themselves, than those ordinarily encountered in daily lives of the 
general population or during the performance of routine physical or psychological 
examinations or tests. 
 
These include: 
1. Possible breach of patient privacy and/or confidentiality 
2. Possible breach of the security of patient and provider online data 
3. Possible patient discomfort due to completing psychological questionnaires 
4. Possible patient and provider discomfort due to having therapy sessions audio 
recorded 
 
14.2 N/A 
14.3 N/A 
 
15.0 Potential Benefits to Subjects or Others 
15.1 Potential direct benefits to participants assigned to the CBT+Adhere.ly condition 
include access to a clinically useful health technology resource, gains in knowledge and 
skill, and better outcomes.  
 

 
 
 
Page 18 of 28 
Revised: 7/27/2023 
 
15.2 Researchers and the general scientific community will benefit from the knowledge 
gained from the study. This includes knowledge about how to design and implement a user-
centered and stakeholder-informed program that supports patient and provider adherence 
to evidence-based practice as well as effective vs. less effective strategies for implementing 
health technology solutions that promote this adherence in mental health practice settings 
in the community.   
 
16.0 Data Management and Confidentiality 
16.1 The primary goal of the trial is to assess the feasibility of the proposed methodology, 
which will be analyzed using the RE-AIM framework.59 Feasibility data will be 
examined using descriptive and inferential statistics where applicable for variables within 
each RE-AIM domain:  
Reach: Number of potential providers and patients available for recruitment; percentage 
eligible/ineligible, invited, and enrolled (benchmark: 60 providers in months 1-2 of trial 
[15/month]; 76 patients in months 2-14 [5/month]); sample characteristics; reasons for 
ineligibility and varying levels of engagement; ambiguities regarding eligibility criteria.  
Effectiveness: Attrition and related patient characteristics; adverse events; satisfaction; 
impact.  
Adoption: Estimated number of settings for future trials; settings and stakeholder 
characteristics.  
Implementation: Percentage of providers and patients who use the app (benchmark: 
70%); average number of modules used by providers and patients (benchmark: 50%); 
percentage of session recordings uploaded (benchmark: 70%) and completed weekly 
HRS (benchmark: 70%); retention of patients at follow-up (benchmark: 70%); level of 
use and adherence; staff/provider/patient time commitments; patient retention; 
assessment time burden; completed assessments and missing items; broken/lost/stolen 
equipment; technical issues. 
Maintenance: Clinical outcomes at 3- and 6-months post-baseline. 
 
The secondary goal of the trial is to assess the preliminary effects of Adhere.ly. The small 
sample size prevents any conclusions about effectiveness; however, exploratory analyses 
will be conducted using repeated measures ANOVA to preliminarily assess-between 
group differences in clinical outcomes (i.e., anxiety, depression, PTSD, disruptive 
behavior) and study targets (i.e., homework use and adherence) across study time points. 
Further, we will examine relations among study variables (i.e., caregiver anxiety and 
depression, therapeutic alliance, satisfaction). 
 
Qualitative Interview Data Analysis. Interview transcripts will be coded by trained 
study staff in NVivo using a hybrid inductive-deductive, consensus-based content 
analysis.60–62 Aim 2 interviews will also be analyzed using the mCFIR tool.58 Coders will 
meet regularly to ensure consistency with code definitions and resolve inconsistencies via 
discussion to achieve consensus. Data aggregation queries will be used to create case 
memos and assign ratings for mCFIR constructs. We will use the Expert 
Recommendations for Implementing Change (ERIC) matching tool to generate a list of 
potential future implementation strategies based on the results of Aim 2 interviews.63  
 
16.2 All data collected during the trial will be securely stored in a REDCap database, 
including informed consent/assent documents, questionnaire data, and audio recordings. 
Data obtained from Adhere.ly platform for the purposes of this study will include names, 

 
 
 
Page 19 of 28 
Revised: 7/27/2023 
 
phone numbers, and email addresses, and usage data relating to providers practicing, 
assigning, and assessing exercises, and families completing those exercises. These data 
will be downloaded from Adhere.ly and merged with the data in REDcap. Only IRB-
approved and trained study personnel will have access to the REDCap project and access 
will be limited to information and modules that are required for them to complete their 
assigned study-related tasks. All identifiers will be marked as such in REDCap and will 
not be included in the final exported dataset, which will instead include assigned ID 
numbers. 
 
We will use the following security measures to protect data sources: 
1. all research data exported from REDCap will include ID numbers only 
2. the codes that link the name of the participant and the study ID will be kept 
confidential in REDCap 
3. computers and servers containing data will be password-protected to prohibit 
unauthorized access 
4. online survey data collection will be accessible only by IRB approved study staff 
with secure logins 
5. Adhere.ly includes state-of-the-art technical infrastructure–including encryption 
and other software, security practices, and business operational practices to ensure 
compliance with all major governing legislation, including HIPAA 
6. all study data will be kept on a secure, USF server 
 
16.3 We will use the following quality assurance measures for subject recruitment, 
enrollment, enrollment targets, and for the validity and integrity of the data: 
1. study staff will complete and maintain up-to-date CITI and GCP training 
2. study staff will be trained and supervised weekly by the PI 
3. Standard Operating Procedures (SOPs) will be used to train all study staff 
4. a Manual of Procedures (MOP) and study checklists will be used to ensure 
fidelity to the study protocol 
5. informed consent will be obtained and documented by study staff to provide an 
audit trail 
6. any contact with study participants or potential study participants will be 
documented to provide an audit trail 
7. participant screening, recruitment, enrollment and enrollment targets, and data 
collection will be tracked to provide weekly updates to the PI 
8. assessment data will be entered directly by participants into REDCap 
9. automated validity checks will be in place for any data collection 
10. data checks for ranges, cross-validity, and completion will be completed proximal 
to data collection 
11. collection of any study data will be documented by study staff to provide an audit 
trail 
12. coders will be trained and appropriate measures will be used to assess interrater 
reliability 
13. coders will be blinded to study aims and hypotheses 
14. the USF Conflict of Interest (COI) Office will maintain and monitor study 
progress according to ongoing conflict of interest management plans to ensure 
compliance with all requirements 
 

 
 
 
Page 20 of 28 
Revised: 7/27/2023 
 
16.4 Identifiable information in this study will include provider, youth, and caregiver 
names, phone numbers, email addresses, and physical addresses. Human subjects 
research records, including the original signed and dated consent documents, will be 
stored for at least 5 years after study completion. Signed and dated HIPAA authorizations 
and consent documents that include HIPAA authorizations will be stored for at least 6 
years after study completion. After this time data will be deleted from REDCap. Youth 
assessment data will be shared with caregivers and providers upon request, as stated in 
the consent.  
 
16.5 
 
Youth PHI that will be disclosed to providers will include total scores on questionnaires 
if families indicate their permission to do so on the consent form. Youth PHI that will be 
obtained from providers will include names, session dates, audio recordings of sessions, 
telephone numbers, and email addresses. We will document in REDCap events where 
PHI is disclosed or obtained. 
 
17.0 Provisions to Monitor the Data to Ensure the Safety of Subjects 
17.1 The questionnaires used in this study are widely used and validated mental health 
instruments that can cause minor discomfort, particularly questions about prior traumatic 
events (e.g., the CATS). Participants will be informed that they can discontinue at any 
time and the PI, a licensed psychologist, will be available to speak with participants to 
address and resolve any issues relating to discomfort. 
 
17.2 N/A 
 
18.0 Provisions to Protect the Privacy Interests of Subjects 
18.1 All research activities will be conducted over web-based surveys or telephone by 
IRB approved study staff located in a secure and private location. Subjects’ assessment 
results will only be shared with their mental health provider if they indicate their 
permission when providing consent. 
 
18.2 Study participants will provide informed consent or assent. We will not access any 
previously existing records. 
 
19.0 Compensation for Research-Related Injury 
19.1 N/A 
  
20.0 Subject Costs and Compensation 
20.1 Providers and families who participate in this study will have internet and 
☒Obtaining Signed Authorization 
☐Waiver of HIPAA Authorization for 
Recruitment/Screening Purposes Only 
☐Obtaining Online or Verbal Authorization 
(Alteration of HIPAA Authorization) 
☐Waiver of HIPAA Authorization for 
Entire Study 
☐Data Use Agreement 
☐Business Associate Agreement 

 
 
 
Page 21 of 28 
Revised: 7/27/2023 
 
smartphone access, so we anticipate that they will have data and SMS plans that will 
allow them to participate without any additional costs. However, there is the small risk 
that some participants may exceed their monthly data or SMS limits during this study, 
resulting in additional costs for which they will be responsible. This risk will be 
discussed during the informed consent process. 
 
20.2  
 
Aim 1. Providers will receive a $50 eGift card after each family enrolls/consents into the 
study, in compensation of their time. When a family transfers to a new study provider, 
that study provider will be compensated with a $50 egiftcard. Currently enrolled 
providers will not be re-consented but will be contacted to be updated with the changes in 
compensation. Families will be sent eGift cards after the completion of each study 
assessment: $20 for baseline assessments, $20 for 3-month follow-up assessments, and 
$30 for 6-month follow-up assessments. In addition, participants who participate in 
individual qualitative interviews will be compensated $20 eGift card following 
completion of the interview. 
 
Aim 2. Participants will receive a $50 eGift card following completion of the interview. 
 
21.0 Consent Process 
21.1  
 
21.2 Aim 1. The project coordinator will obtain signed informed consent from 
providers via REDCap e-Consent during a 30-60-minute, televideo-based consent 
and training process. We will ensure that providers understand all aspects of the 
study and consent form, and adequate time will be provided for questions relating 
to referral procedures, audio recording sessions, and uploading recordings to the 
study server. We will be conducting post-trial interviews with up to ≤20 
providers, ≤10 of whom that referred families and ≤10 of whom did not refer 
families to the study. The trial consent form will also inform providers in the 
CBT+Adhere.ly that they may be asked to participate in post-trial qualitative 
☐No Compensation 
☐Tokens (pens, food items, etc.) 
☒Financial Compensation (cash, gift cards) 
☐Other  
☐Course Credit (i.e. extra credit, SONA 
points) 
 
☒Obtaining Signed Consent (Subject or 
Legally Authorized Representative) 
☐Obtaining Consent Online (Waiver of 
Written Documentation of Consent ) 
☐Obtaining Signed Parental Permission 
☒Obtaining Verbal Consent (Waiver of 
Written Documentation of Consent) 
☒Obtaining Signed Assent for Children 
or Adults Unable to Consent 
☐Waiving Consent and/or Parental Permission 
(Waiver of Consent Process) 
☐Obtaining Verbal Assent for Children 
or Adults Unable to Consent 
☐Waiving Assent/Assent is Not Appropriate 

 
 
 
Page 22 of 28 
Revised: 7/27/2023 
 
interviews if they participate. Currently enrolled providers will not be re-
consented, but we will obtain verbal consent from 10 providers who were not in 
the CBT+Adhere.ly group to participate in post-trial interviews. To minimize the 
possibility of coercion or undue influence we will emphasize that declining to 
participate in the study will not influence their employment. 
 
The project coordinator will obtain signed informed consent from caregivers and assent 
from youth via REDCap e-Consent during a 30-minute televideo-based consent process. 
We will ensure that providers understand all aspects of the study and consent form, and 
adequate time will be provided for questions relating to the assessment process over the 
course of the study. The consent form will include a field for caregivers to indicate their 
consent to have their child’s total scores from questionnaires shared with their providers 
if requested and for their providers to audio record their child’s therapy sessions. The 
consent form will also inform families that they may be asked to participate in post-trial 
qualitative interviews if they participate. To minimize the possibility of coercion or 
undue influence we will emphasize that declining to participate in the study will not 
influence the quality of care that they would have otherwise received from their provider. 
If a family changes therapists to a provider enrolled in the study, they will be re-
consented to confirm that they still want to participate with the new provider and will be 
notified that their baseline questionnaire data will be maintained to avoid the unnecessary 
burden of collecting those data a second time.  
 
Aim 2. The project coordinator will obtain verbal informed consent from providers, 
supervisors, and senior leaders prior to conducting interviews to assess implementation 
strategies. To minimize the possibility of coercion or undue influence we will emphasize 
that declining to participate in the study will not influence their employment.  
 
21.3 This research presents no more than minimal risk of harm to subjects and involves 
no procedures for which written consent is normally required outside of the research 
context. If a youth participant turns 18 years old while participating in the study, study 
staff will obtain their consent verbally. The project coordinator will obtain verbal 
informed consent from providers, supervisors, and senior leaders prior to conducting 
interviews to assess implementation strategies. To minimize the possibility of coercion or 
undue influence we will emphasize that declining to participate in the study will not 
influence their employment.  
 
21.4 N/A 
21.5 N/A 
 
21.6 This study will involve youth 7-16 years old. Signed assent will be obtained from all 
youth participants via REDCap e-Consent in addition to obtaining permission and 
informed consent from their caregivers. Signed permission and informed consent will be 
obtained from one caregiver even if the other caregiver is alive, known, competent, 
reasonably available, and shares legal responsibility for the care and custody of the child. 
 
22.0 Setting 

 
 
 
Page 23 of 28 
Revised: 7/27/2023 
 
22.1 The research will be conducted via telephone, online, and in the USF Department of 
Psychiatry and Behavioral Neurosciences. Providers will practice and audio record 
sessions at their own clinic site, as normal.  
 
 
23.0 References 
1.  Committee on Crossing the Quality Chasm: Adaptation to Mental Health and 
Addictive Disorders, Board on Health Care Services, Institute of Medicine, National 
Academy of Sciences. Improving the quality of health care for mental and 
substance-use conditions. Washington, D.C., DC: National Academies Press; 2006. 
2.  Kazantzis N, Whittington C, Dattilio F. Meta-analysis of homework effects in 
cognitive and behavioral therapy: A replication and extension. Clin Psychol (New 
York). Wiley; 2010 Jun 8;17(2):144–156. 
3.  Decker SE, Kiluk BD, Frankforter T, Babuscio T, Nich C, Carroll KM. Just showing 
up is not enough: Homework adherence and outcome in cognitive-behavioral 
therapy for cocaine dependence. J Consult Clin Psychol. American Psychological 
Association (APA); 2016 Oct;84(10):907–912. PMCID: PMC5341374 
4.  Caudill M, Schnable R, Zuttermeister P, Benson H, Friedman R. Decreased clinic 
use by chronic pain patients: response to behavioral medicine intervention. Clin J 
Pain. 1991 Dec;7(4):305–310. PMID: 1809444 
5.  Koenigsberg MR, Bartlett D, Cramer JS. Facilitating treatment adherence with 
lifestyle changes in diabetes. Am Fam Physician. 2004 Jan 15;69(2):309–316. 
PMID: 14765768 
6.  Kazantzis N, Whittington C, Zelencich L, Kyrios M, Norton PJ, Hofmann SG. 
Quantity and quality of homework compliance: A meta-analysis of relations with 
outcome in cognitive behavior therapy. Behav Ther. 2016 Sep;47(5):755–772. 
PMID: 27816086 
7.  Kazantzis N, Deane FP, Ronan KR. Assessing compliance with homework 
assignments: review and recommendations for clinical practice. J Clin Psychol. 2004 
Jun;60(6):627–641. PMID: 15141396 
8.  Hudson JL, Kendall PC. Showing you can do it: homework in therapy for children 
and adolescents with anxiety disorders. J Clin Psychol. Wiley; 2002 May;58(5):525–
534. PMID: 11967878 
9.  Scheel MJ, Hanson WE, Razzhavaikina TI. The process of recommending 
homework in psychotherapy: A review of therapist delivery methods, client 
acceptability, and factors that affect compliance. Psychotherapy (Chic). American 
Psychological Association (APA); 2004;41(1):38–55. 
10.  Kazantzis N, Lampropoulos GK. Reflecting on homework in psychotherapy: what 
can we conclude from research and experience? J Clin Psychol. Wiley; 2002 
May;58(5):577–585. PMID: 11967882 

 
 
 
Page 24 of 28 
Revised: 7/27/2023 
 
11.  Kazantzis N. Homework assignments in cognitive and behavioral therapy: A meta-
analysis. Clin Psychol (New York). Wiley; 2000 May 1;7(2):189–202. 
12.  Mausbach BT, Moore R, Roesch S, Cardenas V, Patterson TL. The relationship 
between homework compliance and therapy outcomes: An updated meta-analysis. 
Cognit Ther Res. Springer Nature; 2010 Oct;34(5):429–438. PMCID: PMC2939342 
13.  Sukhodolsky DG, Kassinove H, Gorman BS. Cognitive-behavioral therapy for anger 
in children and adolescents: a meta-analysis. Aggress Violent Behav. Elsevier BV; 
2004 May;9(3):247–269. 
14.  Kazantzis N, Deane FP. Psychologists’ use of homework assignments in clinical 
practice. Prof Psychol Res Pr. 1999 Dec;30(6):581–585. 
15.  Kazantzis N, Lampropoulos GK, Deane FP. A national survey of practicing 
psychologists’ use and attitudes toward homework in psychotherapy. J Consult Clin 
Psychol. American Psychological Association (APA); 2005 Aug;73(4):742–748. 
PMID: 16173863 
16.  Helbig S, Fehm L. Problems with homework in cbt: Rare exception or rather 
frequent? Behav Cogn Psychother. Cambridge University Press (CUP); 2004 
Jul;32(3):291–301. 
17.  Gaynor ST, Lawrence PS, Nelson-Gray RO. Measuring homework compliance in 
cognitive-behavioral therapy for adolescent depression: review, preliminary 
findings, and implications for theory and practice. Behav Modif. SAGE 
Publications; 2006 Sep;30(5):647–672. PMID: 16894234 
18.  Simons AD, Marti CN, Rohde P, Lewis CC, Curry J, March J. Does homework 
“matter” in cognitive behavioral therapy for adolescent depression? J Cogn 
Psychother. Springer Publishing Company; 2012;26(4):390–404. 
19.  Clarke G, Hops H, Lewinsohn PM, Andrews J, Seeley JR, Williams J. Cognitive-
behavioral group treatment of adolescent depression: Prediction of outcome. Behav 
Ther. Elsevier BV; 1992;23(3):341–354. 
20.  Danko CM, Brown T, Van Schoick L, Budd KS. Predictors and correlates of 
homework completion and treatment outcomes in parent–child interaction therapy. 
Child Youth Care Forum. Springer Science and Business Media LLC; 2016 
Jun;45(3):467–485. 
21.  Berkovits MD, O’Brien KA, Carter CG, Eyberg SM. Early identification and 
intervention for behavior problems in primary care: a comparison of two abbreviated 
versions of parent-child interaction therapy. Behav Ther. Elsevier BV; 2010 
Sep;41(3):375–387. PMID: 20569786 
22.  Lyon AR, Budd KS. A community mental health implementation of parent-Child 
Interaction Therapy (PCIT). J Child Fam Stud. Springer Science and Business 
Media LLC; 2010 Oct 1;19(5):654–668. PMCID: PMC2945385 

 
 
 
Page 25 of 28 
Revised: 7/27/2023 
 
23.  Dattilio FM, Kazantzis N, Shinkfield G, Carr AG. A survey of homework use, 
experience of barriers to homework, and attitudes about the barriers to homework 
among couples and family therapists. J Marital Fam Ther. Wiley; 2011 
Apr;37(2):121–136. PMID: 21457279 
24.  Bunnell BE, Nemeth LS, Lenert LA, Kazantzis N, Deblinger E, Higgins KA, 
Ruggiero KJ. Barriers associated with the implementation of homework in youth 
mental health treatment and potential mobile health solutions. Cognit Ther Res 
[Internet]. Springer Science and Business Media LLC; 2020 Mar 16; Available 
from: http://link.springer.com/10.1007/s10608-020-10090-8 
25.  Demographics of mobile device ownership and adoption in the United States 
[Internet]. 2019 [cited 2021 Feb 22]. Available from: 
http://www.pewinternet.org/fact-sheet/mobile/ 
26.  Luxton DD, McCann RA, Bush NE, Mishkind MC, Reger GM. mHealth for mental 
health: Integrating smartphone technology in behavioral healthcare. Prof Psychol 
Res Pr. American Psychological Association (APA); 2011 Dec;42(6):505–512. 
27.  Donker T, Petrie K, Proudfoot J, Clarke J, Birch M-R, Christensen H. Smartphones 
for smarter delivery of mental health programs: a systematic review. J Med Internet 
Res. JMIR Publications Inc.; 2013 Nov 15;15(11):e247. PMCID: PMC3841358 
28.  Boschen MJ, Casey LM. The use of mobile telephones as adjuncts to cognitive 
behavioral psychotherapy. Prof Psychol Res Pr. American Psychological 
Association (APA); 2008 Oct;39(5):546–552. 
29.  Krebs P, Duncan DT. Health app use among US mobile phone owners: A national 
survey. JMIR MHealth UHealth. JMIR Publications Inc.; 2015 Nov 4;3(4):e101. 
PMCID: PMC4704953 
30.  Hanson RF, Gros KS, Davidson TM, Barr S, Cohen J, Deblinger E, Mannarino AP, 
Ruggiero KJ. National trainers’ perspectives on challenges to implementation of an 
empirically-supported mental health treatment. Adm Policy Ment Health. Springer 
Science and Business Media LLC; 2014 Jul;41(4):522–534. PMCID: PMC3758397 
31.  Ruggiero KJ, Saunders BE, Davidson TM, Lewsky Cook D, Hanson R. Leveraging 
technology to address the quality chasm in children’s evidence-based 
psychotherapy. Psychiatr Serv. 2017 Jul 1;68(7):650–652. PMID: 28617212 
32.  Tang W, Kreindler D. Supporting homework compliance in cognitive behavioural 
therapy: Essential features of mobile apps. JMIR Ment Health. JMIR Publications 
Inc.; 2017 Jun 8;4(2):e20. PMCID: PMC5481663 
33.  Bakker D, Kazantzis N, Rickwood D, Rickard N. Mental health smartphone apps: 
Review and evidence-based recommendations for future developments. JMIR Ment 
Health. JMIR Publications Inc.; 2016 Mar 1;3(1):e7. PMCID: PMC4795320 
34.  Ehrenreich B, Righter B, Rocke DA, Dixon L, Himelhoch S. Are mobile phones and 

 
 
 
Page 26 of 28 
Revised: 7/27/2023 
 
handheld computers being used to enhance delivery of psychiatric treatment? A 
systematic review. J Nerv Ment Dis. Ovid Technologies (Wolters Kluwer Health); 
2011 Nov;199(11):886–891. PMID: 22048142 
35.  Heron KE, Smyth JM. Ecological momentary interventions: incorporating mobile 
technology into psychosocial and health behaviour treatments. Br J Health Psychol. 
2010 Feb;15(Pt 1):1–39. PMCID: PMC2800172 
36.  Amstadter AB, Broman-Fulks J, Zinzow H, Ruggiero KJ, Cercone J. Internet-based 
interventions for traumatic stress-related mental health problems: a review and 
suggestion for future research. Clin Psychol Rev. Elsevier BV; 2009 Jul;29(5):410–
420. PMCID: PMC2704915 
37.  Reger GM, Hoffman J, Riggs D, Rothbaum BO, Ruzek J, Holloway KM, Kuhn E. 
The “PE coach” smartphone application: an innovative approach to improving 
implementation, fidelity, and homework adherence during prolonged exposure. 
Psychol Serv. 2013 Aug;10(3):342–349. PMID: 23937084 
38.  Miloff A, Marklund A, Carlbring P. The challenger app for social anxiety disorder: 
New advances in mobile psychological treatment. Internet Interventions. 2015 Nov 
1;2(4):382–391. 
39.  Watts S, Mackenzie A, Thomas C, Griskaitis A, Mewton L, Williams A, Andrews 
G. CBT for depression: a pilot RCT comparing mobile phone vs. computer. BMC 
Psychiatry. 2013 Feb 7;13:49. PMCID: PMC3571935 
40.  Ly KH, Trüschel A, Jarl L, Magnusson S, Windahl T, Johansson R, Carlbring P, 
Andersson G. Behavioural activation versus mindfulness-based guided self-help 
treatment administered through a smartphone application: a randomised controlled 
trial. BMJ Open. 2014 Jan 9;4(1):e003440. PMCID: PMC3902198 
41.  Morris ME, Kathawala Q, Leen TK, Gorenstein EE, Guilak F, Labhard M, Deleeuw 
W. Mobile therapy: case study evaluations of a cell phone application for emotional 
self-awareness. J Med Internet Res. 2010 Apr 30;12(2):e10. PMCID: PMC2885784 
42.  Dagöö J, Asplund RP, Bsenko HA, Hjerling S, Holmberg A, Westh S, Öberg L, 
Ljótsson B, Carlbring P, Furmark T, Andersson G. Cognitive behavior therapy 
versus interpersonal psychotherapy for social anxiety disorder delivered via 
smartphone and computer: a randomized controlled trial. J Anxiety Disord. 2014 
May;28(4):410–417. PMID: 24731441 
43.  User’s Guide. Revised children’s anxiety and depression scale [Internet]. [cited 2021 
Feb 24]. Available from: https://www.childfirst.ucla.edu/wp-
content/uploads/sites/163/2020/12/RCADSUsersGuide20201201.pdf 
44.  Sachser C, Berliner L, Holt T, Jensen TK, Jungbluth N, Risch E, Rosner R, 
Goldbeck L. International development and psychometric properties of the Child 
and Adolescent Trauma Screen (CATS). J Affect Disord. Elsevier BV; 2017 
Mar;210:189–195. 

 
 
 
Page 27 of 28 
Revised: 7/27/2023 
 
45.  Burns GL, Taylor TK, Rusby J. Child and Adolescent Disruptive Behavior 
Inventory-Version 2. 
46.  Erhart M, Ottova V, Gaspar T, Jericek H, Schnohr C, Alikasifoglu M, Morgan A, 
Ravens-Sieberer U, HBSC Positive Health Focus Group. Measuring mental health 
and well-being of school-children in 15 European countries using the KIDSCREEN-
10 Index. Int J Public Health. Springer Science and Business Media LLC; 2009 
Sep;54 Suppl 2(S2):160–166. PMID: 19652910 
47.  Source SRL, K K, Jbw W, B L. A brief measure for assessing generalized anxiety 
disorder. Arch Inern Med. 2006;166:1092–1097. 
48.  Kroenke K, Strine TW, Spitzer RL, Williams JBW, Berry JT, Mokdad AH. The 
PHQ-8 as a measure of current depression in the general population. J Affect 
Disord. 2009 Apr;114(1–3):163–173. PMID: 18752852 
49.  Hatcher RL, Gillaspy JA. Development and validation of a revised short version of 
the working alliance inventory. Psychother Res. Routledge; 2006 Jan 1;16(1):12–25. 
50.  Sigel BA, Kramer TL, Conners-Burrow NA, Church JK, Worley KB, Mitrani NA. 
Statewide dissemination of trauma-focused cognitive-behavioral therapy (TF-CBT). 
Child Youth Serv Rev. Elsevier BV; 2013 Jun;35(6):1023–1029. 
51.  Dorsey S, Lyon AR, Pullmann MD, Jungbluth N, Berliner L, Beidas R. Behavioral 
rehearsal for analogue fidelity: Feasibility in a state-funded children’s mental health 
initiative. Adm Policy Ment Health. Springer Science and Business Media LLC; 
2017 May;44(3):395–404. 
52.  Kazantzis N, Deane FP, Ronan KR. The homework rating scale II—Client version. 
Using homework assignments in cognitive behavior therapy New York, NY: 
Routledge. 2005; 
53.  Kazantzis N, Wedge P, Dobson KS. Homework adherence and competence scale 
(HAACS). From the team research project Cognitive Behavior Therapy Homework 
Project Albany, New Zealand: Massey University. 2005; 
54.  Hilsenroth MJ, Blagys MD, Ackerman SJ, Bonge DR, Blais MA. Measuring 
Psychodynamic-Interpersonal and Cognitive-Behavioral Techniques: Development 
of the Comparative Psychotherapy Process Scale. Psychotherapy: Theory, Research, 
Practice, Training. Educational Publishing Foundation; 2005;42(3):340. 
55.  Muse K, McManus F, Rakovshik S, Thwaites R. Development and psychometric 
evaluation of the Assessment of Core CBT Skills (ACCS): An observation-based 
tool for assessing cognitive behavioral therapy competence. Psychol Assess. 
American Psychological Association (APA); 2017 May;29(5):542–555. 
56.  Blokdyk G. System usability scale. North Charleston, SC: Createspace Independent 
Publishing Platform; 2018. 

 
 
 
Page 28 of 28 
Revised: 7/27/2023 
 
57.  Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. 
Fostering implementation of health services research findings into practice: a 
consolidated framework for advancing implementation science. Implement Sci. 
2009 Aug 7;4:50. PMCID: PMC2736161 
58.  Tilahun B, Smillie K, Bardosh KL, Murray M, Fitzgerald M, Cook V, Poureslami I, 
Forrest J, Lester R. Identifying Barriers and Facilitators of 13 mHealth Projects in 
North America and Africa: Protocol for a 5-Year Implementation Science Study. 
JMIR Res Protoc. 2018 Jul 3;7(7):e162. PMCID: PMC6053607 
59.  Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health 
promotion interventions: the RE-AIM framework. Am J Public Health. American 
Public Health Association; 1999 Sep;89(9):1322–1327. PMCID: PMC1508772 
60.  Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services 
research: developing taxonomy, themes, and theory. Health Serv Res. 2007 
Aug;42(4):1758–1772. PMCID: PMC1955280 
61.  Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual 
Health Res. SAGE Publications; 2005 Nov;15(9):1277–1288. PMID: 16204405 
62.  Shenton AK. Strategies for ensuring trustworthiness in qualitative research projects. 
Education for information. IOS press; 2004;22(2):63–75. 
63.  Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, 
Proctor EK, Kirchner JE. A refined compilation of implementation strategies: results 
from the Expert Recommendations for Implementing Change (ERIC) project. 
Implement Sci. 2015 Feb 12;10:21. PMCID: PMC4328074