A Hybrid Effectiveness-Implementation Trial of Group Cognitive Behavior Therapy (CBT) in Urban Schools
Phase: NA
Conditions: Anxiety Disorders
Sponsor: Children's Hospital of Philadelphia

 
Template version: May 28, 2014 
Title: 
A Hybrid Effectiveness-Implementation Trial of Group CBT in 
Urban Schools 
Short Title 
Hybrid Effectiveness-Implementation Trial 
Sponsor: 
National Institute of Mental Health  
eIRB Number 
15-012311 
Protocol Date: 
October 15, 2015 
Amendment 1 Date: September 19, 2016 
Amendment 2 Date: March 2, 2017 
Amendment 3 Date:  April 4, 2017 
Amendment 4 Date:  August 22, 2017 
Amendment 5 Date: October 12, 2017 
Amendment 6 Date: April 24, 2018 
Amendment 7 Date: September 5, 2018 
 
 
Principal Investigator:  Ricardo B. Eiraldi, PhD  
The Children’s Hospital of Philadelphia 
The Roberts Center for Pediatric Research 
2716 South Street, Room 8293 
Philadelphia, PA 19146 
Phone 215-590-7759 
email: eiraldi@email.chop.edu 
 

 
 
 
 
 
 
 
 
 
 
ii
TABLE OF CONTENTS 
Table of Contents .............................................................................................................. ii 
Abbreviations and Definitions of Terms ......................................................................... iv 
Abstract ............................................................................................................................ vi 
1 
BACKGROUND INFORMATION AND RATIONALE .................................................................... 1 
1.1 
INTRODUCTION .............................................................................................................................. 1 
1.2 
NAME AND DESCRIPTION OF INVESTIGATIONAL PRODUCT OR INTERVENTION ................................... 1 
1.2.1 
Interventions implemented for anxious students by city agency therapists: .................. 1 
1.2.2 
Support Interventions (strategies) offered to city agency therapists implementing 
CATS 
2 
1.3 
FINDINGS FROM NON-CLINICAL AND CLINICAL STUDIES ................................................................. 2 
1.4 
RELEVANT LITERATURE AND DATA ................................................................................................ 3 
1.5 
COMPLIANCE STATEMENT .............................................................................................................. 4 
2 
STUDY OBJECTIVES ........................................................................................................................ 5 
2.1 
PRIMARY OBJECTIVE (OR AIM) ....................................................................................................... 5 
2.2 
SECONDARY OBJECTIVES (OR AIM) ................................................................................................. 5 
3 
INVESTIGATIONAL PLAN .............................................................................................................. 5 
3.1 
GENERAL SCHEMA OF STUDY DESIGN ............................................................................................. 5 
3.1.1 
Screening Phase ................................................................................................................ 6 
3.1.2 
Phase I:  Training of Supervisors & Therapists assigned to all 3 conditions ................. 6 
3.1.3 
Phase 2: Therapist Implementation of EBP (FRIENDS & CATS) & Implementation of 
Support Strategies (TT vs. TT+) ........................................................................................................ 7 
3.2 
STUDY DURATION, ENROLLMENT AND NUMBER OF SITES ................................................................ 8 
3.2.1 
Duration of Study Participation .......................................................................................... 8 
3.2.2 
Total Number of Study Sites/Total Number of Subjects Projected ................................. 8 
3.3 
STUDY POPULATION ....................................................................................................................... 9 
3.3.1 
Inclusion Criteria ................................................................................................................. 9 
3.3.2 
Exclusion Criteria .............................................................................................................. 10 
4 
STUDY PROCEDURES .................................................................................................................... 10 
4.1 
STUDENT SCREENING / REFERRAL PROCESS .................................................................................. 10 
4.2 
INTERVENTION IMPLEMENTATION (FOR STUDENTS) ...................................................................... 10 
4.3 
IMPLEMENTATION OF SUPPORT STRATEGIES .................................................................................. 11 
4.4 
SUBJECT COMPLETION/WITHDRAWAL .......................................................................................... 11 
5 
STUDY EVALUATIONS AND MEASUREMENTS ........................................................................ 11 
5.1 
SCREENING MEASUREMENTS (STUDENTS) ..................................................................................... 11 
5.2 
TREATMENT EFFICACY EVALUATIONS (INFORMATION COLLECTED ABOUT THE STUDENTS ENROLLED 
IN FRIENDS OR CATS)............................................................................................................................ 12 
5.3 
IMPLEMENTATION OUTCOME MEASURES ...................................................................................... 12 
6 
STATISTICAL CONSIDERATIONS ............................................................................................... 14 
6.1 
PRIMARY ENDPOINT ..................................................................................................................... 14 
6.2 
SECONDARY ENDPOINTS .............................................................................................................. 15 
6.3 
SAMPLE SIZE AND POWER ............................................................................................................ 15 
7 
SAFETY MANAGEMENT ............................................................................................................... 16 
7.1 
CLINICAL ADVERSE EVENTS ......................................................................................................... 16 
7.2 
ADVERSE EVENT REPORTING........................................................................................................ 16 

 
 
 
 
 
 
 
 
 
 
iii
8 
STUDY ADMINISTRATION ........................................................................................................... 17 
8.1 
TREATMENT ASSIGNMENT METHODS ............................................................................................ 17 
8.1.1 
Randomization .................................................................................................................. 17 
8.2 
DATA COLLECTION AND MANAGEMENT........................................................................................ 18 
8.3 
CONFIDENTIALITY ........................................................................................................................ 19 
8.4 
REGULATORY AND ETHICAL CONSIDERATIONS ............................................................................. 19 
8.4.1 
Data and Safety Monitoring Plan..................................................................................... 19 
8.4.2 
Risk Assessment .............................................................................................................. 19 
8.4.3 
Potential Benefits of Trial Participation ........................................................................... 20 
8.5 
RISK-BENEFIT ASSESSMENT ......................................................................................................... 20 
8.6 
RECRUITMENT STRATEGY ............................................................................................................ 20 
8.7 
INFORMED CONSENT/ASSENT AND HIPAA AUTHORIZATION ......................................................... 21 
8.8 
PAYMENT TO SUBJECTS/FAMILIES ................................................................................................ 21 
9 
PUBLICATION ................................................................................................................................. 22 
10 
REFERENCES .................................................................................................................................. 22 
  

 
 
 
 
 
 
 
 
 
 
iv
ABBREVIATIONS AND DEFINITIONS OF TERMS 
AE 
 
Adverse Event 
APRS 
 
Academic Performance Rating Scale 
CATS 
 
CBT for Anxiety Treatment in Schools; A group CBT 
program for over anxious children 
BDMC 
 
The Biostatistics and Data Management Core  
CBH 
 
Community Behavioral Health 
CBT 
 
Cognitive Behavioral Therapy 
CDI 
 
Children’s Depression Inventory 
CFC 
 
Content Fidelity Checklist 
CHOP 
 
Children’s Hospital of Philadelphia 
COTTAGe 
 
Child & Adolescent OCD, Tic, Trich & Anxiety Group 
CRS 
 
Consultation Rating Scale 
D&I 
 
Dissemination and Implementation 
EBP 
 
Evidence-based Practice 
FRIENDS 
 
A group CBT program for over anxious children 
GAD 
 
Generalized Anxiety Disorder 
GCBP 
 
Group Cognitive Behavioral Therapy 
GLM 
 
Generalized Linear Models 
IC 
 
Independent Coders 
IITC-ESMH 
 
Index of Inter-professional Team Collaboration for 
Expanded School Mental Health 
IRB 
 
Institutional Review Board 
KT 
 
Knowledge Test 
MASC-2 
 
Multidimensional Anxiety Scale for Children 
PA 
 
Planned Adaptation 
PI 
 
Primary Investigator  
REDCap 
 
Research Electronic Data Capture 
SAD 
 
Separation Anxiety Disorder  
SCARED 
 
Screen for Child Anxiety Related Disorders 
SRCM 
 
Self-Report Coping Measure 
SRS 
 
Supervision Rating Scale  
STS 
 
School Therapeutic Services 
SWPBIS 
 
School-wide positive behavioral interventions and 
supports  

 
 
 
 
 
 
 
 
 
 
v
TT 
 
Train the Trainer (support strategy for the implementation 
of FRIENDS & CATS where Masters-level supervisors 
receive training by experts on conducting supervision) 
 
TT+ 
 
Train the Trainer Plus (support strategy for the 
implementation of CATS where supervisors receive 
training and ongoing consultation by experts on 
conducting supervision) 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
vi
 
ABSTRACT 
Context:  Unresolved psychological problems, such as anxiety, affect a significant number 
of our students and interfere with their ability to attend, actively participate, and prosper in 
school. This project will expand the capacity of selected mental health agencies to provide 
services in the participating schools through school therapeutic services (STS). The project 
will provide enhanced training in evidence-based behavioral health interventions to school-
based mental health providers.  The services will be implemented by STS Master’s level 
therapists supervised by their mental health agency supervisors (Internal Support), who are 
in turn supported by the research team (TT) or external consultants (TT+).  
 
Objectives: The purpose of this intervention study is to test the effectiveness of FRIENDS 
(a program for overanxious children) compared to an adapted version (CATS) and to 
compare the total cost between the interventions.  The study also aims to compare support 
mechanisms: Train-the-Trainer (TT) to Train-the-Trainer Plus (TT+) for the implementation 
of CATS and to test mediators and moderators of type of support (TT vs. TT+) on therapist 
fidelity.   
 
Study Design: This is a 3-arm, parallel group, Type 2 hybrid effectiveness-implementation 
trial in K-8 schools in Philadelphia over 5 years.  
 
Setting/Participants: Schools within the School District of Philadelphia that have STS 
programs managed by mental health agencies are potentially eligible for participation.  
Schools (3-6 per agency) will be chosen for participation.  Students who present excessive 
anxiety in grades 4-8 would be potentially eligible to participate in the interventions offered 
by the agencies’ therapists in their school. 
 
Study Interventions and Measures:  
This study is designed to test: a) the effectiveness of an adapted version of a group 
cognitive behavioral therapy program (CATS) compared to the original version of this 
program (FRIENDS), and b) the effectiveness of two implementation strategies for CATS: 
Train-the-Trainer and Train-the-Trainer Plus.  CATS is a shorter, more culturally sensitive, 
focused, and feasible CBT for anxiety in children than FRIENDS. CATS addresses one of 
the major barriers to implementation (intervention-context fit).  
 
Therapists and supervisors will be randomly assigned to one of three conditions: a) 
FRIENDS with Train-the-Trainer implementation strategy (i.e., Masters-level supervisors 
receive training by experts on conducting supervision); b) CATS with Train-the-Trainer 
strategy; c) CATS using the Train-the-Trainer Plus strategy (i.e., supervisors receive 
training and consultation by experts on conducting supervision). Effectiveness will be 
measured by comparing A to B; Implementation will be measured by comparing B to C. 
Agency therapists will conduct all treatment groups in the schools. 

 
  
 
 
 
1 
1 
BACKGROUND INFORMATION AND RATIONALE 
1.1 
Introduction 
Unmet need for mental health services remains extraordinarily high, especially among 
urban low-income children. Anxiety disorders in particular are highly prevalent among 
children in low-income, high-stress communities, and have been linked to academic and 
social problems and emotional maladjustment later in life.  Schools in low-income urban 
communities present a context with great potential for implementation of psychosocial 
EBPs. Cognitive-behavioral therapy (CBT) is an EBP that has been found to be very 
effective in treating anxiety in various settings, including schools. Friends for Life 
(FRIENDS) is an efficacious group CBT protocol for anxiety. Unfortunately, CBT for anxiety 
is seldom employed in under-resourced urban schools, mainly because the treatment 
protocols are not tailored for the context or the population, existing behavioral health staff 
do not receive adequate training or support to allow them to implement the treatment with 
fidelity, and school districts do not have the resources to contract outside providers or 
consultants.  Community Behavioral Health (CBH), a non-profit organization that manages 
Medicaid funds for the City of Philadelphia, has approached us for help to develop cost-
effective implementation strategies for schools. The availability of a contextually relevant, 
feasible, and cost-effective evidence-based CBT protocol for urban schools designed to be 
implemented and supported by existing agencies would address existing gaps in services 
for children who might otherwise go without treatment. 
  
1.2 
Name and Description of Investigational Product or Intervention 
1.2.1 Interventions implemented for anxious students by city agency 
therapists:  
 
Friends for Life (FRIENDS): FRIENDS, a GCBT program, is an effective program for the 
prevention and treatment of anxiety disorders in children evidenced by a meta-analysis on 
school-based interventions for at-risk and clinically anxious youth.  FRIENDS has been 
successfully implemented in several countries with children from diverse ethnic 
backgrounds. FRIENDS was developed based on the view that anxiety is a tripartite 
construct involving physiological, cognitive, and behavioral components.  
Core components: Experts in CBT for childhood anxiety disorders have identified 5 
essential components: psychoeducation, somatic management skills training, cognitive 
restructuring, exposure methods, and contingency management. The FRIENDS protocol 
consists of 10 weekly sessions and two booster sessions. We included the booster 
sessions in the regular protocol for a total of 12 sessions.   
 
Adapted CBT for Anxiety Treatment in Schools (CATS): We conducted planned adaptations 
to the FRIENDS protocol based on our collective experience with the protocol and 
qualitative data (focus group and interviews). We followed procedures developed by Lee 
and colleagues, including surveying service providers and trainers regarding the 
appropriateness of FRIENDS for the target population. Changes were made to the 
language (idioms, metaphors, words), cultural fit (cultural values), methods (session length, 
number of sessions), activities (in-session practices), which resulted in additions and 
substitutions in these areas while maintaining the 5 essential components of the treatment. 
CATS is a briefer (8-session) and more engaging and culturally-sensitive protocol than 
FRIENDS. 

 
  
 
 
 
2 
1.2.2  Support Interventions (strategies) offered to city agency therapists 
implementing CATS 
Train the Trainer (TT) by which agency supervisors are trained to conduct effective 
supervision, and then go on to train therapists. 
 
Train the Trainer Plus (TT+) a modified train-the-trainer approach by which supervisors 
receive training plus extended consultation on conducting effective supervision. 
 
1.3 Findings from Non-Clinical and Clinical Studies  
We pilot-tested FRIENDS with low-income urban children who were at risk or who met 
diagnostic criteria for an anxiety disorder. The treatment was part of a school-wide positive 
behavioral interventions and supports (SWPBIS) program (CDC, R18MN000003) in 2 K-8 
public schools in Philadelphia.  The development and implementation of these programs 
was based on the Public Health Approach.  The proposed project will take place in similar 
schools.  We interviewed the parents of 111 randomly selected children in order to assess 
mental health characteristics of the typical student attending the participating schools. On 
the Child Behavior Checklist, 28% of the children were rated within the borderline range or 
higher (T score > 60) on the Anxious/Depressed subscale and 9% were rated within the 
clinical range (T score > 70). The corresponding percentages for externalizing disorders 
were 29%, and 17%, respectively, indicating that the number of children at risk for anxiety is 
very similar to the number of children at risk for externalizing disorders.  
Six graduate students in applied psychology and three counselors from each school were 
trained in and co led FRIENDS groups. The fidelity scores reflected the ability of graduate 
students and school counselors to implement the interventions with fidelity while receiving 
expert support. Six FRIENDS groups were conducted. All of the sessions were video 
recorded to assess content fidelity. Fidelity ratings were obtained from an independent 
coder (IC). Average fidelity to the FRIENDS manual across sessions and groups was 86%, 
considered moderate to strong fidelity. Of the 33 children who participated, only 10 (30%) 
children participated in all 12 sessions.  Although only 30% of participants attended all 
originally scheduled sessions, 85% of participants received the entire content of the 
intervention. The procedure was for school counselors and project staff to review material 
with absent participants individually, or in groups of two, prior to the upcoming session. 
Following this procedure, 85% received all session content. 
The findings from the pilot study are relevant for the proposed study because they show 
that: a) the prevalence of anxiety problems in the schools where the proposed project would 
take place is high; b) less than a third of the children participated in all 12 session, 
indicating that a briefer protocol would be more feasible in urban schools; c) counselors 
functioning as co-therapists with graduate students can deliver FRIENDS with relatively 
high levels of fidelity; and (d) the treatment led to reductions in diagnostic level and 
symptom severity. 
Furthermore, in preparation for this project, we conducted qualitative interviews with 5 
consultants from our current SWPBIS project (R01HD073430) and focus groups with 
school counselors implementing FRIENDS. A qualitative theme analysis (i.e., frequency of 
themes) about consultants’ perceptions regarding context-fit of FRIENDS revealed that the 
protocol is too long, uses language and examples that are not congruent with the language 
and culture of low income, ethnically diverse children, and does not sufficiently emphasize 
teaching relaxation and exposure. Counselors thought the manual is “dry” and “boring” and 

 
  
 
 
 
3 
that it needs to be “tweaked” in order to increase children’s engagement. Counselors 
requested more optional activities to support main ideas in the manual. 
1.4 Relevant Literature and Data 
Anxiety disorders (i.e., Generalized Anxiety Disorder, Social Phobia, Specific Phobia and 
Separation Anxiety Disorder), affect up to 13% of the child population making them among 
the most common childhood conditions. Anxiety disorders are highly prevalent among inner 
city school children and often go unidentified and untreated. Children with these disorders 
are more likely than their peers to have problems with social, peer and parent-child 
relations, academic achievement school refusal and future socio-emotional adjustment. 
School factors, such as peer problems, academic pressures, and school violence, can 
contribute to and exacerbate symptoms.   
Individual and group CBT (GCBT) have been shown to be highly effective for the treatment 
of anxiety in youth. GCBT requires fewer resources than individual CBT, such as one 
therapist treating several children at once, thus making it less expensive for use in under-
resourced settings such as urban schools. GCBT for anxiety has successfully been 
employed in urban school settings by our team and others. FRIENDS, a GCBT program, is 
an effective program for the prevention and treatment of anxiety disorders in children as 
evidenced by a meta-analysis on school-based interventions for at-risk and clinically 
anxious youth. Also, our previous research demonstrates that FRIENDS is an effective 
protocol for use in urban public schools. 
Public schools have become a common setting for the delivery of mental health services to 
children and may be the ideal context through which to narrow services disparities. The 
school is a convenient location where services can often be provided at little or no cost to 
families. Benefits of providing EBPs in schools include the ability to implement interventions 
in the very environment in which most symptoms are triggered and to incorporate protocol-
specific interventions, with peer and teacher involvement, as needed for generalizability. 
EBPs for anxiety disorders enable overanxious children to overcome behavioral barriers to 
learning. 
The most pressing challenge facing the mental health field is the dissemination and 
implementation (D&I) of EBPs: taking protocols that were developed in research settings 
and adapting them for specific community use. It has been estimated that it can take up to 
17 years for new EBPs to make their way from research to practice. The delay is likely 
longer for child treatments in urban settings. The customary step-wise progression from 
clinical efficacy to clinical effectiveness in practice to implementation contributes to the long 
delay between the development of a treatment and its routine implementation in a 
community setting. Curran and colleagues have proposed three hybrid designs to galvanize 
the translation of efficacious treatments to enhance their public health impact. These 
research designs are thought to facilitate “more rapid translational gains, more effective 
implementation strategies and [yield] more useful information for decision makers. In the 
Type II design, effectiveness and implementation have equal importance. Type II 
effectiveness-implementation hybrid studies are most appropriate for treatments that have a 
strong record of efficacy and effectiveness, such as FRIENDS. 
Treatments that are developed and tested for efficacy in highly controlled research settings 
often do not fit into “real world” contexts. As a result most effective treatments in health and 
mental health settings need to be adapted for new contexts. Unfortunately, in the 
adaptation process, there is a potential for some of the main components of the treatments 
to be altered, rendering them less effective than in their original form. Lee and colleagues 
developed planned adaptation (PA) to address the tension between “implementing 

 
  
 
 
 
4 
programs with fidelity and the need to tailor programs to fit the target population”. PA 
orients the provider to the program and its theoretical orientation. A central tenet of PA is 
that adaptation must be conducted without altering the program’s core components. PA 
helps the provider identify components of the program that can be modified, and provides a 
process for conducting adaptations and a direction on how to develop evaluation strategies. 
We applied PA to FRIENDS in order to make it more feasible for implementation in co-
location settings with low-income urban children. 
Substantial resources have been dedicated to the D&I of mental health EBPs. Key to that 
effort has been the identification of strategies for training community-based clinicians. 
Studies have shown that effective training models include initial workshops for therapists 
that are followed up with supervision, which often includes coaching and performance 
feedback. This was the strategy employed by our research team for an R01 study testing 
the effectiveness of consultation only vs. consultation plus coaching in the implementation 
of an EBP in public schools in the SDP. Another important challenge facing publicly funded 
mental health agencies is finding cost-effective strategies for supporting supervisors. Most 
community mental health agencies, where low-income urban children receive services, do 
not have the resources to pay for ongoing expert direct consultation for therapists who 
implement EBPs. As such, finding time- and cost-effective alternative strategies for 
supporting therapists would have a significant public health impact. An important feasibility 
question is, “Can internal clinical supervisors who are not familiar with EBPs provide 
effective supervision to therapists after participating in training workshops or do they need 
more extensive support?” A mostly self-sustaining system as previously described would 
likely be more practical and less expensive but perhaps lead to fewer positive child 
outcomes compared to a system where supervisors are provided extended consultation. 
Two alternative, and potentially cost effective, strategies are the cascade or train-the-trainer 
(TT) approach, by which agency supervisors are trained to conduct effective supervision, 
and then go on to train therapists; and a modified train-the-trainer approach by which 
supervisors receive training plus extended consultation (TT+) on conducting effective 
supervision. TT has been used with adult mental health populations. Two pilot studies using 
this approach have reported improved therapist knowledge and improved client behavior, 
but it is not known whether these findings could generalize to other settings or populations. 
TT+ has been used by Bruce Chorpita and colleagues with agency supervisors who 
support therapists in the implementation of modular individual CBT. Highlights of this 
approach include initial training, biweekly or monthly phone consultations with supervisors 
and a supervisor promotion review.  This modified train-the-trainer approach has been 
found to lead to moderate outcome effect sizes. 
This current study has the potential to demonstrate that agency therapists and supervisors 
who have had little to no prior exposure to EBPs can implement an anxiety disorders EBP 
with fidelity. The use of group therapy can contribute to lowering services disparities in 
urban schools. Comparisons of the two implementation strategies (internal supervision with 
or without consultation for supervisors) would provide large urban mental health systems 
with data to make decisions about adoption of EBPs. The study is consistent with the goals 
of the Affordable Care Act in that it would generate information on implementation of 
effective practices that optimize delivery of mental health care to traditionally underserved 
communities.   
 
1.5 Compliance Statement 
This study will be conducted in full accordance with all applicable Philadelphia Department 
of Public Health Research Policies and Procedures and all applicable Federal and state 

 
  
 
 
 
5 
laws and regulations including 45 CFR 46. All episodes of noncompliance will be 
documented. 
 
The investigators will perform the study in accordance with this protocol, will obtain consent 
and assent (when appropriate), and will report unanticipated problems involving risks to 
subjects or others in accordance with The Children’s Hospital of Philadelphia and 
Philadelphia Department of Public Health IRB Policies and Procedures and all federal 
requirements. Collection, recording, and reporting of data will be accurate and will ensure 
the privacy, health, and welfare of research subjects during and after the study.  
2 
STUDY OBJECTIVES 
This study is designed to answer two important questions:  
 
1. Can publicly funded mental health therapists in urban schools successfully 
implement EBPs when supported by their own agencies or does successful 
implementation require external, expert support?  
 
2. What is the comparative cost effectiveness of these different implementation 
strategies? 
 
2.1 Primary Objective (or Aim) 
The primary objectives of this study are:  
• 
To compare pre- to post-treatment effectiveness of different CBT treatments 
(FRIENDS vs. CATS) and to compare the total cost between CATS and 
FRIENDS. 
  
• 
To compare Train-the-Trainer (TT) to Train-the-Trainer Plus (TT+) for the 
implementation of CATS. 
2.2 Secondary Objectives (or Aim) 
The secondary objective is to test mediators and moderators of type of support on 
therapist fidelity. 
3 
INVESTIGATIONAL PLAN 
3.1 General Schema of Study Design 
This is a 3-arm, parallel group, Type 2 Hybrid (effectiveness-implementation) design. The 
three groups are:  
 
A) FRIENDS with Train-the-Trainer (TT) implementation strategy (supervisors 
participate in training workshops on conducting supervision) 
 
B) CATS with TT strategy 
 
C) CATS using the Train-the-Trainer Plus (TT+) strategy (supervisors participate in 
training workshop and receive further consultation on conducting supervision).  
 

 
  
 
 
 
6 
The effectiveness part of the trial will compare A-B (different treatments); the 
implementation part of the trial will compare B-C (same treatment, different implementation 
strategies).  
 
Agency therapists will conduct all treatment groups. At the outset of the study, therapists 
will be randomly assigned to condition (A, B, C).  
 
3.1.1 Screening Phase 
Student Screening:  Students in grades 4-8 who are referred to the STS programs in the 
participating schools will be screened for signs of anxiety using the Screen for Child Anxiety 
Related Disorders (SCARED). The screening criterion is a total score > 25 or elevated sub-
scale scores (as noted by the scoring procedures) on the Screen for Child Anxiety Related 
Disorders (SCARED), completed by the student or a parent.  This will become regular 
standard of care anxiety screening procedures for students in the STS Programs.  In 
addition, STS agencies will also be trained and supported in screening students for 
traumatic life events using the Trauma History Questionnaire (THQ, caregiver and child 
report).  This practice will also be encouraged to become regular standard of care.  
Referrals will be made to the research study team on the basis of the SCARED.  For 
parents who are interested in having their children participate, their Therapist will share 
their contact information and SCARED/THQ scores with the research team after obtaining a 
parent signature to release information. 
 
If a student meets screening criteria on the SCARED (as noted above), they will be referred 
to the study and research staff will ask parents /caregivers if they are interested in having 
their child participate.  
 
Alternatively, STS Agencies may also refer students in grades 4-8 who are enrolled in or 
being evaluated for STS services to the study team.  Upon referral, the study team may 
contact parents via telephone to obtain consent for screening procedures.  With parental 
consent, the study team may conduct the SCARED Screener with students (in their school) 
in an effort to identify students who are eligible for participation in the study.  If identified as 
eligible, the study team will contact the parent to further explain the study and schedule a 
consent meeting.    
 
3.1.2 Phase I:  Training of Supervisors & Therapists assigned to all 3 
conditions 
Members of the research team will conduct 1-2 days of training each year (i.e., 5 times over 
the course of the grant) for all new agency therapists (including alternates) and supervisors. 
Training of supervisors and therapists will take place at The Children’s Hospital of 
Philadelphia (CHOP), at the site of a community agency, or at the school. Some of the 
training will be for supervisors only and some for supervisors and therapists together.  
Supervisors will learn a competency framework for supervisors, strategies for identifying 
children who could benefit from the service (e.g., conducting in-service presentations to 
faculty on signs of anxiety in children), and how to use an online platform in the 
school/community setting (e.g., wireless access; using fidelity forms) to conduct, guide and 
track supervision sessions remotely. They will also be trained on the use of fidelity 
monitoring and conducting performance feedback with therapists. Supervisors will learn 
how to prepare therapists for each treatment session and how to conduct performance 
feedback using fidelity forms.  

 
  
 
 
 
7 
Therapists and supervisors will be trained in using a screening instrument (SCARED) for 
the identification of potential participants. They will be introduced to a competency model 
for CBT for anxiety, which delineates generic therapeutic competencies (e.g., professional 
practice), CBT competencies (e.g., relevance of theory and research), and specific CBT 
techniques for anxiety (e.g., managing negative thoughts). They will also learn about how to 
deal with implementation barriers (e.g., scheduling sessions, conducting exposure tasks). 
Therapists and supervisors from the STS programs using FRIENDS (A) and the STS 
programs using CATS (B & C), will be trained separately for activities related to each 
protocol. Trainers will follow training procedures used in other EBP dissemination studies in 
nontraditional settings and other training strategies found to be effective (i.e., active 
learning such as modeling and role-playing). At the conclusion of the training, therapists 
and supervisors will be administered the Knowledge Test (KT) to measure knowledge of 
concepts taught during the training. Participants who score below 80% will be provided 
further training in the areas in which they scored low.  
 
3.1.3 Phase 2: Therapist Implementation of EBP (FRIENDS & CATS) & 
Implementation of Support Strategies (TT vs. TT+)  
STS therapists from the participating agencies will screen students who have been referred 
to the STS program and then refer students, who meet the screening criteria, to the 
research team.  If the child has screened positive on either the parent- or child-form of the 
SCARED, then he or she is eligible to participate in the study.  The research team will 
conduct further evaluations using rating scales to understand symptoms and impairment 
across different aspects of child behavior (e.g., depressive symptoms, coping skills, 
disruptive behavior, etc.).  STS therapists will conduct treatment groups (FRIENDS or 
CATS) in the schools during lunch periods, or a non-academic period during the school 
day.  Therapists will be expected to conduct FRIENDS or CATS groups with fidelity. 
The agency supervisors will be expected to conduct supervision with therapists. 
Supervisors will provide one 50-minute supervision session for each treatment session the 
therapist conducts (12 for FRIENDS, 8 for CATS). The session is divided into group 
preparation (i.e., discussing referrals, preparing for upcoming session, engaging in problem 
solving around implementation barriers), and coaching (i.e., performance feedback). Prior 
to the first session, project staff uploads video-recorded samples of effective 
implementation of the main components of the treatments, as recorded during a previous 
implementation of FRIENDS. Periodically, the supervisor encourages therapists to watch 
the video clips at their convenience. Also, prior to the session, the supervisor will watch a 
recording of the previous session and select clips for discussion. During the coaching 
portion of the session, the supervisor will ask for self-reflection of previous sessions and 
provide performance feedback. Then, the supervisor will provide the therapists with (a) 
fidelity data and (b) video clips bookmarked from the previous session showing effective 
and ineffective implementation of FRIENDS/CATS. 
Simultaneously, TT and TT+ will be implemented by the research team to support the 
agency supervisors in supporting their therapists.   
If substantial time (45-90 days) has lapsed between training and the start of intervention, 
Supervisors will be offered a booster training session after the original training in the weeks 
before starting their group. The purpose of the booster session is to review key points from 
the training (e.g., conducting performance feedback) and to answer questions about any 
aspect of the supervision process. This subsequent training opportunity for agency 

 
  
 
 
 
8 
supervisors (on how to be effective supervisors to the implementing therapists) will be 
conducted remotely, or in person.   
Supervisors in TT+ (Group C) will receive 8 consultation sessions in the first year using the 
online platform. The consultation sessions will center on helping the supervisor conduct 
supervision, and will include performance feedback, implementation fidelity, problem solve 
implementation barriers and will answer supervisors’ questions about the supervision 
manual. We will monitor whether supervision with therapists takes place regularly for the 
expected length of time and whether therapists implement the treatments with fidelity. This 
information will be obtained from group fidelity scores.  
Supervisors will be instructed to supervise therapists based only on what they have learned 
in the consultation sessions. 
 
 
3.2 Study Duration, Enrollment and Number of Sites 
3.2.1 Duration of Study Participation 
Agency Supervisors:  There is no time limit placed on the duration of participation for 
agency supervisors.  They may continue working with the research team for the duration of 
the study (5 years) or as long as their agency remains involved in the study or as long as 
they remain employed by the participating agency.   
Agency (STS) Therapists Implementing EBP Interventions: Therapists are asked to 
commit to participate in the study for one school year.  After the initial year, the study team 
would prefer to train a new therapist in an effort to minimize the effects of differences in 
therapist experience and therapist dropout on outcomes (yearly therapist dropout in the 
agencies is 25%).  However, based on agency need and staff consideration, therapists can 
participate in additional years at the study team’s discretion.  This will be determined on a 
case-by-case situation.  Therapists will be free to continue to implement the intervention 
groups outside the research project after participation in the study has concluded.   
Students:  Students who are determined to be eligible, based on the SCARED, and for 
whom consent/assent is obtained, will participate in a 30 minute baseline data collection 
session and then in the intervention which will consist of 12 (FRIENDS) or 8 (CATS) 40 
minute (or one-class period) intervention sessions, conducted at school.  They will also 
participate in a similar post-treatment data collection session once the intervention (12 or 8 
sessions) is completed.  Parents of the students participating in the intervention will also 
participate in two 30-minute data collection sessions (completion of the CBCL, CDI, and 
MASC-2, see section 5.2 below).   
 
3.2.2 Total Number of Study Sites/Total Number of Subjects Projected 
The study will be conducted in a minimum of 36 (potentially up to 41) public K-8 schools 
within the School District of Philadelphia and in collaboration with community mental health 
agencies, contracted by CBH, in the City of Philadelphia.  However, not every school will 
participate during every project year.  It is anticipated that schools will participate in 2 
project years.  
We expect there to be a minimum of 36-41 agency supervisors (one per school at all 
times).  However, there is a 17% drop-out rate of supervisors across agencies.  Therefore, 
depending on supervisor turnover, the number of enrolled supervisors will be comparable. If 

 
  
 
 
 
9 
all therapists participate once per school year, we’d expect to enroll 90 agency therapists (1 
therapist per school (n=18), per year of the study (n=5) = 18x5 = 90).  However, given that 
therapists may run a second intervention group, we could end up with fewer therapists than 
expected.  Yearly therapist drop out in the agencies is 25%.  Therefore, depending on 
therapist turnover within the agencies, the number of enrolled therapists will also be 
comparable.     
Each of the participating mental health agencies will make available extra supervisors and 
therapists to serve as alternates in case a supervisor or therapist drops out of the study or 
terminates employment with the agency for any reason. Alternate supervisors and 
therapists will be invited to attend the initial trainings. If a supervisor/therapist moves to 
another school within the same study arm, s/he will be allowed to remain in the study and 
will be treated as a returning supervisor. Supervisors/therapists in conditions A and B who 
move to a school assigned to a different study arm can participate in the project pending PI 
review. Supervisors/therapist in condition C who move to a new school assigned to 
condition A or B will not be allowed to continue in the study due to potential condition 
contamination.  They will be counted as drop outs.  
Given agency structure and high staff turnover rate, consented therapists and supervisors 
may switch roles in an additional study year dependent on randomized study arm.  Similar 
to procedure stated above, any therapist or supervisor that switches a role in the study in 
condition A or B can participate in any other condition. If the participant is in condition C, 
then they can only participate in condition C regardless of a change in school or agency 
during the duration of the study.   
We expect each of the therapists to run one intervention group each per year and we 
expect each group to be composed of 4-6 eligible and consenting students. Therapists can 
run a second group if time permits but is not expected for participation. Therefore, we 
expect to enroll 360 students in the intervention sessions.  
Given that some referred students will not meet inclusion criteria and that some will not 
consent or not attend group, we expect that we will need to consider 426 referred students 
to yield 360 participating students (based on a 15% attrition rate). 
 
3.3 Study Population 
3.3.1 Inclusion Criteria 
1) Mental Health Agencies: Any CBH supported agency with 2 or more clinical supervisors 
implementing STS Programs in Philadelphia K-8 Schools.  
2) Supervisors: Any participating agency clinician with a Master’s degree or higher in a 
mental health field, as indicated by the agency and confirmed by self-report on the Staff 
Demographic form. 
3) Therapists: Any STS therapist with a Master’s degree or higher who provides services 
in one of the 18 participating schools. 
4) Students: Any student enrolled in grades 4-8 in one of the participating schools, who 
enrolled in the STS program in their school and who meets study screening criteria. The 
screening criterion is a total score > 25 or elevated sub-scale scores (as noted by the 
scoring procedures) on the Screen for Child Anxiety Related Disorders (SCARED) 
completed by the student or a parent.  
    

 
  
 
 
 
10 
3.3.2 Exclusion Criteria 
1) Mental Health Agencies with less than 2 clinical supervisors and those not 
implementing STS programs within the School District of Philadelphia.    
2) Student, supervisors, or therapists not involved in STS.  
3) Students with Special Education classification of “Intellectual Disability.”  
4) Students with current school refusal (>15 missed days of school in current semester), a 
history of psychotic or autistic spectrum disorders as reported by school or agency staff 
at the time of referral and confirmed by parent/legal guardian at the time of baseline 
data collection. 
5) Students who present with diagnoses that make participation in the study clinically 
inappropriate in that they warrant more intensive, outpatient or potential inpatient 
intervention, (i.e., severe, current, major depressive disorder, PTSD, or bipolar disorder) 
or who present as in acute risk to themselves or others during screening process or 
baseline data collection.  
*Students meeting criteria in #2, #3 & #5 above will be excluded from participation because 
they would be unlikely to benefit from GCBTS. 
Subjects that do not meet all of the enrollment criteria may not be enrolled. Any violations of 
these criteria must be reported in accordance with IRB Policies and Procedures.  
4 
STUDY PROCEDURES 
4.1 Student Screening / Referral Process  
Note Section 3.1.1. 
 
At the beginning of each project year, we will offer each participating school an in-service 
training for teachers and/or behavioral support staff on recognizing warning signs of 
excessive anxiety in children. Following the in-service, STS therapists will ask teachers to 
refer any child showing signs of anxiety to the STS program. Either students or parents will 
be asked to complete the SCARED and THQ for identified students. These are regular 
standard of care screening procedures, so we will not collect these data for research 
purposes. If a student meets screening criteria (a total score > 25 or elevated sub-scale 
scores as noted by the scoring procedures on the SCARED completed by the student or a 
parent), the STS therapist will refer the family to the research team. The study team may 
also assist in collecting the SCARED Screener from students, in cases where parental 
consent is obtained.  In these cases, the THQ will only be collected if the student is eligible 
and the parent/child consents/assents to the full study.  
4.2 Intervention Implementation (For Students) 
Families (students and caregivers) who consent to participate in this intervention, will be 
enrolled in one of two programs (FRIENDS or CATS, depending on their school’s 
randomization), over the course of several weeks.  The students will participate in the 
program during lunch or a non-academic period during the school day. The programs will 
be implemented by the student’s school-based STS Therapist. Refer to section 1.2.1 for a 
description of the programs. 

 
  
 
 
 
11 
Hectic periods of the year when student absenteeism and class trips are more common (1st 
and 4th Quarters) as well as several class breaks, including winter holidays (approximately 
10 days), snow days, Spring Break (1 week) and state mandatory testing of students (2 
weeks) will interrupt group sessions. We plan for the interventions to take place primarily 
during the 2nd and 3rd quarters to eliminate major interruptions in the 1st and 4th quarters. We 
acknowledge that interruptions might affect the interventions differently, partly because one 
of the interventions is longer than the other (8 vs. 12 sessions). We were not able to locate 
a measure that would capture these factors. However, differences in length will be analyzed 
by adding up the number of days dedicated to instruction for each school when groups are 
held and divide it by 5. We will then add up the weekly average for each subsequent week 
and divide it by 8 (CATS) or 12 (FRIENDS). This will give us an index of instruction time for 
each treatment, which we will then compare and use in covariate analysis. 
4.3 Implementation of Support Strategies 
Throughout the duration of the program, implementing therapists will engage in regular 
supervision with their agency supervisor.  Depending on randomization, the research team 
will support the agency supervisors through the implementation of TT or TT+ (refer to 
section 3.1.3 for a description of “supervision” and TT vs. TT +).  
4.4 Subject Completion/Withdrawal 
Subjects may withdraw from the study at any time without prejudice to their care, education 
or employment.  They may also be discontinued from the study at the discretion of the 
Investigator for lack of adherence to the study and/or development of exclusionary criteria.  
Student participants will be removed from the study if they develop exclusionary criteria 
during the trial (e.g. Bipolar Disorder I, acute suicidality, psychotic symptoms, etc) or if it is 
determined that the intervention group is not age/developmentally appropriate based on the 
grade level of the majority of enrolled students. As examples, acutely suicidal participants 
would be excluded from study participation, as would participants who require medical 
attention for a severe medical illness.  It will be documented whether or not each subject 
completes the clinical study. If the Investigator becomes aware of any serious, related 
adverse events after the subject completes or withdraws from the study, they will be 
recorded in the study files. 
5 
STUDY EVALUATIONS AND MEASUREMENTS 
5.1 Screening Measurements (Students) 
Potential student participants will be screened using the SCARED. The SCARED is a 41-
item questionnaire used to screen for children with anxiety disorders, using five sub-scales 
and a Total Score. It has excellent psychometric properties and has been used in 
community settings as a screening instrument for anxiety disorders. Therapists will ask 
students or parents to complete the SCARED if referred to or enrolled into STS Programs.  
 
STS agencies will also be encouraged to collect information about Traumatic Life Events 
using the Trauma History Questionnaire (THQ).  The THQ is a 23-item self-report measure 
for children that examines experiences with potentially traumatic events such as crime, 
general disaster, community violence, and sexual and physical assault using a yes/no 
format.  For each event endorsed, respondents are asked to respond “yes” or “no” if they 
have experienced or witnessed the event. The THQ is used in research and clinical settings 
and is available in English and Spanish. 
 

 
  
 
 
 
12 
For parents who consent to the sharing of this information, the study team will collect this 
information, as well as the SCARED.  However, the THQ will not be used as a screening 
tool.  Students will be referred to the study solely on the basis of the SCARED.     
  
5.2 Treatment Efficacy Evaluations (Information Collected about the 
Students enrolled in FRIENDS or CATS)  
Multidimensional Anxiety Scale for Children (MASC-2).  The MASC-2 is a 50-item rating 
scale for the assessment of anxiety symptoms in children. The MASC 2 has strong 
psychometric properties and is sensitive for measuring treatment effects. We will use the 
Total Anxiety score on the parent and child forms. 
Self-Report Coping Measure for Elementary School Children (SRCM). The SRCM is a 34-item 
self-report measure with good psychometric properties that is used to assess children’s coping 
strategies (Seeking Social Support, Self-Reliance/ Problem-Solving, Distancing, Internalizing, 
and Externalizing).  The Total score for the SRCM will be used in outcome analyses. 
Children’s Depression Inventory – 2nd Edition (CDI-2). The CDI-2 is a child self-report 27-item 
instrument that measures the cognitive, affective and behavioral signs of depression. The CDI 
has high internal consistency, moderate retest reliability, and correlates with measures of 
related constructs such as self-esteem, negative cognitive attributions, and hopelessness. We 
will use the Total CDI score. 
Child Behavior Checklist (CBCL). Completed by parents, this 118-item multi-axial rating scale 
provides a measure of children’s (ages 6-18) internalizing and externalizing difficulties, as well 
as social and academic competencies. Good psychometric properties have been reported. We 
will use the Anxious/Depressed Somatic Complaints, Rule-Breaking Behavior and Aggressive 
Behavior scales in outcome analyses. 
Engagement versus Disaffection with Learning: Teacher & Student Report.  This is a 20 
item teacher and student report measure which includes four sub-scales: Behavioral 
Engagement, Emotional Engagement, Behavioral Disaffection and Emotional Disaffection. 
Validity of scores is supported by findings that teacher reports are correlated with student 
reports, with in vivo observations in the classroom, and with markers of self-system and 
social contextual processes. As such, these measures capture important features of 
engagement and disaffection in the classroom. 
*All measures noted above are validated / standardized. 
5.3 Implementation Outcome Measures   
All treatment sessions (across groups A, B and C) will be video recorded. Implementation 
will be measured by coding twenty-five percent of sessions by 2 Independent Coders (IC’s). 
IC’s will be Master’s level research staff members or advanced doctoral students in 
Psychology who have been trained to a reliable standard. Coder reliability will be checked 
quarterly. Kappa coefficients will be computed; a minimum of .85 will be required for each 
code. Retraining will be conducted should rater agreement fall below .85. For the qualitative 
content analysis.  Coding differences of within 1 point between the 2 ICs will be considered 
an agreement. We will report the average score between the two ICs. If the difference is 
greater than 1 point, a third IC will independently rate the session and the average of the 
three ICs will be reported.  The IC’s will code the videos using the measures below (and 
attached to section 12 of the eIRB application): 
• 
Content fidelity (how well therapists delivered the content as originally intended by 
the treatment developers), which is our main fidelity measure, will be assessed 

 
  
 
 
 
13 
using the Content Fidelity Checklist (CFC). The CFC will reflect each activity 
component of the session agenda or outline of the original and adapted treatment 
protocols. Raters will use a yes-no response scale to indicate whether or not an 
STS therapist covered a particular component.   
• 
Process fidelity (delivering treatment content while actively engaging clients in 
therapy and doing it with a sense of competence) will be measured using an 
instrument based on a 12-item measure developed by John Lochman and 
colleagues (e.g., Counselor stops to clarify material; counselor’s tone is warm and 
positive; stimulated discussion; stopped to clarify material) rated on a 5-point scale 
ranging from “not at all” to “very often.” We will use the total score (Overall Process 
Fidelity). This measure has excellent internal psychometric properties.  
Each supervision/consultation session will also be audio/video-recorded.  After each 
session, supervisors will complete the Supervision Rating Scale (SRS) and the consultants 
(TT+) will complete the Consultation Rating Scale (CRS). Our team developed both 
measures. The SRS and CRS measure supervision/consultation content and process as 
detailed in the training manual (e.g., preparing for next session; Demonstrated empathy and 
provided positive reinforcement).   
The supervisions/consultation sessions will also be coded for interruptions and session 
length. Service interruption is defined as the degree to which therapists are fully available 
for supervision and treatment sessions. We will derive three scores averaged over the total 
number of sessions: a) Projected length of the session in minutes minus time spent on 
interruptions (e.g., answering a phone call, talking about unrelated topics); b) Number of 
interruptions; c) Total number of supervision / consultation and treatment sessions per 
group. 
To measure differences between support strategies (TT vs. TT+), agency supervisors (12) 
and therapists (60) in all groups, including participants who did not complete a treatment 
group but were consented and trained to participate, will be interviewed regarding their 
perception of the appropriateness and acceptability of the support they receive. A study 
staff member will conduct open-ended, semi-structured interviews about the process, 
feasibility and acceptability of the type of support they receive. Interviewers will follow a 
script to ask questions of supervisors and therapists (e.g., “How acceptable is the level of 
support you received for the implementation of the FRIENDS protocol?”; “Was the 
supervision you received from your agency’s supervisor appropriate for running groups with 
children?”; “Was the consultation you received from the research team appropriate for 
preparing you to conduct supervision with STS therapists?”)  These interviews will be 
conducted at the end of each academic year, or once the treatment groups have 
completed. Qualitative methods will be used because there are currently no 
psychometrically validated measures that address these areas.   
To evaluate potential mediators and moderators of type of support on implementation 
fidelity, we will use the Index of Inter-professional Team Collaboration for Expanded School 
Mental Health (IITC-ESMH). The IITC-ESMH is a 26-item measure rated on a 5-point 
Likert-type scale (1=never to 5=always) designed to measure four organizational 
constructs: Reflection on Process; Professional Flexibility; Newly Created Professional 
Activities; Role Interdependence. The measure has excellent psychometric properties. We 
will use the Total Score in mediator analyses.  All school staff members who participate in 
referring students to the STS programs will be invited to participate in the completion of this 
measure.  For those school staff members who are not study participants, an information 

 
  
 
 
 
14 
sheet will be provided to them explaining their participation in the completion of this 
measure.  
An additional measure of mediator and moderators of type of support will include a 
measure of intention, specifically therapists’ intention to conduct treatment sessions and 
participate in supervision. These data will be used to 1) test for differences between the 
conditions. This is an important secondary outcome because high intentions to use specific 
evidence based practices are necessary but not sufficient for therapists do use evidence-
based practices. These data will also 2) explain why child outcomes do (or don’t) differ: 
Intentions of therapist therapist behavior  child outcomes.  We will measure therapists’ 
intentions for each condition after training, and two more times during treatment. 
6 
STATISTICAL CONSIDERATIONS  
Variables pertinent to the study aims will be examined for skewed distributions and/or not 
meeting the assumptions of normality. For such variables, an appropriate method for 
variable transformation will be selected and applied to stabilize outcomes and their 
variances and to approximate the normal distribution. The geometric mean, the log 
transformation and the Box-Cox transformation are examples of such functions. In addition, 
we will consider non-parametric tests such as the Mann-Whitney test for comparing 
pre/post changes between two independent groups. Continuous variables will be 
summarized and described using means, medians, ranges, and standard deviation. 
Categorical data will be presented by frequencies and percentages. All summaries will be 
presented by pre and post intervention and by groups. Since randomization was done at 
the school level, the potential for imbalances at pre-measurements between students who 
will be part of FRIENDS or CATS groups are anticipated. We will first determine whether 
student’s related demographics, diagnostic, mental health, academic functioning, and 
therapists’ level of experience are balanced in both conditions. We will account for any 
differences in the analysis by including such variables as covariates. 
6.1 Primary Endpoint 
Effectiveness between treatment groups (FRIENDS and CATS) will be tested by calculating 
the 95% confidence limits (the width of the limit =|4.20|) for the differences in symptom 
severity, impairment, and academic competence (child outcomes) from pre to post between 
children receiving FRIENDS and children receiving CATS. If the calculated differences are 
within the 95% limit, then we will conclude that the two interventions have an equal impact 
or effect on child outcomes. If any of the differences in child outcomes is/are larger than the 
upper limit, then we will conclude that one intervention is better than the other. The limit of 
|4.20| was chosen based on the distribution of the reported mean effect sizes associate 
with FRIENDS. 
 
Qualitative interviews with supervisors and therapists (perceived acceptability of support 
strategies) will be audio-recorded, transcribed and entered into NVivo 10.0 for coding and 
analysis. For each interview, a codebook (of common themes related to the 
appropriateness and acceptability of the support they received) will be developed and 
trialed by three coders based on an initial reading of the transcripts, and revised through 
discussions and subsequent transcript reviews. Coders will meet regularly to compare 
coding and revise the codebook. This will yield a stable set of qualitative codes that will be 
used to code the transcripts. Two independent coders will then code all of the transcripts. 
This iterative process will be continued until consensus is reached and saturation occurs. 
Coder reliability will be checked quarterly using the inter-rater reliability function in NVivo. 

 
  
 
 
 
15 
Kappa coefficients will be computed; a minimum of .85 will be required for each code. 
Retraining will be conducted should rater agreement fall below .85.  In order to aid with 
interpretation of results, IE’s will randomly select an additional 35% of the interviews and 
code them, as a measure of inter-rater-reliability.  We will conduct a structured content 
analysis for perceived acceptability and appropriateness. The unit of analysis is each 
interview. We will create a case-by-variable matrix from the codes. The matrix will have an 
interviewee identifier number, type of support (i.e., TT, TT+), school, agency, subject 
demographic information, and codes. The code cells will contain the number of times the 
participant emitted words or phrases pertaining to perceived acceptability and 
appropriateness of the support they received. Once acceptability and appropriateness are 
defined for each interview, percentages for acceptability (% acceptance) and 
appropriateness (% appropriate) out of the total number of words will be calculated. 
Comparison of % acceptance and % appropriate between TT, TT+ will be conducted using 
the Wilcoxon sum rank test. 
 
Datasets from the REDCap and NVivo databases will be combined using the statistical 
software SAS. Each session will be considered to be a “case” in the qualitative database 
and will be assigned a unique identifying number that will link it (anonymously) to the 
quantitative database and individual session information such as fidelity scores.  
 
To measure the differences of implementation between therapists whose supervisors 
receive TT vs. TT+, means of content fidelity will be calculated with its 95% Confidence 
interval. Similar analysis will be done regarding process fidelity. 
 
6.2 Secondary Endpoints 
Costs will be assessed by use of generalized linear models (GLM). Links and families for 
the GLM will be empirically fit to the data using diagnostic tests including the Modified Parks 
test, Pregibon-Link test, Hosmer-Lemeshow test, and Pearson’s correlation test.  Measures 
of effectiveness will be derived from the “clinical” analysis of the trial data. Standard errors 
will be derived from a nonparametric bootstrap. Point estimates for the cost-effectiveness 
ratios will be based on the point estimates for the difference in costs and outcomes. 
Confidence intervals for the cost-effectiveness ratio as well as an acceptability curve will be 
derived using the point estimates of the differences, their standard errors, and the 
correlation between the differences. Acceptability curves will allow agency administrators to 
judge whether the gains in children’s outcomes/fidelity are sufficient to justify any increases 
in cost. 
 
6.3 Sample Size and Power 
Number of students and statistical power. It is anticipated that a total of 284 (142 students 
per treatment protocol) will complete the proposed study. The published reported effect size 
of FRIENDS has been estimated to equal 0.56 (cohen’s d). In testing for equivalence of 
FRIENDS and CATS, using two one-sided tests with sample sizes of 142 in FRIENDS and 
142 in CATS achieves 88% power at a 5% significance level when the true difference 
between the means is 0 (null hypothesis), the standard deviation of the differences in 
means is assumed to be 11, and the equivalence limits are  
-4.20 and 4.20.  
 
Number of therapist and statistical power. To compare TT to TT+ for the implementation of 
CATS (H2a), with 30 therapists in TT and 30 in TT+ achieve 86% power to detect a 

 
  
 
 
 
16 
difference of 8% between the null hypothesis that both group means are 80% (content 
fidelity) and the alternative hypothesis that the mean of group TT is 80% with estimated 
group standard deviations of 10 and 10 (an estimated effect size=.8) and with a significance 
level (alpha) of 0.05 using a two-sided two-sample t-test. Similarly, group sample sizes of 
30 and 30 achieve 84% power to detect a difference of 0.7 in process fidelity between the 
null hypothesis that both group means are 4 and the alternative hypothesis that the mean of 
group TT+ is 4.7 with estimated groups standard deviations of 0.9 and 0.9 (an estimated 
ES= .78) and with a significance level (alpha) of 0.05 using a two-sided two-sample t-test.  
 
7 
SAFETY MANAGEMENT 
7.1 Clinical Adverse Events 
Clinical adverse events (AEs) will be monitored throughout the study.  
7.2 Adverse Event Reporting 
Since the study procedures are not greater than minimal risk, SAEs are not expected. If any 
unanticipated problems related to the research involving risks to subjects or others happen 
during the course of this study (including SAEs) they will be reported to the IRB in 
accordance with CHOP IRB SOP 408: Unanticipated Problems Involving Risks to Subjects. 
AEs that are not serious but that are notable and could involve risks to subjects will be 
summarized in narrative or other format and submitted to the IRB at the time of continuing 
review.  
For students enrolled in the treatment sessions:  Careful baseline screening will be 
conducted to identify participants whose risk for potential adverse effects are elevated were 
they to utilize the study treatment.  If a participant becomes more symptomatic (based on 
parent/child report or agency/clinician judgment), the STS provider will review the 
participant’s clinical status with the project team and PI.  Concerns about deterioration will 
trigger an evaluation by the STS Therapist and then discussion with the PI regarding whether 
discontinuation from the study is in the best interest of the student.  Such participants will be 
removed from the study and treated openly with appropriate clinical care.   
In the event that suicidality is noted during the course of the eligibility evaluation or during 
participation in the study, CHOP evaluator will communicate finding to STS therapist to ensure 
that this information is incorporated into their standard care of practice.  CHOP evaluator will 
also inform on-call CHOP-based licensed clinical provider to ensure any other potential safety 
measures that may need to be put in place to provide appropriate care, including but not limited 
to informing parents, STS staff, and other professionals involved with the child’s mental health 
care. If CHOP staff learns or observes of any changes in a child’s mental health status that 
would affect their eligibility status while participating in treatment groups, the child may be 
withdrawn from the study at the discretion of the PI. All eligibility decisions will be made in 
collaboration with the STS team.   
The procedures of reporting adverse events include immediately notifying their supervisor 
and the PI and following procedures to maintain participant safety. Study therapists will 
have the PI’s cell phone number and beeper or cell phone contact information for their 
clinical supervisor. Children and guardians will also be given emergency contact information 
should they require assistance or in the case of an emergency. 

 
  
 
 
 
17 
8 
STUDY ADMINISTRATION 
8.1 Treatment Assignment Methods 
8.1.1 Randomization 
Selection and Randomization of schools, therapists and supervisors to intervention 
conditions: 
At least thirty-six schools (potentially up to 41) from mental health agencies will participate 
in this study. Computerized randomization lists will be generated with random assignment 
of schools (and therefore, the schools’ therapists and supervisors of the agencies 
participating in this study) to Condition A (FRIENDS/TT), B (CATS/TT), or C (CATS/TT+) 
with the goal of exposing all agencies to each of the three conditions over the course of the 
study.   
 
Schools will be stratified according to their size (greater than 625 students (yes/no)). 
Therefore, a list of 18 schools (including their therapists) will be randomly assigned to the 
treatment groups and will participate in year 1 of the study (Randomize 1).   Schools listed 
in Randomize 1 will also be used in year 2 while recruiting new therapists and their 
supervisors (potentially repeat supervisors). We expect this to add 6 new therapists (and 
their supervisors), per intervention condition, to the study.  Schools spanning elementary 
and middle school grades will run interventions with middle school grades in year 1 and 
elementary grades in year 2.   
   
From the remaining 18 schools (not randomized yet), a stratified new randomization list will 
be generated for year two and to be used again in year four (Randomize 2) of the study. As 
a result, a group of six new therapists and their supervisors (potentially repeat supervisors) 
per intervention will be recruited for year 3 and a new group of six therapists and their 
supervisors will be recruited for year 4.  Schools spanning elementary and middle school 
grades will run interventions with middle school grades in year 3 and elementary grades in 
year 4. 
 
For the 5th year, from the original list of 36 schools using the same stratified randomization 
method, a new randomization list of schools (Randomize 3) will be generated to recruit a 
new group of 18 therapists and their supervisors (6 per treatment group). Therefore, a new 
group of 18 therapists per year (a total of 90 therapists) will participate in this study.  Each 
therapist will participate only one time in this study. In schools spanning elementary and 
middle school grades, the study conditions will be evenly distributed across grade levels.  
  
The three randomization lists may result in assigning the same supervisors to different 
treatment groups. This may result in modifying the intervention effect due to the 
supervisor’s knowledge gained by participating in a previous different treatment arm. 
Therefore, working with the agencies, we will make sure that supervisors will stay in the 
same support condition (i.e.: A or B; C) they were originally assigned using randomize 1 
and randomize 2.  
New schools will replace dropped schools by matching stratification variables (size and 
type).  

 
  
 
 
 
18 
8.2 
Data Collection and Management 
The data collection and management plan is consistent with CHOP Policy A-3-6: 
Acceptable Use of Technology Resources that defines the requirements for encryption and 
security of computer systems.  
• 
Identifiable data will be collected as part of this study.  This data includes full 
names/initials, date of birth, address, telephone number, e-mail addresses.  
However, participants will be identified by alphanumeric code only.  This 
precautionary step allows for the electronic transfer of data without using data 
encryption techniques. At each stage of data collection and maintenance, measures 
are taken to ensure that all identifying information is taken out of data archives, and 
any hard copies of data that could identify participants are stored in locked file 
cabinets with restricted access, and that data files are password protected.  
Participant identification numbers are used that do not reveal the identity of 
participants (e.g., no use of birth dates, initials, social security numbers, etc). Only 
members of the research team will have access to the data.  If the results of this 
study are presented at scientific meetings or published in professional journals, they 
will not contain information that could be used to identify parents, teachers, students 
or agency staff members.  
• 
Hard data will be kept in a locked file cabinets at the Roberts Center for Pediatric 
Research.  Digital video and audio recordings (containing participant identifiers) will 
also be stored in locked file cabinets at CHOP.  The files will be transferred and 
stored to a CHOP computer network drive and analyzed by study team members for 
the coding of integrity/fidelity scoring.  After all analyses are complete, the files will 
be destroyed and personal identifiers will not be retained with the data.  All 
computerized study databases for questionnaire data will be kept on a secure 
Windows NT server located at one of the hospital’s research buildings. This server 
is also protected by a firewall to reduce the risk of unauthorized access to study 
information. 
• 
Data collected as part of this study will be entered (data entry from CRFs or upload 
from Excel files) and stored using REDCap (Research Electronic Data Capture) 
database, a secure web-based software database supporting clinical and 
translational research databases. The database will be password-protected, stored, 
and backed up on a daily basis by CHOP’s Research Institute.  REDCap provides 
data management functionality; including automated export procedures for 
seamless data downloads to Excel and commonly used statistical packages (SPSS, 
SAS, Stata, R). The database will incorporate range checks and between-variables 
consistency checks to ensure quality control. The system will signal the presence of 
questionable or potentially incorrect items.  After data cleaning and quality 
assurance procedures are completed, pertinent sets of data will be converted into 
SAS format for statistical analysis. The Biostatistics and Data Management Core 
(BDMC) of the Research Institute will provide staff expertise in REDCap database 
design and implementation, randomization, and statistical analysis under the 
direction of the study’s biostatistician. 

 
  
 
 
 
19 
8.3 Confidentiality 
All data and records generated during this study will be kept confidential in accordance with 
Institutional policies and HIPAA on subject privacy and that the Investigator and other site 
personnel will not use such data and records for any purpose other than conducting the study.  
The following steps will be taken to maintain confidentiality:  (1) the interview protocols and case 
record forms will be transported from schools to the research offices at CHOP in HIPAA 
compliant lockable bags and kept in a locked file cabinet in a locked office; (2) subject identity 
will be de-identified using numbers keyed to a master list; (3) de-identified data will be entered 
directly into files that will be password protected; (4) all project staff will be trained in the 
importance of confidentiality, and will certify in writing to protect subject confidentiality; and (5) if 
the results of the study are published, data which might reveal the identity of any particular 
subject will be disguised.  Subjects and their families also will be informed about the limits of 
confidentiality, e.g., in cases of danger to self or others or when previously undocumented 
physical or sexual abuse is discovered.  No identifiable data will be retained or used for future 
studies. The investigator will obtain a data use agreement between the provider (the PI) of the 
data and any recipient researchers (including others at CHOP) before sharing a limited dataset 
(PHI limited to dates and zip codes).  
8.4 Regulatory and Ethical Considerations 
8.4.1 Data and Safety Monitoring Plan 
The PI (Dr. Eiraldi) will assume overall responsibility for monitoring the integrity of data 
collection and the safety of the interventions. The PI will meet on a weekly basis with 
members of the core research team, which has responsibility for data collection and 
supervising the STS clinical supervisors who will also oversee data collection conducted by 
STS therapists.  The purpose of the meeting will be to monitor and ensure data collection 
and safety. Each member of the research team will be informed of his or her responsibility 
to ensure the integrity of data collection and the safety of children and families participating 
in the study. All adverse events will be reported immediately to the IRBs at CHOP, the City 
of Philadelphia and the Philadelphia School District and to NIMH. The IRB at CHOP will 
inform NIMH about actions taken in response to the reporting of adverse events. 
8.4.2 Risk Assessment 
Risks are not greater than minimal.  There are no known physical or legal risks to participating 
in the study interventions.  
Potential Risks Associated with Assessment.  Masters level clinicians, Bachelors and 
Masters level research staff and undergraduate research staff will collect rating scales from 
parents, children and teachers for screening, to determine eligibility and to assess 
treatment outcomes. Children, parents and teachers could develop mild emotional 
discomfort or frustration associated with filling out questionnaires.  In the event that a child, 
parent or teacher becomes upset or irritated as a result of filling out the questionnaire, the 
clinicians or research staff collecting the data will try to determine the reason for the 
discomfort and provide any assistance that the informant might need.  If an informant 
expresses a desire to not complete the questionnaire, clinicians and research staff will no 
longer ask the informant to complete the questionnaire.   
Potential Risks Associated with FRIENDS.  There are no known physical or legal risks to 
participating in the study. One risk associated with the treatments, as with all therapeutic 
interventions, is that it might not be effective for every participant that enters treatment. If, 
after participation in the original or CATS treatment, no improvement in symptoms is noted, 

 
  
 
 
 
20 
or, if deterioration is noted during treatment (based on self-report, parent-, child-, teacher-
report or clinical judgment), or if a family is interested in further treatment, we will work with 
them to identify appropriate referrals. This will include the option to work with a clinician, 
either at the Department of Child and Adolescent Psychiatry and Behavioral Sciences 
(DCAPBS) of the Children’s Hospital of Philadelphia or at Child & Adolescent OCD, Tic, Trich 
& Anxiety Group (COTTAGe) at the University of Pennsylvania, Department of Psychiatry. 
Participants may experience at least some subjective distress during treatment, and their 
reactions will be closely monitored and addressed therapeutically.  In the unlikely event that 
a child manifests unusually high levels of distress, suggesting prohibitive side effects or 
intolerance of treatment, he or she will be withdrawn from the study by the PI and clinical 
recommendations in the best interest of the child will be made on a case by case basis. In-
person parent meetings will be scheduled as needed should clinical need arise (e.g., address 
clinical worsening, address parent concerns, prevent premature termination).   
Potential Risks to STS Therapists and Clinical Supervisors. There are no known 
physical or legal risks for STS therapists and clinical supervisors to participating in the 
study. One potential risk might be related to the video recording of treatment and 
supervision sessions. Therapists might be apprehensive about being video-recorded while 
conducting treatment sessions with children and therapists and supervisors might feel 
apprehension about being video-recorded while participating in supervision. In a current 
study we are conducting, school-based therapists did not object to and did not report any 
apprehension about being video recorded while conducting groups and receiving 
consultation from members of the research group.  Supervisors and therapists will be told 
that the video-recordings will be used for research purposes only and that they will not be 
shared with agency or school administrators. If a therapist or supervisor raises strong 
concerns about being video recorded either before or after consenting to participate in the 
study, we will not insist on having them participate or allow them to drop out of the study.      
 
8.4.3 Potential Benefits of Trial Participation 
Student participants in this study will benefit from receiving an evaluation, continuous 
monitoring and treatment for anxiety. Information from this study will directly benefit 
participants by guiding appropriate treatment design and planning.  It will be of general 
benefit as it will add to the body of knowledge concerning the types of training supervision 
school-based therapists need in order to be able to provide services with high levels of 
fidelity and acceptability. Thus, benefits to participants are direct (amelioration or 
elimination of symptoms) and indirect (helping to evaluate the effectiveness of levels of 
support provided to school-based therapists). If parents express interest in additional 
treatment for behavioral health concerns outside of current STS services at the end of study 
participation, a list of community behavioral health providers will be distributed.  The child’s 
STS team will also be informed of the parents’ expressed wish for additional services.  
8.5 Risk-Benefit Assessment 
8.6 Recruitment Strategy 
In collaboration with CBH (during the grant proposal and start-up phases), the research 
team has identified the participating mental health agencies.  Each agency will facilitate 
school selection as described in Sections 3.2 & 8.1.1.  Supervisors and therapists from the 
agencies who are embedded within the selected schools will be approached for 
participation. However, participation is voluntary, so if a supervisor or therapist does not 

 
  
 
 
 
21 
consent to participate, we will ask a different supervisor/therapist (already in the school) to 
participate. All student recruitment will be based upon school/STS referrals as noted in 
Sections 3.1.1.  Parents of students who meet screening criteria and who have indicated an 
interest in our program, will be contacted via telephone by a team member about the study. 
 
8.7 Informed Consent/Assent and HIPAA Authorization 
Informed consent (parent) will be obtained by the PI or research team prior to intervention 
treatment.  Student assent will also be obtained.  In all cases, study procedures, as well as 
potential risks, benefits, and treatment alternatives will be clearly explained to both the 
parent and the child in a language understandable to each.  
Study procedures as well as potential risks, benefits and treatment alternatives will be 
described to subjects at the time participation in the study intervention is sought. No student 
will be asked to participate if demonstrably unwilling, even if his or her parent wishes it. 
Students and their parents will be clearly informed of the voluntary and confidential nature 
of their participation, that treatment services will be provided in the school setting, and of 
their right to terminate their participation at any time without penalty.  Ample opportunity will 
be provided for them to ask any questions they may have of research staff.  If the family 
agrees to participate, the consent form will be signed. The PI will retain copies of the signed 
consent and assent forms, and parents will be given a copy of these forms.  If the study 
team has exhausted all possible ways to meet a parent for the consent process and it is just 
not feasible, the study team member will discuss the ICF with the parent over the phone 
and answer any questions they may have about the study.  Afterward, if the parent/student 
is interested in participating, the parent may return a copy of the signed consent form to the 
STS or CHOP team member via fax or email.  Once parental consent is 
obtained/documented via fax/email, student assent will then be obtained in-person by a 
study team member in the school.  No study procedures will take place in the time between 
obtaining documentation of consent and obtaining student assent.  Children or families who 
are in need of treatment but do not qualify for this study or decide not to participate will be 
provided with appropriate treatment referrals. 
All treatment sessions are video recorded for fidelity monitoring. Parent consent for 
videotaping will be obtained using the CHOP Consent and Release for Recording and 
Filming, so that the videos may be used for the training of future staff members and 
education purposes, potentially outside of CHOP (i.e.: presentations to professional and/or 
medical boards).  
Consent for the participation of supervisors and therapists in this study will be obtained 
verbally. They will be provided with a consent letter, outlining what their participation will 
entail, and will be given the same opportunity as families to ask questions and make an 
informed decision regarding their involvement.  Consent will be obtained by Research Staff 
Members, which will include Co-Investigators, Post-Doctoral Fellows, and other research 
staff.  Refusal to participate by a supervisor or STS therapist will not affect their 
employment at the participating mental health agency. 
8.8 Payment to Subjects/Families 
The agencies providing the treatment will bill Community Behavioral Health (CBH) for 
services (i.e., fee for service) related to treatment provided for this study.  

 
  
 
 
 
22 
Therapists will be provided a $37 stipend for each supervision session attended with their 
supervisor. Therapists can also receive an extra $18.50 stipend per half-hour of “prep time” 
(e.g., preparation to run group, preparation for supervision, etc.)  Therefore, therapists can 
receive up to a maximum of $74 per session implemented, for which he/she prepared to 
implement and engaged in supervision with his/her supervisor.   
Supervisors will be reimbursed at a rate of $42 per hour of time invested in the project.  
This will include the one hour supervision session conducted with their implementing 
therapist (as is existing standard of practice via agency requirement), any time spent 
preparing for supervision), and for those supervisors enrolled in Condition C, any 
consultation session he/she participates in with the CHOP study team. 
Parents will receive $10 for completing baseline measures (pre-intervention time point) and 
$10 for completing outcome measures at post.   
Teachers will receive $10 for completing measures pre-intervention and $10 for completing 
outcome measures at post.   
Students will receive small gifts (e.g., pencils, notebooks, small toys) as reinforcement for 
participating and reaching certain goals in group sessions. 
  
9 
PUBLICATION 
Describe the plans for publication. If the CHOP investigator will not have access to the 
complete trial data, describe how publication will proceed. 
10 REFERENCES 
1. Copeland VC. African Americans: disparities in health care access and utilization. Health 
& social work.Aug 2005;30(3):265-270. 
2. Kataoka SH, Zhang L, Wells KB. Unmet need for mental health care among U.S. 
children: variation by ethnicity and insurance status. The American journal of psychiatry. 
Sep 2002;159(9):1548-1555. 
3. Van Amerigen M, Manicini C, Farvolden P. The impact of anxiety disorders on 
educational achievement. Journal of Anxiety Disorders. 2003;17:561-571. 
4. Beidel DC. Social phobia and overanxious disorder in school-age children. Journal of the 
American Academy of Child and Adolescent Psychiatry. Jul 1991;30(4):545-552. 
5. Cappella E, Jackson DR, Bilal C, Hamre BK, Soulé C. Bridging Mental Health and 
Education in Urban Elementary Schools: Participatory Research to Inform Intervention 
Development. School psychology review. 2011;40(4):486-508. 
6. Cappella E, Frazier SL, Atkins MS, Schoenwald SK, Glisson C. Enhancing schools' 
capacity to support children in poverty: an ecological model of school-based mental health 
services. Administration and policy in mental health. Sep 2008;35(5):395-409. 
7. Atkins MS, McKay M, Arvanitis BA, et al. An ecological model for school-based mental 
health services for urban low-income aggressive children. The Journal of Behavioral Health 
Services & Research.1998;5(1):64-75. 
8. Langley A, Bergman R, McCracken J, Piacentini J. Impairment in childhood anxiety 
disorders: Preliminary examination of the Child Anxiety Impact Scale - Parent Version. . 
Journal of Child and Adolescent Psychopharmacology. 2004;14:105-114. 
9. Barrett P, Turner C. Prevention of anxiety symptoms in primary school children: 
preliminary results from a universal school-based trial. Br J Clin Psychol. Nov 2001;40(Pt 
4):399-410. 

 
  
 
 
 
23 
10. Farrell L, Barrett P, Ollendick T. School-based interventions for anxiety disorders in 
children and youth. In: Evans SW, Weist MD, Zerpell ZN, eds. Advances in school-based 
mental health interventions: Best Practices and program models. Kingston, NJ: Civic 
Research Institute; 2007:15 11-20. 
11. Barrett P. Friends for Life: Group leaders' manual for children. Sydney, Australia: 
Pathways Health and Research Centre, Australia; 2008. 
12. Weist MD, Evans SW. Expanded school mental health: Challenges and opportunities in 
an emerging field. Journal of Youth and Adolescence. 2005;3:3–6. 
13. Eiraldi R, McCurdy B, Khanna M, et al. A cluster randomized trial to evaluate external 
support for the implementation of positive behavioral interventions and supports by school 
personnel. Implementation Science. 2014;9(12). 
14. Garbacz A, Watkins N, Barnabas E, Diaz Y, Schwartz B, Eiraldi R. Using Conjoint 
Behavioral Consultation to Implement Evidence-based Practices for Students in Low-
income Urban Schools. Journal of Educational and Psychological Consultation. Under 
review. 
15. Eiraldi R, Watkins N, McCurdy B, Diaz Y, Mautone J, Nastasi B. Development and 
Implementation of a SWPBIS Program with Mental Health Supports in Urban Schools. 
Journal of Positive Behavior Interventions. Under review. 
16. Sholomskas DE, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF, Carroll KM. 
We don't train in vain: a dissemination trial of three strategies of training clinicians in 
cognitive-behavioral therapy. J Consult Clin Psychol. Feb 2005;73(1):106-115. 
17. Accurso EC, Taylor RM, Garland AF. Evidence-based Practices Addressed in 
Community-based Children's Mental Health Clinical Supervision. Training and education in 
professional psychology. May 1 2011;5(2):88-96. 
18. Miller WR, Yahne CE, Moyers TB, Martinez J, Pirritano M. A randomized trial of 
methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical 
Psychology. 2004;72(6):1050–1062. 
19. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation 
hybrid designs: combining elements of clinical effectiveness and implementation research 
to enhance public health impact. Medical care. Mar 2012;50(3):217-226. 
20. Wandersman A, Duffy J, Flashpohler P, et al. Bridging the gap between prevention 
research and practice: The Interactive Systems Framework for dissemination and 
implementation. American journal of community psychology. 2008;41(3-4):171-181. 
21. Wandersman A, Chien VH, Katz J. Toward an evidence-based system for innovation 
support for implementing innovations with quality: tools, training, technical assistance, and 
quality assurance/quality improvement. American Journal of Community Psychology. 
2012;50(3-4). 
22. National Institute of Mental Health. Strategic Plan: Envisioning a World in Which Mental 
Illnesses are Prevented and Cured. Washington, DC: NIMH;2008. 
23. Canino G, Shrout PE, Rubio-Stipec M, et al. The DSM-IV rates of child and adolescent 
disorders in Puerto Rico: prevalence, correlates, service use, and the effects of impairment. 
Archives of general psychiatry. Jan 2004;61(1):85-93. 
24. Costello EJ, Mustillo S, Erkanli A, Keeler G, Angold A. Prevalence and development of 
psychiatric disorders in childhood and adolescence. Archives of general psychiatry. Aug 
2003;60(8):837-844. 
25. Pina AA, Silverman WK. Clinical phenomenology, somatic symptoms, and distress in 
Hispanic/Latino and European American youths with anxiety disorders. Journal of clinical 
child and adolescent psychology : the official journal for the Society of Clinical Child and 
Adolescent Psychology, American 
Psychological Association, Division 53. Jun 2004;33(2):227-236. 

 
  
 
 
 
24 
26. Costello EJ, Angold A, Burns BJ, et al. The Great Smoky Mountains Study of Youth. 
Goals, design, methods, and the prevalence of DSM-III-R disorders. Archives of general 
psychiatry. Dec1996;53(12):1129-1136. 
27. Angold A, Erkanli A, Farmer EM, et al. Psychiatric disorder, impairment, and service 
use in rural African American and white youth. Archives of general psychiatry. Oct 
2002;59(10):893-901. 
28. Greco L, Morris T. Factors influencing the link between social anxiety and peer 
acceptance: Contributions of social skills and close friendships during middle childhood. 
Behavior Therapy. 2005;36:197-205. 
29. Strauss CC, Forehand R, Smith K, Frame CL. The association between social 
withdrawal and internalizing problems of children. J Abnorm Child Psychol. Dec 
1986;14(4):525-535. 
30. Bergeron L, Valla JP, Smolla N, Piche G, Berthiaume C, St-Georges M. Correlates of 
depressive disorders in the Quebec general population 6 to 14 years of age. Journal of 
Abnormal Child Psychology. Jun 2007;35(3):459-474. 
31. Fergusson DM, Woodward LJ. Mental health, educational, and social role outcomes of 
adolescents with depression. Archives of general psychiatry. Mar 2002;59(3):225-231. 
32. Kearney CA, Bensaheb A. Assessing Anxiety Disorders in Children and Adolescents. 
In: Smith S, Handler L, eds. The clinical assessment of children and adolescents: A 
practitioner's handbook. Mahwah, NJ: Lawrence Erlbaum Associates Publishers.; 
2007:467-483. 
33. Kearney CA, Albano AM. The functional profiles of school refusal behavior. Diagnostic 
aspects. Behavior modification. Jan 2004;28(1):147-161. 
34. Feehan M, McGee R, Williams SM. Mental health disorders from age 15 to age 18 
years. Journal of the American Academy of Child and Adolescent Psychiatry. Nov 
1993;32(6):1118-1126. 
35. Fleming JE, Offord DR. Epidemiology of childhood depressive disorders: a critical 
review. Journal of the American Academy of Child and Adolescent Psychiatry. Jul 
1990;29(4):571-580. 
36. Kovacs M, Feinberg TL, Crouse-Novak MA, Paulauskas SL, Finkelstein R. Depressive 
disorders in childhood. I. A longitudinal prospective study of characteristics and recovery. 
Archives of general psychiatry. Mar 1984;41(3):229-237. 
37. Birmaher B, Ryan ND, Williamson DE, et al. Childhood and adolescent depression: a 
review of the past 10 years. Part I. Journal of the American Academy of Child and 
Adolescent Psychiatry. Nov 1996;35(11):1427-1439. 
38. Ialongo N, Edelsohn G, Werthamer-Larsson L, Crockett L, Kellam S. The significance of 
self-reported anxious symptoms in first grade children: prediction to anxious symptoms and 
adaptive functioning in fifth grade. Journal of child psychology and psychiatry, and allied 
disciplines. Mar 1995;36(3):427-437. 
39. Silverman WK, Kurtines WM, Ginsburg GS, Weems CF, Lumpkin PW, Carmichael DH. 
Treating anxiety disorders in children with group cognitive-behaviorial therapy: a 
randomized clinical trial. J Consult Clin Psychol. Dec 1999;67(6):995-1003. 
40. Pahl KM, Barrett PM. Interventions for anxiety disorders in children using group 
cognitive-behavioral therapy with family involvement. In: J.R. Weisz, Kazdin AE, eds. 
Evidence-Based Psychotherapies for Children and Adolescents. New Yprk: The Guilford 
Press; 2010:61-79. 
41. Ollendick TH, King NJ. Empirically supported treatments for children with phobic and 
anxiety disorders: Current status. Journal of Clinical Child Psychology. 1998;27:156-167. 
42. Kendall PC. Treating anxiety disorders in children: results of a randomized clinical trial. 
J Consult Clin Psychol. Feb 1994;62(1):100-110. 

 
  
 
 
 
25 
43. Liber JM, Van Widenfelt BM, Utens EM, et al. No differences between group versus 
individual treatment of childhood anxiety disorders in a randomised clinical trial. Journal of 
child psychology and psychiatry, and allied disciplines. Aug 2008;49(8):886-893. 
44. Cooley MR, Boyd RC, Grados JJ. Feasibility of an anxiety preventive intervention for 
community violence exposed African American children. Journal of Primary Care 
Prevention. 2004;25:105-123. 
45. Briesch AM, Sanetti LMH, Briesch JM. Reducing the Prevalence of Anxiety in Children 
and Adolescents: An Evaluation of the Evidence Base for the FRIENDS for Life Program. 
School mental health. 2010;2(4):155–165. 
46. Barrett P, Duffy AL, Dadds MR, Rapee RM. Cognitive-behavioral treatment of anxiety 
disorders in children: long-term (6-year) follow-up. J Consult Clin Psychol. Feb 
2001;69(1):135-141. 
47. Shortt AL, Barrett PM, Fox TL. Evaluating the FRIENDS program: A cognitive-
behavioral group treatment for anxious children and their parents. J Clin Child Psychol. Dec 
2001;30(4):525-535. 
48. Barrett P, Donderegger R, Xenos S. Using FRIENDS to combat anxiety and adjustment 
problems among young migrants to Australia: A national trial. Clinical Child Psychology and 
Psychiatry. 2003;8:241-260. 
49. Liddle I, Macmillan S. Evaluating the FRIENDS programme in a Scottish setting. 
Educational Psychology in Practice. 2010;26:53-67. 
50. Bernstein GA, Layne AE, Egan EA, Tennison DM. School-based interventions for 
anxious children. Journal of the American Academy of Child and Adolescent Psychiatry. 
Nov 2005;44(11):1118-1127. 
51. Rones M, Hoagwood K. School-based mental health services: a research review. 
Clinical child and family psychology review. Dec 2000;3(4):223-241. 
52. Taras HL. School-based mental health services. Pediatrics. Jun 2004;113(6):1839-
1845. 
53. McKay MM, McCadam K, Gonzales J. Addressing barriers to mental health services for 
inner-city children and their caretaker. Community mental health journal. 1996;32(4):353-
361. 
54. Masia-Warner C, Klein RG, Dent HC, et al. School-based intervention for adolescents 
with social anxiety disorder: results of a controlled study. J Abnorm Child Psychol. Dec 
2005;33(6):707-722. 
55. Evans SW. Mental health services in schools: utilization, effectiveness, and consent. 
Clinical psychology review. Mar 1999;19(2):165-178. 
56. Kazdin AE, Blase SL. Rebooting Psychotherapy Research and Practice to Reduce the 
Burden of Mental Illness. Perspectives on Psychological Science. Jan 2011;6(1):21-37. 
57. Balas EA, Boren SA. Managing clinical knowledge for health care improvement. In: 
Bemmel J, T. MA, eds. Yearbook of Medical Informatics 2000: Patient-Centered Systems. 
Stuttgart, Germany: Schattauer; 2000:65-70. 
58. Pumariega AJ, Rogers K, Rothe E. Culturally competent systems of care for children's 
mental health: advances and challenges. Community mental health journal. Oct 
2005;41(5):539-555. 
59. Glasgow RE, Lichtenstein E, Marcus AM. Why Don’t We See More Translation of 
Health Promotion Research to Practice? Rethinking the Efficacy-to-Effectiveness 
Transition. American Journal of Public Health. 2003;93(8):1261–1267. 
60. Cully JA, Armento ME, Mott J, et al. Brief cognitive behavioral therapy in primary care: a 
hybrid type 2 patient-randomized effectiveness-implementation design. Implementation 
Science. 2012;7(64). 

 
  
 
 
 
26 
61. Public Citizens for Children and Youth. Behavioral Health Goes to School: An Analysis 
of School-Based Behavioral Health Services in Philadelphia. Philadelphia, PA: Public 
Citizens for Children and Youth (PCCY);2011. 
62. Foster S, Rollefson M, Doksum T, Noonan D, Robinson G. School mental health 
services in the United States, 2002-2003. Rockville, MD: Center for Mental Health Services, 
Substance Abuse and Mental Health Administration;2005. DHSS Publication No. SMA 05-
4068). 
63. Cammack NL, Brandt NE, Slade E, Lever NA, Stephan S. Funding expanded school 
mental health programs. In: Weist MD, Lever NA, Bradshaw CP, Owens JS, eds. Handbook 
of School Mental Health:Research, Training, Practice, and Policy. New York: Speringer; 
2014:17-30. 
64. Weist MD. Expanded school mental health services: A national movement in progress. 
In: Ollendick TH, Prinz RJ, eds. Advances in clinical child psychology. Vol 19. New York: 
Plenum; 1997:319-352. 
65. Aarons GA, Green AE, Palinkas LA, et al. Dynamic adaptation process to implement an 
evidence-based child maltreatment intervention. Implementation science : IS. 2012;7:32. 
66. Carvalho ML, Honeycutt S, Escoffery C, Glanz K, Sabbs D, Kegler MC. Balancing 
fidelity and adaptation: implementing evidence-based chronic disease prevention programs. 
Journal of public health management and practice : JPHMP. Jul-Aug 2013;19(4):348-356. 
67. Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of 
new programs and innovations: a review of the empirical literature and recommendations 
for future research. Implementation Science. 2012;7(17). 
68. Backer TE. Finding the balance: Program fidelity and adaptation in substance abuse 
prevention: A state-of-the-art review. Rockville, MD: Department of Health and Human 
Services, Substance Abuseand Mental Health Services Administration;2001. 
69. Lee SJ, Altschul I, Mowbray CT. Using planned adaptation to implement evidence-
based programs with new populations. American journal of community psychology. Jun 
2008;41(3-4):290-303. 
70. McHugh RK, Barlow DH. Dissemination and implementation of evidence-based 
psychological interventions. New York: Oxford University Press; 2012. 
71. McHugh RK, Barlow DH. The dissemination and implementation of evidence-based 
psychological treatments. A review of current efforts. The American psychologist. Feb-Mar 
2010;65(2):73-84. 
72. Dorsey S, Pullmann MD, Deblinger E, et al. Improving practice in community-based 
settings: a randomized trial of supervision - study protocol. Implementation science : IS. 
2013;8:89. 
73. Southam-Gerow MA, Daleiden EL, Chorpita BF, et al. MAPping Los Angeles County: 
Taking an Evidence-Informed Model of Mental Health Care to Scale. Journal of Clinical 
Child & Adolescent Psychology. 2014;43(2):190-200. 
74. Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the 
implementation of psychosocial treatments: a review and critique with recommendations. 
Clinical psychology review. Jun 2010;30(4):448-466. 
75. Shore BA, Iwata BA, Vollmer TR, Lerman DC, Zarcone JR. Pyramidal staff training in 
the extension of treatment for severe behavior disorders. Journal of applied behavior 
analysis. Fall 1995;28(3):323-332. 
76. Demchak M, Browder DM. An evaluation of the pyramid model of staff training in group 
homes for adults with severe handicaps. Education and Training in Mental Retardation. 
1990;25(2):150-163. 
77. Anderson SE, Youngson SC. Introduction of a child sexual abuse policy in a health 
district. Children and Society. 1990;4(4):401-419. 

 
  
 
 
 
27 
78. Chambers D. The Interactive Systems Framework for dissemination and 
implementation: Enhancing the opportunity for implementation science. American Journal of 
Community Psychology. 2012;50:282-284. 
79. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: 
conceptual distinctions, measurement challenges, and research agenda. Administration 
and policy in mental health. Mar 2011;38(2):65-76. 
80. Kutash K, Duchnowski AJ, Lynn N. School-based mental health: An empirical guide for 
decisionmakers. Tampa, FL: University of South Florida, The Louis de la Parte Florida 
Mental Health Institute, Department of Child & Family Studies, Research and Training 
Center for Children’s Mental Health;2006. 
81. Strein W, Hoagwood K, Cohn A. School psychology: A public health perspective I. 
Prevention, populations, and systems change. Journal of School Psychology. 2003;41:23-
38. 
82. Weist MD. Fulfilling the promise of school-based mental health: moving toward a Public 
Mental Health Promotion approach. Journal of Abnormal Child Psychology. Dec 
2005;33(6):735-741. 
83. Achenbach T, Rescorla L. Manual for the ASEBA School-Age Forms & Profiles: An 
Integrated System of Multi-informant Assessment. Burlington, VT: University of Vermont, 
Research Center for Children Youth & Families; 2001. 
84. Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME. NIMH Diagnostic 
Interview Schedule for Children Version IV (NIMH DISC-IV): description, differences from 
previous versions, and reliability of some common diagnoses. Journal of the American 
Academy of Child and Adolescent Psychiatry. 
Jan 2000;39(1):28-38. 
85. March J. Manual for the Multidimensional Anxiety Scale for Children (MASC). Pearson 
Education, Inc.;1997. 
86. Axelson DA, Birmaher B. Relation between anxiety and depressive disorders in 
childhood and adolescence. Depression and anxiety. 2001;14(2):67-78. 
87. Birmaher B, Brent DA, Chiappetta L, Bridge J, Monga S, Baugher M. Psychometric 
properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED): a 
replication study. Journal of the American Academy of Child and Adolescent Psychiatry. 
Oct 1999;38(10):1230-1236. 
88. Compton SN, Walkup JT, Albano AM, et al. Child/Adolescent Anxiety Multimodal Study 
(CAMS): rationale, design, and methods. Child Adolesc Psychiatry Ment Health. 2010;4:1. 
89. Eiraldi RB, Mazzuca LB, Clarke AT, Power TJ. Service Utilization among ethnic minority 
children with ADHD: a model of help-seeking behavior. Administration and policy in mental 
health. Sep 2006;33(5):607-622. 
90. Eiraldi R, Mazzuca LB. Factors affecting utilization of mental health services by low-
income Latino children: A model of parental help-seeking behavior. In: Maume AADJ, ed. 
Child Poverty in America Today: Health and Medical Care. Vol 2. Westport, CT: Praeger; 
2007:74-92. 
91. Beidas RS, Aarons G, Barg F, et al. Policy to implementation: evidence-based practice 
in community mental health - study protocol. Implementation Science. Mar 24 2013;8. 
92. Beidas R, Marcus S, Aarons GA, et al. Individual and organizational factors related to 
community clinicians’ use of therapy techniques in a large public mental health system 
JAMA Pediatrics in press. 
93. Beidas RS, Edmunds JM, Cannuscio CC, Gallagher M, Downey MM, Kendall PC. 
Therapists perspectives on the effective elements of consultation following training. 
Administration and policy in mental health. Nov 2013;40(6):507-517. 

 
  
 
 
 
28 
94. Ditty MS, Landes SJ, Doyle A, Beidas RS. It Takes a Village: A Mixed Method Analysis 
of Inner Setting Variables and Dialectical Behavior Therapy Implementation. Administration 
and policy in mental health. Oct 16 2014. 
95. Beidas RS, Cross W, Dorsey S. Show Me, Don't Tell Me: Behavioral Rehearsal as a 
Training and Analogue Fidelity Tool. Cognitive and behavioral practice. Feb 2014;21(1):1-
11. 
96. Kendall PC, Choudhury MS, Hudson JL, Webb A. The C.A.T. Project Manual: Manual 
for the Individual Cognitive-Behavioral Treatment of Adolescents with Anxiety Disorders. 
Ardmore, PA: Workbook Publishing Inc.; 2002. 
97. Walkup JT, Albano AM, Piacentini J, et al. Cognitive behavioral therapy, sertraline, or a 
combination in childhood anxiety. N Engl J Med. Dec 25 2008;359(26):2753-2766. 
98. Kendall PC. Camp Cope-A-Lot: The Coping Cat CD Rom [CD Rom]. Ardmore, PA: 
Workbook Publishing Inc.; 2008. 
99. Kendall PC, Khanna MS. CBT4CBT: Computer-Based Training to become a Cognitive-
Behavioral Therapist [CD Rom]. Ardmore, PA: Workbook Publishing Inc.; 2008. 
100. Khanna M, Kendall PC. Computer-assisted treatment for the treatment of anxiety in 
youth: A randomized controlled trial. Journal of Consulting and Clinical Psychology. 
2010;78(5):737-745. 
101. Glick HA, Doshi JA, Sonnad SS, Polsky D. Economic Evaluation in Clinical Trials. 
Oxford: University Press; 2007. 
102. Glick HA, Doshi JA, Sonnad SS, Polsky D. Economic Evaluation in Clinical Trials. 
Second ed. Oxford: Oxford University Press; 2015. 
103. Lochman JE, Boxmeyer C, Powell N, Qu L, Wells K, Windle M. Dissemination of the 
Coping Power program: importance of intensity of counselor training. J Consult Clin 
Psychol. Jun 2009;77(3):397-409. 
104. Bickman L, Kelley SD, Breda C, de Andrade AR, Riemer M. Effects of routine 
feedback to clinicians on mental health outcomes of youths: results of a randomized trial. 
Psychiatric services. Dec 2011;62(12):1423-1429. 
105. Schoenwald SK, Mehta TG, Frazier SL, Shernoff ES. Clinical Supervision in 
Effectiveness and Implementation Research. 2013;20:44-59. 
106. Beidas RS, Kendall PC. Training Therapists in Evidence-Based Practice: A Critical 
Review of Studies From a Systems-Contextual Perspective. Clinical Psychology Science 
and Practice. Mar 2010;17(1):1-30. 
107. American Psychological Association (APA). Guidelines for clinical supervision in health 
service psychology. American Psychologist. 2015;70(1):33-46. 
108. Sburlati ES, Schniering CA, Lyneham HJ, Rapee RM. A Model of Therapist 
Competencies for the Empirically Supported Cognitive Behavioral Treatment of Child and 
Adolescent Anxiety and Depressive Disorders. Clinical child and family psychology review. 
2011;14(1). 
109. Beidas RS, Mychailyszyn MP, Edmunds JM, Khanna MS, Downey MD, Kendall PC. 
Training school mental health providers to deliver cognitive-behavioral therapy. School 
mental health. 2012;4:197-206. 
110. Martin JL, Weisz JR, Chorpita BF, Higa CK, Wells KC, Southam-Gerow MA. Moving 
evidence-based practices into everyday clinical care settings: Addressing challenges 
associated with pathways to treatment, child characteristics, and structure of treatment. 
Emotional & Behavioral Disorders in Youth. 2007;7:5-21. 
111. Beidas RS, Barmish AJ, Kendall PC. Training as usual: Can therapist behavior change 
after reading a manual and attending a brief workshop on cognitive behavioral therapy for 
youth anxiety? The BehaviorTherapist. 2009;32:97-101. 

 
  
 
 
 
29 
112. Silverman WK, Albano AM. Anxiety Disorders Interview Schedule for Children, DSM-
IV edition, Child and Parent Versions. San Antonio, TX.: Graywind Publications, A Division 
of The Psychological Corporation.; 1997. 
113. Silverman WK, Albano AM. The Anxiety Disorders Interview Schedule for DSM-IV-
Child and Parent Versions. San Antonio, TX: Graywind Publication, A Division of The 
Psychological Corporation.; 1996. 
114. Guy W. Clinical Global Impression (CGI)1976:217-222. Located at: ECDEU 
Assessment Manual for Psychopharmacology Revised,, Rockville, MD. 
115. Shaffer D, Gould MS, Brasic J, et al. A children's global assessment scale (CGAS). 
Archives of general psychiatry. Nov 1983;40(11):1228-1231. 
116. March JS. Manual for the Multidimensional Anxiety Scale for Children-2nd Edition 
(MASC 2). North Tonawanda, NY: MHS; 2012. 
117. DiPerna J, Elliott S. Academic Competence Evaluation Scales. San Antonio, TX: The 
Psychological Corporation.; 2000. 
118. DiPerna JC, Elliott SN. The development and validation of the Academic Competence 
Evaluation Scale. Journal of Psychoeducational Assessment. 1999;17:207-225. 
119. DiPerna JC, Volpe RJ, Elliott SN. A model of academic enablers and academic 
achievement. . School psychology review. 2002;31:298‑312. 
120. Demaray MK, Jenkins LN. Relations among academic enablers and academic 
achievement in children with and without high levels of parent-rated symptoms of 
inattention, impulsivity, and hyperactivity Psychology in the Schools. 2011;48:6. 
121. Rutherford LE, DuPaul GJ, Jitendra AK. Examining the relationship between treatment 
outcomes for academic achievement and social skills in school-age children with attention-
deficit hyperactivity disorder. Psychology in the Schools. 2008;45(2):145-157. 
122. Allen JD, Linnan LA, Emmons KM. Fidelity and its relationship to implementation 
effectiveness, adaptation, and dissemination. In: Brownson RC, Colditz GA, Proctor EK, 
eds. Dissemination and Implementation Research in Health: Translating Science to 
Practice. New York: Oxford University Press; 2012:281-304. 
123. Linnan L, Steckler A. Process evaluation and public health interventions: An overview. 
In: Steckler A, Linnan L, eds. Process evaluation in public health interventions and 
research. San Francisco, CA: Jossey-Bass; 2002:1-23. 
124. Elliot D, Mihalic S. Issues in disseminating and replicating effective prevention 
programs. Prevention Science. 2004;5:47-53. 
125. Forgatch MS, Patterson GR, DeGarmo DS. Evaluating fidelity: predictive validity for a 
measure of competent adherence to the Oregon model of parent management training. 
Behav Ther. 2005;36(1):3-13. 
126. Cloitre M, Stovall-McClough KC, Miranda R, Chemtob CM. Therapeutic alliance, 
negative mood regulation, and treatment outcome in child abuse-related posttraumatic 
stress disorder. J Consult Clin Psychol. Jun 2004;72(3):411-416. 
127. Hogue A, Dauber S, Stambaugh LF, Cecero JJ, Liddle HA. Early therapeutic alliance 
and treatment outcome in individual and family therapy for adolescent behavior problems. J 
Consult Clin Psychol. Feb 2006;74(1):121-129. 
128. Huey SJ, Jr., Henggeler SW, Rowland MD, et al. Multisystemic therapy effects on 
attempted suicide by youths presenting psychiatric emergencies. Journal of the American 
Academy of Child and Adolescent Psychiatry. Feb 2004;43(2):183-190. 
129. Fishbein M, Ajzen I. Predicting and changing behavior: The reasoned action approach. 
New York: Psychology Press; 2010. 
130. Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision 
Processes. 1991;50:179-211. 
131. Ajzen I, Fishbein M. Understanding attitudes and predicting social behavior. 
Englewood Cliffs , NJ: Prentice Hall; 1980. 

 
  
 
 
 
30 
132. Fishbein M, Cappella J. The role of theory in developing effective health 
communication. Journal of Communication. 2006;56:S1-17. 
133. Becker MH. The health belief model and personal health behavior. Health Education 
Monographs. 1974;2:324–473. 
134. Bandura A. Social cognitive theory of self regulation. Organizational Behavior and 
Human Decision Processes. 1991;50:248–285. 
135. Mellin EA, Bronstein L, Anderson-Butcher D, Amorose AJ, Ball A, Green J. Measuring 
interprofessional team collaboration in expanded school mental health: Model refinement 
and scale development. Journal of interprofessional care. Sep 2010;24(5):514-523. 
136. French MT, Dunlap LJ, Zarkin GA, McGeary KA, McLellan AT. A structured instrument 
for estimating the economic cost of drug abuse treatment. The Drug Abuse Treatment Cost 
Analysis Program (DATCAP). J Subst Abuse Treat. Sep-Oct 1997;14(5):445-455. 
137. French MT, Roebuck MC, McLellan AT. Cost estimation when time and resources are 
limited: the Brief DATCAP. J Subst Abuse Treat. Oct 2004;27(3):187-193. 
138. Barlow DH. Anxiety and its disorders. . New York: The Guilford Press; 1988. 
139. Kendall PC, MacDonald J. Cognition in the psychopathology of youth. . In: Dobson K, 
Kendall PC, eds. Psychopathology and cognition. Orlando, FL: Academic Press.; 1993. 
140. Kendall PC. Child and adolescent therapy: Cognitive-behavioral procedures. 2nd 
edition ed. New York: the Guilford Press; 2000. 
141. Bernal G, Bonilla J, Bellido C. Ecological validity and cultural sensitivity for outcome 
research: Issues for the cultural adaptation and development of psychosocial treatments 
with Hispanics. Journal of Abnormal Child Psychology. 1995;23(1):67-82. 
142. Molenberghs G, Kenward MG. Missing Data in Clinical Studies. New York: John Wiley 
& Sons; 2007. 
143. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic 
data capture (REDCap) - A metadata-driven methodology and workflow process for 
providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. 
144. Grol R, Wensing M. What drives change? Barriers to and incentives for achieving 
evidence-based practice. The Medical journal of Australia. Mar 15 2004;180(6 Suppl):S57-
60. 
145. Aarons GA. Mental health provider attitudes toward adoption of evidence-based 
practice: the Evidence- Based Practice Attitude Scale (EBPAS). Mental health services 
research. Jun 2004;6(2):61-74. 
146. Glisson C, Schoenwald SK. The ARC organizational and community intervention 
strategy for implementing evidence-based children's mental health treatments. Mental 
health services research. 
Dec 2005;7(4):243-259. 
147. Armitage CJ, Conner M. Efficacy of the Theory of Planned Behaviour: a meta-analytic 
review. The British journal of social psychology / the British Psychological Society. Dec 
2001;40(Pt 4):471-499. 
148. Godin G, Kok G. The theory of planned behavior: a review of its applications to health-
related behaviors. American journal of health promotion : AJHP. Nov-Dec 1996;11(2):87-
98. 
149. Sheeran P. Intention - behavior relations: A conceptual and empirical review. In: 
Stroebe W, Hewstone M, eds. European Review of Social Psychology. Chichester, 
England: Wiley; 2002. 
150. Albarracin D, Johnson BT, Fishbein M, Muellerleile PA. Theories of reasoned action 
and planned behavior as models of condom use: a meta-analysis. Psychological bulletin. 
Jan 2001;127(1):142-161. 

 
  
 
 
 
31 
151. Preacher KJ, Rucker DD, Hayes AF. Addressing Moderated Mediation Hypotheses: 
Theory, Methods, and Prescriptions. Multivariate Behavioral Research. 2007;42(1):185-
227. 
152. Baron RM, Kenny DA. The moderator mediator variable disctinction in social 
psychological research:Conceptual, strategic and statistical considerations. . Journal of 
Personality and Social Psychology. 1986;51(6):1173-1182. 
153. Hayes AF. Beyond Baron and Kenny: Statistical Mediation Analysis in the New 
Millennium. Communication Monographs. 2009 2009;76(4):408-420. 
154. Sobel ME. Some new results on indirect effects and their standard errors in covariance 
structure models. In: Tuma N, ed. Sociological methodology. Washington, DC: American 
Sociological Association; 1986:159-186. 
155. Hayes AF. PROCESS: A versatile computational tool for observed variable mediation, 
moderation, and conditional process modeling. Retrieved from http://www.afhayes.com/ 
public/process2012.pdf2012. 
156. SAS Institute Inc. [computer program]. Cary, NC, USA2002-2012. 
157. IBM SPSS Statistics for Windows, Version 21.0 [computer program]. Armonk, NY: IBM 
Corp.