Feedback and Outcomes for Clinically Useful Student Services (FOCUSS)
Phase: NA
Conditions: School-based Mental Health Treatment Services
Sponsor: Yale University

CLINICAL STUDY PROTOCOL 
Observational Study of Individual or Group Template 
 
 
 
Feedback and Outcomes for Clinically 
Useful Student Services (FOCUSS) 
 
 Protocol Number 
 
2000030199 
  
Protocol Version 
Version Date 7/28/22 
 Version #2 
 
 
Confidentiality Statement: This document is confidential and is to be distributed for review 
only to investigators, potential investigators, consultants, study staff, and applicable 
independent ethics committees or institutional review boards.  The contents of this document 
shall not be disclosed to others without written authorization from the Principal Investigator. 
 
 
 
 

Protocol Number 2000030199 
2 
Synopsis 
Purpose 
Feedback and Outcomes for Clinically Useful Student Services (FOCUSS) is a pilot 
implementation-effectiveness study to increase the use of measurement-based care 
(MBC) among school-based mental health clinicians who provide psychosocial treatment 
interventions to K-12 students with various mental health needs. The purpose of the study 
is to examine the implementation outcomes (i.e., feasibility, appropriateness, 
acceptability, and fidelity) of MBC in school-based mental health treatment services. We 
also plan to observe the effectiveness of MBC on student engagement and treatment 
outcomes as a secondary goal. This pilot implementation-effectiveness trial is designed to 
inform a future, large-scale trial with more participants.  
Potential Impact 
Millions of children and adolescents are diagnosed with a mental health disorder each 
year. Of these, only half receive any mental health treatment. More mental health 
treatment services are delivered in schools than any other child-serving sector. Schools 
offer an unparalleled opportunity to improve children’s access to mental health treatment, 
but the quality of care provided in the school setting is highly variable and generally not 
evidence-based. School-based mental health clinicians lead the provision of mental 
health services for children; they must provide effective treatment. 
A vital opportunity to improve school mental health (SMH) quality is the successful 
implementation of measurement-based care (MBC). MBC is the routine collection and use 
of student- or parent-reported symptom and functioning data to monitor progress 
collaboratively with the family and inform treatment decisions. MBC can be applied to a 
wide variety of mental health disorders and treatment plans. MBC is associated with 
greater and faster reduction of symptoms, improvements in functioning, patient 
engagement, and shared decision-making, but clinicians do not use it regularly and little is 
known about its implementation and effectiveness in SMH. 
Planned analyses 
Data analysis will assess the implementation outcomes (i.e., acceptability, 
appropriateness, feasibility and fidelity) of MBC resulting from implementation strategies 
used to increase MBC among SMH clinicians. I hypothesize that clinicians’ MBC 
implementation outcome ratings will increase following the implementation strategies 
provided as compared to their ratings during the baseline period. Secondary analyses will 
explore the effect of MBC on engagement and mental health outcomes to support a future 
R01 application. Linear mixed effects modeling will be used to account for correlation over 
time and clustering within clinician.  
Primary Objective 
The primary objective of this study is to determine whether a multi-method implementation 
support package will help school-based mental health clinicians use MBC strategies, as 
measured by MBC implementation outcomes (e.g., feasibility, acceptability, 
appropriateness, and use or fidelity) with K-12 students. The multi-method implementation 
strategy package was designed based on prior research, and includes initial training, 
ongoing monthly consultation, tailoring implementation supports to needs identified by 
clinicians prior to implementation, and working with school district leadership to promote 
implementation (e.g., by reducing burdensome documentation requirements).  I 
hypothesize that clinicians’ ratings of MBC implementation outcomes will increase 

Protocol Number 2000030199 
3 
following the implementation strategies provided as compared to their ratings during the 
baseline period without strategies. 
Secondary Objective 
The secondary objective of this study is to determine whether MBC implementation in 
schools is associated with engagement in treatment and student outcomes (i.e., reduction 
in symptoms and improvements in functioning) over the course of SMH treatment.   
Study Design 
A clustered, multiple-baseline design will be used to examine the impact of 
implementation support on clinicians’ fidelity, use and ratings of MBC appropriateness, 
acceptability and fidelity. Approximately 50 SMH clinicians will participate from up to three 
school districts as a part of their regular professional development activities. In these 
school districts, measurement based care implementation is a district-wide quality 
improvement initiative. All clinicians will receive the same implementation supports; there 
is no random assignment to condition. Following an initial control period of at least 1 
month, school districts will start receiving implementation supports. Baseline MBC use, 
attitudes, acceptability, feasibility and appropriateness (per clinician self-report) will be 
collected, as well as needs assessment data from clinician surveys to inform necessary 
adjustments to the implementation supports. Baseline engagement and student outcomes 
will be collected after initial clinician training session. MBC implementation outcomes (i.e., 
MBC use, attitudes, acceptability, appropriateness, and feasibility), engagement and 
student outcomes will be collected at 3-month and 6-month follow-up during intervention 
supports, plus a 9-month follow-up interval. This allows clinicians in the school districts to 
be compared to each other and to their own baseline. The primary comparison is pre-post 
ratings of implementation outcomes for all N=50 clinicians. The secondary comparisons 
are pre-post ratings of engagement and student outcomes, and between-agency 
differences. 
Clinicians will receive a 3-hour professional development on MBC in Fall 2022 that is 
required by the three participating school districts. Monthly, post-training group 
consultation calls will be conducted for 6-months, and online, self-report surveys will be 
collected at baseline, 3-month, 6-month and 9-month intervals. Clinicians will be asked to 
implement measurement-based care, using the Better Outcomes Now (BON) online 
feedback system, with at least 5 students each. Data in BON are deidentified and this 
activity is part of routine care provided by clinicians, so does not require special consent. 
These data will also be used to inform group consultation supporting clinicians’ practices. 
Approximately 150 parents of students working with the participating clinicians will be 
recruited and enrolled using active consent. Parents will be asked to consent to 1) 
permission to review student records for goals and progress; and 2) 25-minute study 
phone calls at baseline, 3-months, 6-months and 9-months to provide parent-reported 
information on engagement and child outcomes throughout treatment. These data will be 
used to assess family engagement and child outcomes associated with measurement-
based care delivered by the clinicians.  
This study will be conducted in established, commonly accepted educational settings 
where mental health services are routinely provided. The interactions between children, 
families, their SMH clinicians on school campuses are part of their regular service 
delivery. The MBC training clinicians provide will be integrated into services as usual. The 
MBC measures and practices are considered best practices, and evidence-based 
practices, for children’s mental health services. However, clinicians in usual care settings 
including schools under-implement MBC. They do not pose any risk when integrated into 
usual care. There are no group assignments. Clinician and parent reported study 

Protocol Number 2000030199 
4 
measures are retrospective throughout the school year that SMH services are being 
provided.  
Study Date Range and Duration 
This study is expected to last approximately two calendar years, with each cohort of 
clinicians and parents recruited within the 2021-2022 or 2022-23 school year as the 
participating sites are school districts. Parents will be recruited in fall of 2021, and then 
again in the fall of 2022, based on clinicians who are signed up to receive the district-
required MBC training each fall. Data collection will occur at baseline, 3 months, 6 months 
and 9 months.   
Number of Study Sites (2)  
[REDACTED] Public Schools, [ADDRESS REDACTED] 
[REDACTED] Public Schools, [ADDRESS REDACTED] 
*Other school district sites may be identified depending on their interest in participation. 
Primary Outcome Variables 
The primary objective of this study is to examine implementation outcomes. 
Implementation outcomes are selected from a published taxonomy of outcomes include 
acceptability, appropriateness, feasibility and fidelity. Acceptability, appropriateness, 
and feasibility will be measured by clinician self-report on the Acceptability of 
Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility 
of Intervention Measure (FIM) via web-based, clinician-report survey. These 4-item 
measures were developed based on input from 36 implementation scientists and 27 
mental health professionals; structural validity, test-retest reliability, and sensitivity to 
change have been established. MBC use (fidelity) will be evaluated by chart review in 
BON for evidence of two progress monitoring measures, the Session Rating Scale 3.0 
(SRS) and Outcomes Rating Scale 3.0 (ORS). The MBC training provided as part of this 
study will include how to use MBC and document it in BON. The ORS will be completed 
before each session to assess prior-week functioning, and the SRS will be completed at 
the end of the session to assess student/parent perceptions of alliance with the clinician.  
Secondary and Exploratory Outcome Variables  
The secondary objective of this study is to examine effectiveness outcomes. 
Effectiveness outcomes of MBC will be measured using actual scores on the SRS and 
ORS, as documented in the student’s service record. The SRS and ORS are part of the 
Partners for Change Outcome Management System (PCOMS), a system used to 
enhance patient involvement in mental health services and improve therapeutic alliance 
and treatment progress. Student engagement will be measured by the SRS. Parent 
engagement will be measured by the Therapeutic Alliance Scales for Caregivers and 
Parents. Symptoms and functioning will be measured using the ORS. Emotional and 
behavioral symptoms will also be measured by the Pediatric Symptom Checklist, 35-item 
version (PSC-35) which has well-established reliability and validity to assess and monitor 
progress of internalizing and externalizing symptoms for children ages 4-18. The 
Therapeutic Alliance Scales for Caregivers and Parents and the PSC-35 will be collected 
directly from parents/guardians over the phone at baseline and follow-up intervals.  
 
Study Population 
This study plans to enroll up to 150 parents/legal guardians of children or adolescents 
receiving SMH treatment from one of the 50 participating clinicians. We anticipate SMH 

Protocol Number 2000030199 
5 
clinicians to be primarily licensed, Masters-level (~95%), in fields of social work (~70%) 
and counseling (~30%), female (~80%), Caucasian (~70%), median age of 30 years old 
and 1 to 20 years practicing in SMH. Parents will be enrolled if their child or adolescent is 
enrolled in mental health services in a participating district and served by a participating 
SMH clinician. We anticipate parents to be primarily female (~70%) and their children to 
be representative of students served by SMH clinicians in the districts serving as study 
sites.  
Number of Participants 
We plan to work with 50 clinicians and enroll up to 150 parents of children or adolescents 
with mental illness. The rationale for this sample size is based on having a large enough 
sample of clinicians trained that a range of implementation outcomes can be observed 
and described as pilot data for a larger-scale, fully-powered study. The rationale for the 
150 students is based on our prior research in quality improvement and implementation 
research in schools, where it is more feasible to introduce a new practice to clinicians by 
telling them they can select a subset of their caseload for MBC implementation. 
Clinicians will introduce the opportunity to participate in the study to the parents of the 
students they implement MBC with.  
Study Schedule 
Clinician participation will require approximately 9 hours of study participation during the 
2021-22 or 2022-23 school year. This includes 6 hours of training and ongoing 
consultation (e.g., 6, 60-minute post-training group consultation calls) plus 2 hours and 30 
minutes of online surveys (five surveys in total; one baseline survey before the study 
begins, one post-training survey, and then follow-ups at 3 months, 6 months and 9 
months).  
 
Parent/guardian participation will require approximately 1 hour, 40 minutes of study 
participation during the 2021-22 or 2022-23 school year. This includes phone consent and 
four phone interviews of approximately 20-25 minutes each at baseline, 3 months, 6 
months and 9 months.  
 
 

Protocol Number 2000030199 
6 
Protocol Revision History 
Version 
Date 
Summary of Substantial Changes 
8/1/22 
1. The prior protocol involved active consent for clinicians as study participants 
to receive post-training consultation and study measures following a 
required, district-wide, 3-hour training on measurement-based care (MBC). 
The revised protocol waives documentation of consent for clinicians. Instead, 
clinicians will receive an informational email from their district leader about 
this project as a district-wide quality improvement initiative to implement 
measurement-based care (MBC). The revised protocol includes the 3-hour 
training, post-training consultation and study measures for all clinicians in the 
district. A waiver of documentation of consent is appropriate due to the low-
risk nature of the study and no assignment to different study conditions; all 
clinicians receive training and support for MBC implementation. This change 
is at the request of our district partners (see two support letters) to include 
more clinicians in the project because they are implementing MBC as part of 
routine care in the schools. This change has also been documented with 
NIMH. There will be no impact on data safety or monitoring plan, 
methods/procedures of the study, or budget. Parents of children served will 
still be consented to study participation, involving phone surveys of child 
outcomes and permission to access basic service data (see below), which is 
optional. 
2. Previously we stated that progress notes in medical records would be 
accessed from the school site and reviewed. We learned that session by 
session documentation of school-based services does not exist in this way. 
Therefore, the current modification reflects updated language about 
requesting parental consent to request basic treatment information about 
their student such as diagnosis, goals, and any documentation of progress 
using the online system (Better Outcomes Now or BON) we are training them 
on. All student data are entered in a deidentified manner to BON as a part of 
routine services provided, and our team has administrator access to BON to 
provide ongoing consultation for clinicians about how to use the system 
clinically. For parents who consent to study participation, we will ask 
clinicians and/or their supervisors to securely transfer basic student services 
information (e.g., diagnosis, goals, services start date) and their BON 
identifier so we can review their progress data and link it to parent-reported 
outcomes. There is no change to patient safety and in fact we anticipate 
parents may view this as less invasive than access to progress notes in 
medical records because we are now asking for access to more basic and less 
sensitive PHI and treatment-related data. 
3. We added the Oldenburg Burnout Inventory (Demerouti & Mostert, 2010) to 
the clinician survey to assess for impact of training on staff wellbeing, and 
staff wellbeing as a potential moderator of implementation outcomes.  
 
 

Protocol Number 2000030199 
7 
Statement of Compliance  
This document is a protocol for a human research study. The purpose of this protocol is to 
ensure that this study is to be conducted according to the Common Rule at 45CFR46 
(human subjects) and other applicable government regulations and Institutional research 
policies and procedures. 
 
Abbreviations 
Abbreviation  
Explanation 
SMH 
School-based mental health 
MBC 
Measurement-based care 
 
Glossary of Terms 
Glossary 
Explanation 
School-based mental health treatment 
Individual, group, and or family 
psychotherapy provided on school 
grounds to reduce symptoms related to 
an identified mental health concern.  
Measurement-based care 
Routine collection and use of student- 
and/or parent/guardian-reported 
progress measures to inform shared 
decision making and treatment planning 
with the student and parent/guardian.  
Table of Contents 
Preface .................................................................................. Error! Bookmark not defined. 
Synopsis ............................................................................................................................... 2 
Purpose ............................................................................................................................. 2 
Primary Objective .............................................................................................................. 2 
Secondary Objective ......................................................................................................... 3 

Protocol Number 2000030199 
8 
Study Design ..................................................................................................................... 3 
Study Date Range and Duration ........................................................................................ 4 
Number of Study Sites ....................................................................................................... 4 
Primary Outcome Variables ............................................................................................... 4 
Secondary and Exploratory Outcome Variables (if applicable) .......................................... 4 
Number of Participants ...................................................................................................... 5 
Study Schedule ................................................................................................................. 5 
Protocol Revision History .................................................................................................. 6 
Statement of Compliance ...................................................................................................... 7 
Abbreviations ........................................................................................................................ 7 
Glossary of Terms ................................................................................................................. 7 
1 
Background/Literature Review ..................................................................................... 11 
1.1 
Background........................................................................................................... 11 
1.2 
Prior Experience (if applicable) ............................................................................. 11 
2 
Rationale/Significance.................................................................................................. 12 
2.1 
Rationale and Study Significance .......................................................................... 12 
2.2 
Purpose of Study/Potential Impact ........................................................................ 12 
2.3 
Potential Risks and Benefits ................................................................................. 12 
2.3.1 
Potential Risks ............................................................................................... 12 
2.3.2 
Potential Benefits ........................................................................................... 13 
3 
Study Purpose and Objectives ..................................................................................... 13 
3.1 
Hypothesis ............................................................................................................ 13 
3.2 
Primary Objective.................................................................................................. 14 
3.3 
Secondary Objective (if applicable) ....................................................................... 14 
4 
Study Design ............................................................................................................... 14 
4.1.1 
General Design Description ........................................................................... 14 
4.1.2 
Study Date Range and Duration .................................................................... 14 
4.1.3 
Number of Study Sites ................................................................................... 15 
4.2 
Outcome Variables ............................................................................................... 15 
4.2.1 
Primary Outcome Variables ........................................................................... 15 
4.2.2 
Secondary and Exploratory Outcome Variables (if applicable) ....................... 15 

Protocol Number 2000030199 
9 
4.3 
Study Population ................................................................................................... 15 
4.3.1 
Number of Participants .................................................................................. 15 
4.3.2 
Eligibility Criteria/Vulnerable Populations ....................................................... 16 
5 
Study Methods/Procedures ....................................................................................... 16 
5.1 
Study Procedures ................................................................................................. 16 
5.1.1 
Data Collection .............................................................................................. 17 
5.1.2 
Adverse Events Definition and Reporting ....................................................... 19 
5.2 
Study Schedule ..................................................................................................... 19 
5.3 
Informed Consent ................................................................................................. 19 
5.3.1 
Screening (if applicable) ................................................................................ 20 
5.3.2 
Recruitment, Enrollment and Retention (if applicable) .................................... 20 
5.3.3 
Study Visits (is applicable) ............................................................................. 21 
5.4 
Statistical Method ................................................................................................. 21 
5.4.1 
Statistical Design ........................................................................................... 21 
5.4.2 
Sample Size Considerations .......................................................................... 21 
5.4.3 
Planned Analyses .......................................................................................... 22 
5.4.4 
Analysis of Subject Characteristics (if applicable) .......................................... 22 
5.4.5 
Interim Analysis (if applicable) ....................................................................... 23 
5.4.6 
Handling of Missing Data ............................................................................... 23 
6 
Trial Administration ...................................................................................................... 23 
6.1 
Ethical Considerations: Informed Consent/Assent and HIPAA Authorization ........ 23 
6.2 
 Institutional Review Board (IRB) Review .............................................................. 24 
6.3 
Subject Confidentiality .......................................................................................... 24 
6.4 
Deviations/Unanticipated Problems ...................................................................... 25 
6.5 
Data Quality Assurance ........................................................................................ 25 
6.6 
Study Records ...................................................................................................... 26 
6.7 
Access to Source .................................................................................................. 26 
6.8 
Data or Specimen Storage/Security ...................................................................... 26 
6.9 
Retention of Records ............................................................................................ 26 
6.10 Study Monitoring ................................................................................................... 26 
6.11 Study Modification................................................................................................. 27 

Protocol Number 2000030199 
10 
6.12 Study Completion ................................................................................................. 27 
6.13 Funding Source .................................................................................................... 27 
6.14 Conflict of Interest Policy....................................................................................... 27 
6.15 Publication Plan .................................................................................................... 27 
Appendices ............................................................................ Error! Bookmark not defined. 
List of Tables ......................................................................... Error! Bookmark not defined. 
 
 
 
 

Protocol Number 2000030199 
11 
1 Background/Literature Review 
1.1 
Background 
Underutilization of mental health treatment among children and adolescents is a 
public health crisis that has improved little over the last several decades. 
[SUPPORTING LITERATURE REDACTED]  
Despite improvements in access to care, school mental health treatment quality is 
inconsistent. [SUPPORTING LITERATURE REDACTED]. One evidence-based approach 
to improve the quality SMH services is increasing clinicians’ use of symptom rating scales at 
the beginning and throughout treatment to drive clinical decision making and chart progress. 
This practice is referred to as measurement-based care (MBC), or the routine collection and 
use of student data throughout treatment, including initial screening and assessment, 
problem definition and analysis, finalizing treatment objectives and intervention tactics, and 
monitoring treatment progress. [SUPPORTING LITERATURE REDACTED] 
MBC is an effective, evidence-based practice to improve treatment quality. 
[SUPPORTING LITERATURE REDACTED] 
Although there is strong support for MBC in primary care and community settings, including 
some studies with children, MBC effectiveness in SMH has not been rigorously tested. 
Moreover, the mechanisms of MBC effects on patient outcomes are not well known. We 
propose to pilot test a set of implementation strategies (that were identified in a previous aim 
of this same grant award) to increase school mental health (SMH) clinician use of 
measurement-based care (MBC), and further explore the effectiveness of MBC on 
student/family engagement and child outcomes.  
1.2 
Prior Experience  
Preliminary research was conducted to inform this study. Specifically, Dr. Connors and her 
team conducted a prior study aim within this grant award to first identify the implementation 
strategies being tested in this current study phase. In the prior study aim, 52 SMH 
Stakeholder Expert Panelists (i.e., SMH clinicians, supervisors, and researchers who 
actively partner with schools) were purposively sampled using the National SMH Census, a 
nationally-representative survey of SMH school and district teams’ services and data usage. 
Using a modified Delphi technique, panelists reviewed the definition of MBC as well as 
clinical vignettes demonstrating how MBC would be used by a clinician providing mental 
health treatment to students. Then, panelists responded to two rounds of online surveys to 
1) rate an existing list of 33 implementation strategies for feasibility and importance, 2) 
identify synonyms, definition changes, and recommendations about how the strategy should 
be used, and 3) suggested additional strategies not on the existing list. Mixed methods 
analysis and GoZone plots revealed six top-rated implementation strategies for MBC in 
schools, which include the following: 
1. Assess for readiness and identify barriers and facilitators 
2. Identify and prepare champions 
3. Develop a usable implementation plan 
4. Offer a clinician-informed menu of free, brief measures 
5. Develop and provide access to training materials 
6. Make implementation easier by removing burdensome documents 
These top-rated strategies will be the foundation of the implementation strategy package 
being tested in the current FOCUSS study. As a result of strategy #1 above, baseline 
clinician surveys will include a needs assessment to inform additional implementation 
strategies and/or tailoring of strategies that should be applied. 

Protocol Number 2000030199 
12 
We also found that mean importance and feasibility ratings were high overall, with 
importance higher than feasibility on average. Ratings were not significantly different 
between researchers and practitioners with three exceptions. First, clinicians reported it is 
more important and more feasible to make implementation easier by removing burdensome 
paperwork (feasibility t(44)=-2.96, p=0.01; importance (t(29)=-2.72, p=0.01). Second, 
researchers reported it’s more important to monitor the implementation effort (t(41)=2.51, 
p=0.02).Third, clinicians reported train-the-trainer is more feasible (t(45)=-2.06, p=0.05). 
In addition to this prior research, Dr. Connors has also led several implementation studies 
with school district personnel to implement various elements of MBC in the past.  
 
2 Rationale/Significance 
2.1 
Rationale and Study Significance  
The fact that the evidence-base of SMH services is highly inconsistent, this has deleterious 
effects on the welfare of all students. The routine use of psychosocial symptom and 
functioning assessment data (MBC) to inform SMH treatment is an effective, evidence-based 
practice that can improve service quality at low cost. In practice, clinicians over-rely on 
clinical judgment and have little time to learn and use more effective practices. It is 
imperative that we change clinician practice to include MBC. The proposed research 
includes cutting-edge implementation science methods to identify and test the most 
promising strategies to increase MBC practice in the school context. This study seeks to 
generate initial knowledge about contextually-specific implementation strategies to increase 
school-based clinicians’ use of MBC with their patients.  
2.2 
Purpose of Study/Potential Impact  
The goals of this study are to collect preliminary data on implementation outcomes 
associated with MBC implementation strategies identified in our prior research, as well as on 
effectiveness of MBC to improve student engagement and treatment outcomes. The logic for 
this line of research is that the right implementation strategies will produce desired 
implementation outcomes, most importantly clinician fidelity (i.e., use of MBC in their 
practice). The effect of implementation strategies on MBC use is likely mediated by how 
acceptable, appropriate, and feasible MBC is rated by clinicians (key implementation 
outcomes that will be measured). MBC use is hypothesized to improve student progress and 
outcomes such as reduced symptoms and improved functioning, possibly through the 
mechanism of engagement.  
If successful, this pilot study will provide a foundation for future investigation to determine 
exactly how MBC can be implemented in “real world” SMH services, which could have a 
dramatic public health impact on the quality of mental health services received by children 
and adolescents in school. This study also has the potential to generate generalizable 
knowledge about the impact of MBC on student engagement and treatment outcomes, to 
further bolster the growing evidence-base of MBC effectiveness with children and 
adolescents. 
2.3 
Potential Risks and Benefits 
2.3.1 Potential Risks  
Risks related to chart review for Parent participants. The risk from chart reviews for 
parent participants is minimal and relatively uncommon. There is a slight risk of breach of 
confidentiality. All data collection forms will be coded with an ID number that is unique. Only 
the study team will have access to the link between the ID and participant’s name. All data 
security procedures commensurate with the clinical and/or school site’s maintenance of 

Protocol Number 2000030199 
13 
secure health records will be followed in detail. For paper charts that might involve secure 
procedures for signing out and returning charts, as well as reviewing charts on premises in a 
secure location at the clinic. For electronic medical records that may involve a data request 
to the participating agency and secure data transfer. Electronic data files with identifiable 
information will be maintained separately from other data files and will only be used for 
administrative purposes (e.g., preparing aggregate reports to share with the study team). 
Data containing names and personal information will never be included in published 
materials. All personnel will receive certification in human subjects’ protection from the Yale 
HRPP prior to beginning work on this project. 
 
Risks related to Clinicians participating in training and implementation supports and 
providing feedback via electronic surveys. The risk from participating in MBC training and 
implementation supports and providing feedback via electronic surveys for clinicians is 
minimal and relatively uncommon. The information page will ensure clinicians are aware that 
their responses to implementation feedback surveys will only be shared in aggregate and will 
not affect their employment or performance review in any way. Moreover, clinical 
administrators will not be included in the data collection process whereby clinicians provide 
self-reported implementation experiences and outcomes. At the start of the study, the PI and 
at least one local study team member will meet with each administrator to identify possible 
sources of bias and generate solutions. District leadership will know which participants are 
involved in the project because they will attend district-organized professional development 
trainings and consultation. However, all clinician-reported survey data regarding their 
implementation experiences will be coded with an ID number that is unique. Only the study 
team will have access to the link between the ID and participant’s name. Electronic data files 
with identifiable information will be maintained separately from other data files and will only 
be used for administrative purposes (e.g., preparing aggregate reports to share with the 
study team). Data containing names and personal information will never be included in 
published materials. All personnel will receive certification in human subjects’ protection from 
the Yale HRPP prior to beginning work on this project. 
2.3.2 Potential Benefits 
Clinicians and the students/patients they serve will be offered a potentially effective 
intervention for collaboratively monitoring their progress in mental health treatment over 
time. Clinician participants who use the measurement-based care processes that they will be 
trained and supported to use will have the opportunity to learn an evidence-based practice 
they can use with any of their patients as they wish. Clinicians will also receive one year user 
access to Better Outcomes Now (www.betteroutcomesnow.com) which is an online system 
to collect, score, view and discuss student progress on the Outcome Rating Scale and 
Session Rating Scale. They will receive training and consultation on how to use this 
evidence-based, practical system with students and families, whether you are meeting with 
them virtually or in person. 
Children and parents who engage in the measurement-based care processes that clinicians 
will be trained and supported to use may experience greater therapist/patient alliance and 
improvement in symptoms and functioning at the end of treatment. We feel the potential 
benefits to be gained outweighs the risks of the study. 
 
3 Study Purpose and Objectives 
3.1 
Hypothesis 
Hypothesis 1: Clinicians’ ratings of MBC implementation outcomes will increase following 
implementation strategies provided as compared to ratings during the baseline period. 
(Primary Objective) 

Protocol Number 2000030199 
14 
Hypothesis 2: MBC fidelity (per MBC feedback system review) and use (per clinician self-
report) will increase following implementation strategies provided as compared to ratings 
during the baseline period, with documentation of measure collection more prevalent than 
documentation of discussing progress or changing the treatment plan in accordance with 
measures. (Primary Objective) 
Hypothesis 3: MBC fidelity and use will be positively associated with engagement and 
mental health symptom and functioning progress over time. (Secondary Objective) 
Although underpowered to establish outcomes as a result of strategies applied with 
certainty, this pilot will provide mentored research experience and preliminary data for 
planning a future large-scale hybrid trial.  
3.2 
Primary Objective  
The primary objective of this study is to determine whether a multi-method implementation 
support package will help school-based mental health clinicians use MBC strategies, as 
measured by MBC implementation outcomes (e.g., feasibility, acceptability, appropriateness, 
and use or fidelity) with K-12 students. The multi-method implementation strategy package 
was designed based on prior research, and includes initial training, ongoing monthly 
consultation, tailoring implementation supports to needs identified by clinicians prior to 
implementation, and working with school district leadership to promote implementation (e.g., 
by reducing burdensome documentation requirements).  I hypothesize that clinicians’ ratings 
of MBC implementation outcomes will increase following the implementation strategies 
provided as compared to their ratings during the baseline period without strategies and 
fidelity (i.e., use of) MBC with students served in SMH treatment.  
3.3 
Secondary Objective  
The secondary objective of this study is to determine whether MBC implementation in 
schools is associated with engagement in treatment and student outcomes (i.e., reduction in 
symptoms and improvements in functioning) over the course of SMH treatment.   
4 Study Design 
4.1.1 General Design Description 
A clustered, multiple-baseline design will be used to examine the impact of the 
implementation support condition on clinicians’ fidelity, use and ratings of MBC 
appropriateness, acceptability and fidelity. Approximately 50 SMH clinicians will be recruited 
to participate from two school districts. All clinicians will receive the same implementation 
supports; there is no random assignment to condition. Following an initial control period of at 
least 1 month, school districts will start receiving implementation supports. During the initial 
control period, baseline MBC use, attitudes, acceptability, feasibility and appropriateness 
(per clinician self-report) will be collected, as well as needs assessment data from clinician 
surveys to inform necessary adjustments to the implementation supports. Baseline 
engagement and student outcomes will be collected after initial clinician training session. 
MBC implementation outcomes (i.e., MBC use, attitudes, acceptability, appropriateness, and 
feasibility), engagement and student outcomes will be collected at 3-month and 6-month 
follow-up during intervention supports, plus a 9-month follow-up interval. This allows 
clinicians in the two school districts to be compared to each other and to their own baseline. 
The primary comparison is pre-post ratings of implementation outcomes for all N=50 
clinicians. The secondary comparisons are pre-post ratings of engagement and student 
outcomes, and between-agency differences. 
Study Date Range and Duration 

Protocol Number 2000030199 
15 
This study is expected to last approximately one calendar year from recruitment to follow-up, 
aligned with the 2021-2022 and 2022-23 academic years as the participating sites are 
school districts. Clinicians will start training and consultation in the fall of 2021 or 2022 and 
parents/guardians will be recruited in the fall of 2021 or 2022. Data collection will occur at 
baseline, 3 months, 6 months and 9 months.   
4.1.2 Number of Study Sites (2) 
1. [REDACTED] Public Schools, [ADDRESS REDACTED] 
2. [REDACTED]  Public Schools, [ADDRESS REDACTED] 
*Other school district sites may be identified depending on their interest in participation 
4.2 
Outcome Variables 
4.2.1 Primary Outcome Variables 
The primary objective of this study is to examine implementation outcomes. Implementation 
outcomes are selected from a published taxonomy of outcomes include acceptability, 
appropriateness, feasibility and fidelity. Acceptability, appropriateness, and feasibility will 
be measured by clinician self-report on the Acceptability of Intervention Measure (AIM), 
Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM) 
via web-based, clinician-report survey. These 4-item measures were developed based on 
input from 36 implementation scientists and 27 mental health professionals; structural 
validity, test-retest reliability, and sensitivity to change have been established. MBC use 
(fidelity) will be evaluated by chart review for evidence of two progress monitoring 
measures, the Session Rating Scale 3.0 (SRS) and Outcomes Rating Scale 3.0 (ORS). The 
ORS should be completed before each session to assess prior-week functioning, and the 
SRS will be completed at the end of the session to assess student/parent perceptions of 
alliance with the clinician. These will be stored in the Better Outcomes Now (BON) secure 
online system.  
4.2.2 Secondary and Exploratory Outcome Variables  
The secondary objective of this study is to examine effectiveness outcomes. Effectiveness 
outcomes of MBC will be measured using actual scores on the SRS and ORS, as 
documented in the student’s service record. The SRS and ORS are part of the Partners for 
Change Outcome Management System (PCOMS), a system used to enhance patient 
involvement in mental health services and improve therapeutic alliance and treatment 
progress. Student engagement will be measured by the SRS. Parent engagement will be 
measured by the Therapeutic Alliance Scales for Caregivers and Parents. Symptoms and 
functioning will be measured using the ORS. Emotional and behavioral symptoms will also 
be measured by the Pediatric Symptom Checklist, 35-item version (PSC-35) which has well-
established reliability and validity to assess and monitor progress of internalizing and 
externalizing symptoms for children ages 4-18. The Therapeutic Alliance Scales for 
Caregivers and Parents and the PSC-35 will be collected directly from parents/guardians 
over the phone at baseline and follow-up intervals. 
4.3 
Study Population 
This study plans to enroll 50 SMH clinicians and 150 parents/guardians of children or 
adolescents receiving SMH treatment from one of the 50 participating clinicians. We 
anticipate SMH clinicians to be primarily licensed, Masters-level (~95%), in fields of social 
work (~70%) and counseling (~30%), female (~80%), Caucasian (~70%), median age of 30 
years old and 1 to 20 years practicing in SMH. Parents will be enrolled if their child or 
adolescent is enrolled in mental health services in a participating district and served by a 
participating SMH clinician. We anticipate parents to be primarily female (~70%) and their 
children to be representative of students served by SMH clinicians in the districts serving as 
study sites. 

Protocol Number 2000030199 
16 
4.3.1 Number of Participants 
We plan to work with 50 SMH clinicians and up to 150 parents of children or adolescents 
with mental illness receiving MBC in SMH services. All SMH clinicians at each district will be 
invited to participate, and we anticipate approximately 10-40 clinicians from each district 
depending on their size. All 150 parents of children and adolescents with mental illness 
receiving MBC in SMH services will be invited to participate. The rationale for this sample 
size is based on having a large enough sample of clinicians trained that a range of 
implementation outcomes can be observed and described as pilot data for a larger-scale, 
fully-powered study. The rationale for the 150 students is based on our prior research in 
quality improvement and implementation research in schools, where it is more feasible to 
introduce a new practice to clinicians by telling them they can select a subset of their 
caseload for MBC implementation. We will outreach to students the clinician reports they are 
using MBC with. Selection of a subset of cases will involve a random selection of recently 
consented cases. 
4.3.2 Eligibility Criteria/Vulnerable Populations 
Parent/Guardian Participants 
To be eligible for inclusion in the study, a parent or legal guardian must meet all of the 
following criteria: 
• 
Female, male or other gender ≥18 years of age at the time of recruitment. 
• 
Parent or legal guardian of a child age 5 to 18 at the time of recruitment who is 
enrolled in school-based mental health treatment services with one of the clinician 
participants included in the study (see inclusion criteria for clinician participants 
above). 
• 
Enrolled in school-based mental health services within 3 months prior to the 
recruitment date. 
• 
English speaking. 
• 
Ability to provide informed consent.  
Parents/guardians who do not meet all inclusion criteria are excluded. There are no other 
exclusion criteria.  
 
5 Study Methods/Procedures  
5.1 
Study Procedures 
Study procedures involve collecting implementation data from clinicians (approximately 
N=50) as part of a district-wide measurement-based care training in initiative and recruiting 
parent/guardian (up to N=150) participants to provide feedback on the implementation 
outcomes of a pilot training and post-training consultation on measurement-based care. 
Study information will be provided to clinicians via email and secure, electronic survey and 
study consent will be provided to parents/guardians by a study team member conducting 
phone consent. Study procedures are built around a 3-hour professional development and 
monthly, post-training consultation calls that are required by participating school districts. 
Clinicians will also be asked to use MBC measures (e.g., Outcome Rating Scale and 
Session Rating Scale) with 5 students on their caseload, and completing study evaluation 
surveys (for which clinicians will be compensated with $25 Amazon gift cards for each 
survey completed at baseline, post-training, 3-, 6-, and 9-month follow-up). 
Parents/guardians of youth served by participating clinicians will be asked to consent to 
phone check ins (at baseline, 3-, 6-, and 9-month intervals) to report child symptoms and 
parent engagement. Parent/guardian participants will also be asked to consent to records 

Protocol Number 2000030199 
17 
review in order for the study team to assess fidelity to MBC training and consultation. 
Additional study procedure details are described in the following sections. 
Study Procedures vs Standard of Care 
Youth (ages 5-18) who receive school-based mental health services from participating 
clinicians will receive measurement-based care (MBC) as part of their usual care services. 
MBC is aligned with best practices in psychosocial treatment delivery but is under-
implemented in usual care due to numerous barriers. MBC practices, which include routine 
progress monitoring and feedback to inform data-driven, client-centered treatment plan 
adjustments, only stands to improve the quality and standard of usual care delivered to 
children and their families in school settings.  
5.1.1 Data Collection  
Data Sources. Data sourced for this study will include computerized and hard copy research 
records. Any hard copy research records will be scanned and saved electronically on a Yale 
secure server before shredding the paper copy. Data sources will include: 
(1) Quantitative data from clinician self-report surveys about MBC acceptability, 
appropriateness, and feasibility as well as MBC use every three months during 
implementation supports. Some qualitative feedback will also be solicited on the 
survey for clinicians to expand on any details the standardized measures of these 
implementation outcomes could not capture. (Primary outcome variables) 
(2) Quantitative data on MBC use (clinician fidelity) coded by chart review protocol, in 
which fidelity includes presence of SRS and ORS measures in the online 
BON/PCOMS system to approximate fidelity per clinician. (Primary outcome 
variables) 
(3) Quantitative data on proximal and distal student engagement and functioning 
outcomes collected by the clinician as part of the project, retrieved from medical 
records and/or the PCOMS system directly (SRS, ORS). (Secondary outcome 
variables) 
(4) Quantitative data on parent engagement and distal student symptoms collected by 
the study team as part of the project from parents over the phone (Therapeutic 
Alliance Scales, PSC-35). (Secondary outcome variables) 
(5) Tracking data on parent recruitment, clinician attendance/engagement in 
implementation supports. (Implementation mechanisms) 
Data sources 1-4 above refer to primary and secondary outcomes of explicit focus in the 
analyses and are listed in the table below. 
 
Table 1. List of Measures and Data Collection Methods 
Outcome 
Measure 
Metho
d 
Reporter 
Interval 
MBC Acceptability 
 (content, complexity, 
relative advantage) 
Acceptability of Intervention 
Measure (AIM) 
Online 
survey 
 
Clinician 
Pre-training 
3M follow up 
6M follow up 
9M follow up 
MBC 
Appropriateness  
(for setting, provider, 
students served) 
Intervention Appropriateness 
Measure (IAM) 
Feasibility  
(for SMH practice) 
Feasibility of Intervention Measure 
(FIM) 
MBC Use 
 
Current Assessment Practice 
Evaluation  - Revised (CAPER) 

Protocol Number 2000030199 
18 
 
Secure Data Recording and Maintenance: 
Online survey data will be maintained in the PI’s Yale Qualtrics account, accessible only by 
members of the study team listed on the IRB protocol. Online survey data will be 
downloaded and maintained on secure Yale storage servers and networks (i.e., Yale Box, 
network drive). Clinicians will provide their identifying information (name, email address) 
when they consent as well as a secure 4-digit anonymous code. They will be asked to 
submit this same 4-digit code with each follow-up survey. This will ensure the study team 
can track individual clinician responses over time, but each survey is completed 
confidentially. A password-protected Excel file including clinician identifying information and 
4-digit codes will be maintained on secure Yale storage servers and networks, accessible 
only to study team members.  
Chart review data may be in various formats depending on the format of documentation 
and storage systems at each district (study site). We only need basic information such as 
child diagnosis, date entered services, treatment goals, and number of sessions attended. 
We will also ask for their anonymous BON code so we can link those data (see below). 
Consistent with best practices in chart review methodology, we will work with each district to 
identify a “data extractor” at their site who can obtain the relevant data for students of 
consented parents within the requisite time frame and prepare them for the study team. Our 
team will provide a list of names of children based on parental consent obtained, as well as 
the data we are requesting. Once we receive the chart data, we will deidentify it with a 
unique study identifier and keep the list of linked study IDs to names in a separate 
password-protected file. We expect that the requested chart review material can be 
transferred in electronic format to our site at Yale, we will use Yale’s secure file transfer 
system, save the material on our secure Yale storage server or network, and then complete 
the enter the services data for each child. These files will be saved on our secure Yale 
storage server and network and entered into a database.  
BON/PCOMS system data (including the Outcome Rating Scale and Session Rating Scale) 
will be maintained in the Better Outcomes Now (www.betteroutcomesnow.com) secure 
online system. Clinicians enter these data directly into PCOMS using their own anonymous 
code for students. There are no identifiable information in PCOMS. Our study team will be 
the PCOMS trainers and also have administrative privileges in PCOMS to support MBC 
MBC Attitudes 
Attitudes toward Standardized 
Assessment – Monitoring and 
Feedback (ASA-MF) 
Implementation 
Determinants 
MBC Feedback Form – Assess 
Barriers and Facilitators 
 
Pre-training 
ORS/SRS 
Acceptability 
Usage Rating Profile – 
Assessment (URP-A) 
 
3M follow up 
6M follow up 
9M follow up 
Fidelity  
(MBC Use) 
Retrospective coding of PCOMS 
measures (SRS, ORS) 
Chart 
review 
and/or 
BON 
system 
 
Study 
Team 
Monthly 
Student Engagement 
 
Session Rating Scale (SRS) 
Student Functioning 
 
Outcome Rating Scale (ORS) 
Student Symptoms 
Pediatric Symptom Checklist 
(PSC-35)  
Phone 
Parent  
Baseline*  
3M follow up 
6M follow up 
9M follow up 
Parent Engagement 
Therapeutic Alliance Scales for 
Caregivers and Parents 
*after clinician consent, near initial clinician training 

Protocol Number 2000030199 
19 
implementation. In order to protect the confidentiality of parents consenting to the study, the 
study team will not share the names of consented families with clinicians in order to access 
their PCOMS data. Instead, we will use aggregate, deidentified PCOMS data at the clinician 
level to estimate MBC fidelity. This is a HIPAA-compliant site developed specifically for 
secure progress monitoring data entry and viewing my clinicians and their clients.  
Parent-reported data collected via study phone calls will be recorded in “real time” by 
study team members on the Parent Survey Form either in hard copy form or electronically. 
All hard copy record forms will be scanned in and saved electronically on a Yale secure 
server before hard copy records are shredded. Parent-reported data will be entered into 
SPSS and saved on a Yale secure server.  
The highest standards of participant confidentiality will be kept, and no participants will have 
identifiable information available. Computerized records of data are kept in a password-only 
accessible computer in a locked room. Appropriate firewalls and protections of computerized 
data are maintained to ensure that entry by those other than research personnel is not 
possible. Only study team members (the PI, her mentor, and her research assistant) will 
have access to individually identifiable private information about human subjects, which will 
only be used for tracking and follow-up purposes until data collection and compensation 
procedures are finalized. 
 
5.1.2 Adverse Events Definition and Reporting 
This protocol presents minimal risks to the subjects and Unanticipated Problems Involving 
Risks to Subjects or Others (UPIRSOs), including adverse events, are not anticipated. In the 
unlikely event that such events occur, Reportable Events (which are events that are serious 
or life-threatening and unanticipated, or anticipated but occurring with a greater frequency 
than expected) and/or Unanticipated Problems Involving Risks to Subjects or Others that 
may require a temporary or permanent interruption of study activities will be reported 
immediately (if possible), followed by a written report within 5 calendar days of the Principal 
Investigator becoming aware of the event to the IRB (using the appropriate forms from the 
website) and any appropriate funding and regulatory agencies. Dr. Connors will apprise 
fellow investigators and study personnel of all UPIRSOs and adverse events that occur 
during the conduct of this research project.  Dr. Connors will be responsible for monitoring 
the data and assuring protocol compliance and safety reviews. She will conduct these 
reviews during her weekly supervision with research staff. Study staff will be trained to 
inform the PI of any adverse event or unanticipated problem in a timely fashion. Dr. Connors 
will evaluate whether the study should continue unchanged, require 
modification/amendment, or close to enrollment based on the severity of the event. 
5.2 
Study Schedule 
Clinician involvement will require approximately 9 hours of project activities during the 2021-
22 or 2022-23 school year. All activities are part of their routine services and district-
approved professional development schedule. Clinician involvement includes 6 hours of 
training and ongoing consultation plus 2 hours and 30 minutes of online surveys.  
 
Project activities listed on Clinician Information Sheet will include: 
• A 3-hour interactive training on measurement-based care in schools. This training 
will include how to collect, score, and use student- and parent-reported progress 
measures with students and families to improve engagement, personalize services 
and inform intervention planning and adjustments throughout the school year. 
• Six, 60-minute, monthly post-training consultation calls for six months following 
the training. These calls will be with other participating clinicians and the PI to 
facilitate peer learning and support for implementing measurement-based care in 

Protocol Number 2000030199 
20 
schools. Strategies and adaptations to match the school setting and student and 
family strengths and needs will be discussed. 
• Four, 30-minute online feedback surveys from clinicians to understand 
measurement-based care acceptability, appropriateness, and feasibility in schools, 
plus clinician use of measures and related experiences with implementation. These 
will occur before the first training and 3 months, 6 months, and 9 months after the 
training throughout the school year. You will receive a $25 Amazon gift card for each 
survey you complete.  
• Collecting student progress data. You will be asked to collect brief measures of 
student progress periodically throughout treatment, review it with the student and 
family and document this in an online system that we will grant you access to called 
Better Outcomes Now (BON). Collecting and discussing these measures will take 5-
10 minutes of session time, 2-4 times per month and will be part of the regular 
services you already provide. We ask that you do this with at least five (5) of your 
students. 
• Parent recruitment. We will ask you to refer at least five (5) parents to the research 
study component of this project. We will ask you to present the opportunity to 
participate in the study to the parents of students you work with. Parent participation 
is optional and will involve completing four brief phone check ins with one of our 
study team members during the school year. Parents will receive a $25 Amazon gift 
card for each phone check in they complete. If the parent agrees to participate, the 
study team will also review their BON account for documentation of measures being 
collected and discussed. We will do this by asking you to provide our study team with 
their anonymous BON code so we know who the student is in BON. (As 
administrators, we can access all BON data, but it is deidentified). Records review in 
BON will occur confidentially by the study team and only if parent permission is 
provided.  
 
Parent/guardian participation will require approximately 1 hour, 40 minutes of study 
participation during the 2021-22 or 2022-23 school year. This includes phone consent and 
four phone interviews of approximately 20-25 minutes each at baseline, 3 months, 6 months 
and 9 months.  
5.3 Informed Consent 
Consent forms describing in detail the study intervention, study procedures, and risks are 
given to the participant and written documentation of informed consent is required prior to 
starting intervention/administering study intervention. Study information will be provided via 
online survey for clinicians and study consent will be provided over the phone for parents. 
The following consent materials are submitted with this protocol: 
• 
Clinician Information Form 
• 
Parent Consent from  
 
5.2.1 Screening (if applicable)  
Not applicable. 
5.2.2 Recruitment, Enrollment and Retention (if applicable) 
5.3.3  Clinician Participants 
Clinicians (N=50) who provide psychosocial interventions to K-12 students in our identified 
districts will be informed of the project using the following steps: 
1. Clinicians receive an email from their supervisor (e.g., a “site leader” which could be 
a district or community mental health agency administrator, depending on whether 

Protocol Number 2000030199 
21 
they are school- or community-employed) that the FOCUSS study team will contact 
them directly about participation. See “Clinician Notice of FOCUSS Study Email from 
Site Leader” . This will include “FOCUSS Flyer – Clinician”.  
2. Site leaders will send the study team a list of names and email addresses of eligible 
clinicians.  
3. Study team members will email clinicians directly with the “FOCUSS Flyer – 
Clinician” form attached. The email message will remind clinicians about the project 
and include a link to the first online survey. See “Qualtrics Invitation Email from Study 
Team – Clinicians” 
4. Clinicians can view the Clinician Information Sheet and FOCUSS Flyer-Clinician on 
the first page of the online Qualtrics survey and provide contact information for the 
study team for any questions or clarifications they may have. 
5. Two (2) reminders will be sent to clinicians who do not open the survey link. All 
clinicians will receive emails about the 3 month, 6 month, and 9 month surveys 
regardless of baseline survey completion. All follow-up surveys will include a link to 
the Clinician Information Sheet and FOCUSS Flyer-Clinician on the first page, as a 
reminder, and for any clinicians completing a follow-up survey as their first survey.  
 
Parent/Guardian Participants 
Consent will be collected over the phone for parents. Parent/guardians (N=150) who have a 
child receiving school-based mental health treatment from one of the participating clinicians 
will be recruited and consented using the following steps:  
1. The clinician and/or designated site clinical staff will ask the parent/guardian of each 
recruited student if they are willing to be contacted by the study team. They will do 
this using the “Clinician Parent/Guardian Recruitment Script”. 
2. If so, the parent’s contact information will be shared with relevant study team 
member using the “Research Interest Sheet – FOCUSS”, who will contact the parent 
via phone and/or email and then introduce the study, read the consent form, and 
request informed consent over the phone.  
3. The study team member will indicate the parent’s consent (yes or no), and if yes, 
document the parent’s name, relationship to child, date, and their name and 
signature.  
Study Visits (if applicable) 
We will conduct study phone calls with parents at baseline, 3 months, 6 months, and 9 
months. Each phone call will last approximately 20-25 minutes. Parents will be asked to 
provide ratings over the phone about their engagement in their child’s mental health services 
and their child’s mental health symptoms.  
5.3 
Statistical Method 
5.3.1 Statistical Design 
Quantitative Analyses 
Quality control checks of the data and examination of variable distributions will be performed 
prior to formal analysis. There are three types of outcomes: (1) clinician report of 
acceptability, appropriateness, and feasibility (implementation outcomes), (2) clinician use of 
MBC (also an implementation outcome, fidelity), and (3) student engagement, progress and 
outcomes (effectiveness of MBC). For (1) and (2), the primary analysis will be a multilevel 
growth model with time (Level 1) nested within clinician (Level 2) for each of the 
implementation outcomes as they are clinician self-reported on clinician surveys. We will 
examine linear change of time as a continuous measure as well as time by time transitions 
across the longitudinal intervals of data collection. Attitudes toward MBC and other 

Protocol Number 2000030199 
22 
independent variables (e.g., clinician professional and demographic characteristics, 
consultation call attendance, school district) will be entered into the models to examine 
correlations. In particular, we hypothesize parallel growth between attitudes and use of MBC 
over time.  
For (3) we will conduct a 3-Level multilevel growth model with time (Level 1) nested within 
student/parent reported outcome (Level 1) nested within clinician (Level 3). Models will be 
estimated for each parent-reported outcome (e.g., parent-reported engagement, parent-
reported symptoms on the PSC-35). Independent variables that will be entered as covariates 
in these models include MBC implementation outcomes, with the specific hypothesis that 
MBC use will be predictive of positive engagement and symptom reduction.  
Also, for (3), the ORS and SRS are assessed at each session, hence, a mixed effects 
regression model will be used that accounts for intra-student correlation across student 
sessions and clustering of students within clinician. The unit of analysis will be session, 
however in primary analyses, to avoid an overly complicated model, session will not be 
modeled as a fixed effect in the regression model for the mean. Fixed effects will include 
time (baseline and post MP assessment for MP1, 2, or 3) and treatment condition (i.e. 
whether the clinician’s agency is under the intervention or control condition).  
Results of all the above analyses will be expressed in terms of 95% confidence intervals for 
effects and contrasts which together with estimates of intra-clinician correlation and variance 
will inform the power analysis for a larger trial.   
Mixed Methods Analyses 
For (2) we will also analyze chart review data. We will develop and test our MBC fidelity 
rubric on a subset of consented students at midpoint in the school year to assess intercoder 
reliability, test our coding form on the text of progress notes, and refine both our coding form 
and process for accessing records in an efficient, reliable manner. These analyses will yield 
mixed methods data on clinician use of MBC to be triangulated with clinician self-reported 
use of MBC on surveys to evaluate relative advantages and disadvantages of these 
methods to assess use of MBC. Qualitative data generated from chart reviews will be 
analyzed using Rapid Qualitative Analysis whereby two coders plus the PI will create 
summaries of chart review findings per student using a standard summary form, which is 
then entered into a matrix for the entire sample to be reviewed, coded iteratively, and 
synthesized for main themes and sub codes. These findings will directly inform the future 
R01 application methods regarding the collection and use of chart review and/or clinician 
surveys to assess MBC use/fidelity following implementation supports.  
5.3.2 Sample Size Considerations 
We plan to enroll 50 SMH clinicians and up to 150 parents of children or adolescents with 
mental illness. The rationale for this sample size is based on having a large enough sample 
of clinicians trained that a range of implementation outcomes can be observed and 
described as pilot data for a larger-scale, fully-powered study. The rationale for the 150 
students is based on our prior research in quality improvement and implementation research 
in schools, where it is more feasible to introduce a new practice to clinicians by telling them 
they can select a subset of their caseload for MBC implementation. We will outreach to 
students the clinician reports they are using MBC with. Selection of a subset of cases will 
involve a random selection of recently consented cases. 
5.3.3 Planned Analyses 
Please see details in 5.4.1  
5.3.4 Analysis of Subject Characteristics (if applicable) 

Protocol Number 2000030199 
23 
Quality control checks of the data and examination of variable distributions will be performed 
prior to formal analysis. Descriptive statistics and intercorrelations will be examined for all 
datasets. Demographic characteristics of clinician participants and parent/guardian 
participants will be examined and reported in all scholarly reports and products. If any 
demographic characteristics are significantly related to implementation or effectiveness 
outcomes, they will be entered as control variables in the linear mixed analyses.  
5.3.5 Interim Analysis (if applicable) 
Not applicable. 
5.3.6 Handling of Missing Data 
Missing data will be analyzed to examine any patterns of missingness not at random, 
including case-wise (i.e., overall percentage, patterns, various longitudinal intervals of 
missingness) and variable-wise (i.e., overall percentage, patterns) missing values analysis. 
Linear mixed modeling is robust to missing data and does not require listwise deletion of 
cases with missing data. Therefore, if missing data are missing at random (MAR) all data will 
be used. Otherwise, analyses with and without missing data will be compared to examine 
whether results differ when including cases and variables with data missing not at random.  
6 Trial Administration  
6.1 
Ethical Considerations: Informed Consent/Assent and HIPAA Authorization 
Consent forms will be Institutional Review Board (IRB)-approved and the participant/legally 
authorized representative (LAR) will be asked to read and review the document.  
 
For clinicians, the online survey will contain the Clinician Information Sheet to explain the 
project and answer contact information is available for study team members if any questions 
that may arise. A copy of the Clinician Information Sheet will also be emailed to the 
participants/LAR for their records 
 
For parent participants, consent forms will be Institutional Review Board (IRB)-approved and 
the participant/LAR will be asked to read and review the document. The study team member 
will explain the research study to the parent or guardian and answer any questions that may 
arise. This conversation will take place in a private room or over a private zoom call due to 
COVID-19 restrictions. Participants/LAR will have the opportunity to carefully review the 
written consent form and ask questions prior to signing. The participants/LAR will have the 
opportunity to discuss the study with their family or think about it prior to agreeing to 
participate. The parent or guardian will indicate their consent via verbal authorization with 
the research study team member prior to any procedures being done specifically for the 
study. Participants/LAR will be informed that participation is voluntary and that they may 
withdraw from the study at any time, without prejudice. A copy of the informed consent 
document will be given to the participants/LAR for their records 
 
Additional consent details are as follows: 
- 
No deception will be used.  
- 
Clinicians will be compensated up to $100 via Amazon Gift Cards for their 
participation. Parent participants will be compensated up to $100 via Amazon Gift 
Cards  for their participation upon completion of follow up phone calls with the study 
team.  
- 
All research records will be kept in password-protected, Yale-approved network 
locations in a locked room with participants’ evaluation study materials identified by 
code only. A separate file will hold the password. Appropriate firewalls and 
protections of computerized data are maintained to ensure that entry by those other 

Protocol Number 2000030199 
24 
than research personnel is not possible. Only study team members (the PI, her 
mentor, and her research assistant) will have access to individually identifiable 
private information about human subjects, which will only be used for tracking and 
follow-up purposes until data collection and compensation procedures are finalized. 
- 
Participants will not be personally identified in any publications or reports of the 
study. Any data used will be re-copied to research files with the participant identified 
by code only. The highest standards of participant confidentiality will be kept, and no 
participants will have identifiable information available.  
- 
Parents enrolled will be asked to agree report symptom data on the Pediatric 
Symptom Checklist, 35 items (PSC-35) as well as to access their students’ progress, 
goals, and related services data from documentation maintained by their clinician.As 
such, at least one family member or guardian will consent to secondary data 
collection and record review from their students’ treatment record. 
- 
Child consent will not be obtained. 
6.2 
 Institutional Review Board (IRB) Review 
The protocol will be submitted to the IRB for review and approval. Approval of the protocol 
must be obtained before initiating any research activity. Any change to the protocol or study 
team will require an approved IRB amendment before implementation. The IRB will 
determine whether informed consent and HIPAA authorization are required. All reportable 
events and unanticipated problems will be reported to the IRB by the PI within 5 working 
days after she becomes aware of the event or problem. This study is prospective research, 
conducted in established and commonly accepted educational settings where children 
receive mental health treatment services. This study focuses on implementation of a best 
practice in clinical service delivery to improve the quality of usual care. A study closure 
report will be submitted to the IRB after all research activities have been completed. 
 
6.3 
Subject Confidentiality 
Participant confidentiality and privacy is strictly held in trust by the participating investigators 
and their staff. Therefore, the study protocol, documentation, data, and all other information 
generated will be held in strict confidence. No information concerning the study, or the data 
will be released to any unauthorized third party without prior written approval of the sponsor. 
 
All research activities will be conducted in as private a setting as possible. 
 
The study monitor, representatives of the Institutional Review Board (IRB), or regulatory 
agencies may inspect all documents and records required to be maintained by the 
investigator, including but not limited to, medical records (office, clinic, or hospital) for the 
participants in this study. The clinical study site will permit access to such records. 
 
The study participant's contact information will be securely stored at each clinical site for 
internal use during the study. All data will be stored on Yale secure file storage platforms 
and servers, with password protection and accessible only to the study team. Our team will 
not retain any paper copy study records; any paper copies will be scanned in, saved 
electronically, then shredded. At the end of the study, all records will continue to be kept in a 
secure location for as long a period as dictated by the reviewing IRB, Institutional policies, 
or, if applicable, sponsor requirements. 
 
Study participant research data, which is for purposes of statistical analysis and scientific 
reporting, will be transmitted to and stored at Yale University, Department of Psychiatry, 
Division of Prevention and Community Research. This will not include the participant's 
contact or identifying information. Rather, individual participants and their research data will 
be identified by a unique study identification number. The study data entry and study 

Protocol Number 2000030199 
25 
management systems used will be secured and password protected. At the end of the study, 
all study databases will be de-identified and archived at Yale University.  
 
6.4 
Deviations/Unanticipated Problems 
A protocol deviation is any noncompliance with the study protocol. The noncompliance may 
be either on the part of the participant, the investigator, or the study site staff. As a result of 
deviations, corrective actions are to be developed by the site and implemented promptly. 
 
It is the responsibility of the site investigator to identify and report deviations 5 working days 
of identification of the protocol deviation. All deviations must be addressed in study source 
documents, reported to the study sponsor, and the reviewing Institutional Review Board 
(IRB) per their policies. 
 
Unanticipated problems involving risks to participants or others include, in general, any 
incident, experience, or outcome that meets all of the following criteria: 
• 
Unexpected in terms of nature, severity, or frequency given (a) the research procedures 
that are described in the protocol-related documents, such as the Institutional Review 
Board (IRB)-approved research protocol and informed consent document; and (b) the 
characteristics of the participant population being studied; 
• 
Related or possibly related to participation in the research (“possibly related” means 
there is a reasonable possibility that the incident, experience, or outcome may have 
been caused by the procedures involved in the research); and 
• 
Suggests that the research places participants or others at a greater risk of harm 
(including physical, psychological, economic, or social harm) than was previously known 
or recognized. 
 
The investigator will report unanticipated problems (UPs) to the reviewing Institutional 
Review Board (IRB) and to the study sponsor. The UP report will include the following 
information:  
• 
Protocol identifying information: protocol title and number, PI's name, and the IRB project 
number; 
• 
A detailed description of the event, incident, experience, or outcome; 
• 
An explanation of the basis for determining that the event, incident, experience, or 
outcome represents an UP; 
• 
A description of any changes to the protocol or other corrective actions that have been 
taken or are proposed in response to the UP. 
 
To satisfy the requirement for prompt reporting, UPs will be reported using the following 
timeline: 
• 
UPs that are serious adverse events (SAEs) will be reported to the IRB and study 
sponsor, if applicable within [insert timeline in accordance with of the investigator 
becoming aware of the event. 
• 
Any other UP will be reported to the IRB and study sponsor within [insert timeline in 
accordance with policy] of the investigator becoming aware of the problem. 
• 
All UPs should be reported to appropriate institutional officials (as required by an 
institution's written reporting procedures), the supporting agency head (or designee), and 
the Office for Human Research Protections (OHRP) within & [insert timeline in 
accordance with policy] of the IRB's receipt of the report of the problem from the 
investigator. 
6.5 
Data Quality Assurance 
The study team will be appropriately trained in handling and reviewing participant data for 
data quality assurance. The PI will use scaffolded supervision and oversight (i.e., modeled 

Protocol Number 2000030199 
26 
with debrief, then co-completed, then observed with feedback followed by check ins at 
weekly study team meetings) to ensure quality control and data assurance as it pertains to 
participant recruitment, tracking and compensation as well as data collection, storage, 
management, and analyses.  
6.6 
Study Records 
Recruitment Flyers and Communications: 
1. Clinician Notice of FOCUSS Study Email from Site Leader 
2. Qualtrics Invitation Email from Study Team – Clinicians 
3. FOCUSS Clinician Information Sheet 
4. FOCUSS Flyer – Clinician 
5. FOCUSS Flyer - Parents  
6. Clinician Parent/Guardian Recruitment Script 
7. Research Interest Sheet - FOCUSS 
Consent Forms: 
8. FOCUSS - Parent Consent 
Data Collection Surveys and Forms: 
9. FOCUSS - Clinician Welcome Survey 
10. FOCUSS - Clinician Follow-Up Survey 
11. FOCUSS - Parent Phone Survey 
6.7 
Access to Source 
Source data will be maintained per Medical Records policy in a password protected, secure, 
Health Insurance Portability and Accountability Act (HIPAA) compliant, web-based electronic 
database with a built-in audit trail. 
 
Only Institutional Review Board (IRB) approved research team members who have current 
HIPAA and Collaborative Institutional Training Initiative (CITI) Good Clinical Practice (GCP) 
and human subject’s protection training will be authorized to access records. 
6.8 
Data or Specimen Storage/Security 
See Section 5.5.1 “Secure Data Recording and Maintenance” .  
6.9 
Retention of Records 
Study records (both deidentified data and password-protected file with links to identifiers) will 
be retained for a minimum of three (3) years after the completion of the study. All records will 
be electronic. The PI (Dr. Connors) will need to be contacted to move or destroy the records. 
The master list linking the unique study identification number to the research data will be 
destroyed after all analyses are completed.   
6.10 Study Monitoring 
The data safety and monitoring plan is consistent with what is required for clinical trials 
funded by NIMH, as follows: 
The PI (Connors) is ultimately responsible for study monitoring to ensure the safety of 
participants and the validity and integrity of the data. With assistance from the primary 
mentor (Tebes), the PI will ensure that all individuals involved in the research are familiar 
with the data and safety monitoring plan and have adequate systems in place for adverse 
events reporting, data integrity, confidentiality, and protection of participants’ safety.  
Data and Safety Monitoring will be overseen by a Data and Safety Monitoring Board (DSMB) 
established for this study. The Board will be comprised of three investigators at Yale with 
relevant research expertise in mental health services research and/or implementation 

Protocol Number 2000030199 
27 
science who are two full professors and one associate professor of psychiatry (psychology). 
One member’s expertise is in implementation science and community-engaged research 
with children and families and is a former member of Yale School of Medicine’s IRB (the 
Yale Human Subjects Committee, or HSC). Another member’s expertise is in school-based 
research focused on the implementation and evaluation of social emotional learning 
programs with children, families, and school personnel. The third member’s expertise is in 
applied, community-partnered research to promote mental health and wellness and prevent 
substance use with a focus on issues pertaining to equity, social justice and systemic 
oppression for marginalized and underrepresented groups. All are knowledgeable about the 
responsible conduct of research with children, families, and community-based organizations. 
The DSMB will meet annually in person to review study procedures, including: 1) the study 
protocol; 2) the consent form; 3) occurrences of side effects/adverse events; and 4) the 
study data management system. As needed, the three investigators comprising the DSMB 
will meet without ex-officio members for closed discussions about serious adverse events 
(SAE). As needed, the DSMB will review de-identified data, but may review identified data 
when discussing SAEs. The DSMB will also be notified of the occurrence of any adverse 
events (AE) and SAEs, and may choose to meet on an ad hoc basis when notified of either 
of these events. DSMB members will also be notified when AEs and SAEs are reported to 
the Yale IRB. 
6.11 Study Modification 
Any study modifications will be submitted to the IRB for review and approval with a revised 
protocol. Changes will not be implemented in the study until modifications are approved. 
6.12 Study Completion 
The study will be completed on or before June 30, 2023. Data collection will be completed 
by June 30, 2023, with data cleaning, analysis and reporting to follow. We will notify the IRB 
when the study is completed and submit a closure report or other required paperwork at that 
time to close the study.  
6.13 Funding Source 
Salary support and other research funding for this study is provided by the National Institute 
of Mental Health (K08 MH116119).  
6.14 Conflict of Interest Policy 
The independence of this study from any actual or perceived influence, such as by the 
school district (for SMH clinicians) and SMH clinicians (for parents), is critical. Therefore, any 
actual conflict of interest of persons who have a role in the design, conduct, analysis, 
publication, or any aspect of this trial will be disclosed and managed. Furthermore, persons 
who have a perceived conflict of interest will be required to have such conflicts managed in a 
way that is appropriate to their participation in the trial. The study leadership in conjunction 
with the appropriate conflict of interest review committee has established policies and 
procedures for all study group members to disclose all conflicts of interest and will establish 
a mechanism for the management of all reported dualities of interest. 
 
Any investigator who has a conflict of interest with this study (patent ownership, royalties, or 
financial gain greater than the minimum allowable by their institution, etc.) must have the 
conflict reviewed by a Quality Assurance Committee from YCCI or the Psychiatry 
Department with a Committee-sanctioned conflict management plan that has been reviewed 
and approved by the study sponsor prior to participation in this study. All investigators will 
follow the applicable conflict of interest policies. 
6.15 Publication Plan 

Protocol Number 2000030199 
28 
Scholarly publications and presentations resulting from this study will only report participant 
data in aggregate. Primary and secondary outcomes will be analyzed by subgroups of 
clinicians and parents, which may further be disaggregated by district, child age, and other 
independent variables. No analyses will report subgroups small enough such that any 
individual participant would be potentially identifiable (e.g., school counselor in an 
elementary school in a named district may be too identifying in a small district). If illustrative 
cases appear in publications, presentations or other reports, care will be taken to ensure 
details provided are in no way a disclosure of identifiable information. The PI holds primary 
responsibility for publishing the study results. Per the funder’s guidelines, all published 
results will be made available in open access format.