Patient Centered Enhancements in School Behavioral Health: A Randomized Trial
Phase: NA
Conditions: Mental Health
Sponsor: University of South Carolina

Patient Centered Enhancements in School Behavioral 
Health: A Randomized Trial 
NCT number: NCT03901274 
Last date approved: 5/26/2023

PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
1
PCORI Cycle 1 2018 Broad PFA: Research Plan Template 
PCORI RESEARCH PLAN TEMPLATE 
RESEARCH STRATEGY 
A. Specific Aims. Will test the incremental impacts of evidence-based Partnership enhancements to a framework for
evidence-based practice (EBP) in school behavioral health (SBH) termed Clinical Services Supports in a four-year
intervention for students in middle school.
●
Aim 1: Evaluate the impact of the Partnership intervention in increasing the number of students and families
receiving services and expressing satisfaction with them.
●
Aim 2: Evaluate the impact of the Partnership intervention on students’ social, emotional/behavioral (EB) and
academic outcomes.
The original research plan included a third study aim: Evaluate the follow-up effects of the intervention on social, EB, 
academic and health outcomes in a sample of students followed into high school. This aim was removed in a 
subsequent revision of the research plan (May 2023) due to significant challenges in contacting students who have 
been transferred to a new therapist in high school or who have been discharged from services. Additional references 
to Aim 3 have been removed from this plan to reduce confusion, but will be preserved in the appendix.  
B. Background. There is an alarming gap between the EB needs of youth and available services, with up to one third of
youth needing mental health intervention (1-3) but less than one third receiving any services, with more significant gaps
for cultural/ethnic minorities and other underrepresented groups (4).  When EB problems in children and youth go
undetected and/or untreated, they tend to cause functional impairment that persists over time (5,6) contributing to
problems including school failure, dropout, underemployment, involvement in the criminal justice system, and early
death (7-10).  Middle school is a particularly important time in child and adolescent development (11,12), with more
serious EB problems often emerging during this time period (13,14), and offering an important window to provide
effective services to youth to help to assure a positive life trajectory through K-12 education and beyond (15,16).  School
behavioral health (SBH) programs, involving school-community-mental health partnerships increase access to needed
mental health services for all students, and for minority youth (16).  Increasingly, education and mental health systems
are collaborating to provide effective programs within a multi-tiered system of support (MTSS) involving promotion and
prevention for all students at Tier 1; early intervention for students showing beginning problems or with elevated risk at
Tier 2, and treatment services for students presenting more intensive EB problems at Tier 3 (17).
For over a decade, the Principal Investigator (PI), Weist, and Co-Investigator (Co-I) and MD site PI, Hoover have 
been involved in a range of research, practice and policy efforts to improve the depth, quality and impact of Tier 1, 2, 
and 3 services within the MTSS (17) and have led two consecutive randomized controlled trials (RCTs; 2003-07; 2010-15) 
funded by the National Institute of Mental Health to test a Clinical Services Support (CSS) framework for enhancing 
evidence-based practice (EBP) in SBH, with this being the core intervention to be implemented in middle schools 
involved in this study, termed the Wellness Group.  The CSS framework packages together EBPs of family engagement, 
modular evidence-based practice, quality assurance, and implementation support. The first trial showed improvements 
in quality services and EBP (18), and the second trial replicated these findings and also documented increased numbers 
of students/families receiving effective services (19). Through a recently completed PCORI engagement award, intensive 
qualitative analysis with families, clinicians and diverse stakeholders documented two additional critical theme areas to 
further improve SBH services – enhancing mental health literacy and stigma reduction, and improving family-school-
mental health partnerships.  Of more than 18 specific recommendations of stakeholders, these were the two top-rated 
services enhancements, with stakeholders emphasizing that these enhancements will increase the likelihood that 
students in need will receive services (through mental education/stigma reduction) and that these services will be more 
acceptable and effective (throraugh enhanced partnerships with families/students). Families and youth may not use 
mental health or SBH services related to poor knowledge of them, stigma, and perceptions of unequal, and even 
condescending relationships with education and mental health system professionals. This study addresses a critical gap 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
2 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
in evidence on the influence of these two partnership enhancements as factors that we hypothesize will enhance 
effectiveness and impact of the evidence-based strategies included in the Wellness intervention. The intervention 
condition, or Partnership Group, will consist of these two stakeholder enhancements in addition to the CSS framework. 
As such, we believe this study will support improved decision making and resource allocation by education and mental 
health systems in placing greater priority on evidence-based SBH and predicted amplifying factors (the two stakeholder 
enhancements (SE)).  The primary decisional dilemma addressed is should significant efforts be made to enhance 
knowledge and reduce stigma, and improve collaborative approaches to amplify the impacts of evidence-based SBH, or 
are these efforts not necessary? 
 
 
The study will test the contribution of these two SE’s in improving the productivity of clinicians and positive impact of 
SBH services. The study involves 22 middle schools, with 10 in Maryland (MD) and 12 in South Carolina (SC), with all 22 
schools receiving the Wellness intervention, and half of them (5 in MD and 6 in SC) randomly assigned to the Partnership 
condition, with impacts explored through a longitudinal randomized controlled trial (RCT) operating for three years in 
grades 6-8.  We hypothesize that the Partnership enhancements will help to foster a more supportive culture for SBH, 
and increase the impact of evidence-based services toward significantly improved social, EB, and academic outcomes for 
participating students, and provide a critical precedent for prioritizing patient and stakeholder involvement and active 
guidance in SBH, an increasingly prominent field. 
C. Significance. The proposed study is positioned to offer essential guidance to collaborative partnerships involving 
schools, families and youth, mental health and other youth serving systems to maximize effective decision making on 
the ongoing improvement of SBH programs toward enhanced social, EB, and academic outcomes.  To enhance SBH with 
significant and meaningful input from diverse stakeholders, the PI and colleagues have been interacting with 
stakeholders including youth and families receiving services for EB problems, clinicians, researchers and diverse youth-
serving system staff and leaders to obtain guidance on the most important enhancements to these services.  Beginning 
in 2015, forums were held with these stakeholders and five initial critical themes were identified (on partnerships, 
school-wide approaches, quality, effective implementation, and cultural competence). The PI and team were fortunate 
to receive a PCORI Eugene Washington Engagement award in late 2015 enabling research forums with diverse 
stakeholders held throughout SC in 2016 on each of these five themes, and formal qualitative analyses of them. From 
analysis of transcripts of these research forums, two overarching themes representing critically important 
enhancements to SBH services were identified by patients and stakeholders: 1) Broadly train all school stakeholders in 
mental health and purposefully reduce the stigma involved in seeking SBH services, and 2) increase partnerships among 
youth, families, schools and mental health clinicians.  For both of these critical enhancements the research team has 
carefully identified evidence-based interventions that will augment the existing evidence-based CSS framework for SBH.  
While each of the Wellness intervention components is evidence-based and commonly implemented (16), the impact of 
services has been limited by underutilization of services related to poor knowledge of mental health and stigma (20,21), 
and constrained quality of services related to limited partnerships among patients, families, and providers (20,22).  We 
hypothesize that findings from this study will have significant impacts on practice and policy in the SBH field through 
inclusion of the Partnership enhancements that address these limiting factors and enable increased productivity and 
impact of services.    
 
 
This application is highly responsive to PCORI's call for comparative effectiveness research that explores alternative 
features of healthcare systems.  While the SBH field has developed considerably in recent years (16-18), efforts to 
significantly involve patients and diverse stakeholders in improving the depth, quality and impact of services have been 
severely lacking (20).  This will be the first study to test SBH enhanced by critical recommendations from families, 
educators, clinicians and other stakeholders that we predict will be associated with new levels of productivity and 
impact.  To obtain stakeholder guidance on the Partnership intervention, a survey was given to attendees of the 2018 
Southeastern School Behavioral Health Conference. The survey asked participants to rate 10 service components, 4 
Wellness and 2 Partnership and 4 alternative components (e.g., "helping students have insight into their problems").  
Ninety-two attendees from a wide variety of disciplines and backgrounds, including school administrators, educators, 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
3 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
school counselors, school-based clinicians, psychologists and research staff, and community and family organizations 
(including patients and parents), and school resource officers, responded. The six Wellness and Partnership components 
(Mean = 5.7; α = 0.89) were on average rated as significantly more important than the four comparison components 
(Mean = 5.3; α = 0.79): t (92) = 96.29, p < .01. The minimum, maximum, mean, and standard deviation (SD) for each 
component’s ratings are presented in Tables 1 and 2 in the Appendix. 
D. Study Design/Approach. The study will involve 22 middle schools (12 in SC, 10 in MD) in a cluster-RCT, with all 
schools receiving the Wellness intervention and half of the schools (6 in SC, 5 in MD) randomized to receive enhanced 
intervention including the two SEs (Partnership).  Stratified random assignment of schools to the Wellness Group or the 
Partnership Group will be conducted. We will then use validated procedures (23, 24) to further stratify schools within 
each state. We will construct a composite inflate-suppress score derived from school data on rurality, average number 
of students per grade, percent white, percent receiving free/assisted lunch, and average school test performance. 
Rurality within the two study sites will be operationalized by using a federally developed classification system used by 
The US Census Bureau (i.e., a 9-point, “Rural-Urban Continuum”). The coding scheme will distinguish three metro county 
categories based on population size, and six non-metro county categories based on degree of urbanization, as well as 
adjacency to other areas and metro proximity. Per operationalization of rurality in the federally-based coding 
continuum, we can confirm that the SC study sites contain county classifications of rural and suburban, while the MD 
site is exclusively urban. We will separate this composite into three strata and randomly assign schools from these strata 
to receive either the Wellness Group or the Partnership Group.   
 
In the 5-year longitudinal design, Year 1 will involve administrative start up, collection of pre-assessments and 
intervention launch; Years 1-4 will involve intervention versus comparison while participants are in 6th through 8th 
grades. Year 5 will involve analyses and dissemination. A comprehensive implementation manual will guide the study 
and will be updated and improved regularly through active guidance of a Stakeholder Advisory Board (SAB, see Tables 3 
and 4 in Appendix). The SAB will provide significant input as this manual is developed during the study start-up period 
from January through July, 2019, and progressively improved through study duration (see Gantt chart in Figure 1 of the 
Appendix for a review of all study activities in time).  
      Conceptual Framework. We predict the Partnership intervention will improve school culture so that staff and 
students are more knowledgeable about mental health, less concerned about stigma, and more proactive in identifying 
students in need and connecting them to more intensive Tier 2 or Tier 3 services; more students/families receive Tier 2 
and 3 services and receive them in a more timely manner; students and families are more invested in services, 
empowered to influence treatment directions, experience greater therapeutic alliance with SBH staff, and receive a 
higher dose of evidence-based services; and, impacts of evidence-based services are amplified based on greater 
knowledge of mental health.  These factors will be associated with better performance on measures of social, EB, and 
academic functioning than students in the Wellness Group alone.    
       Engagement Plan. The engagement plan has been informed by lessons learned from the recent PCORI Eugene 
Washington Engagement award, and reviewer feedback from the prior application. The SAB has actively guided our 
work and this resubmission, with some members providing input since 2015, and new members (i.e., educators, 
additional family members, payers) added based on PCORI guidance.  The SAB has 13 members in SC and 10 in MD (23 
total) and includes: patients/caregivers (some of whom are end users at study schools), educators, mental 
health/community partners, public and private funders, and community partners (see Tables 3 and 4 in Appendix for a 
full listing of members by site).  Please note that the SAB is representative of study sites in relation to 
racial/ethnic/cultural background of students. The SAB will collaborate at multiple levels, as a full 23-member board, 
within SC and MD, and in three subgroups of patients/caregivers, educators, and mental health/community partners. 
The SAB subgroups will convene in bi-monthly conference calls for one hour to provide input and feedback in all three 
domains of the PCORI engagement rubric: Planning, Conduct and Dissemination. Each SAB member will be compensated 
$50 per hour for their time spent on the calls, as well as the time they spend reviewing materials offline (estimated at 6 
hours annually). Additionally, the SAB will meet in person once annually, to engage in more comprehensive discussion of 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
4 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
the study, with travel stipends planned for all travel expenses incurred to assure that travel is feasible for all members.  
In years 1, 3 and 5 the full 23-member board will convene in person, in years 2 and 4, the SC and MD SAB members will 
convene in separate locations but with opportunities for virtual audiovisual interaction. During year 2, the SC and MD 
SAB meetings will be held via Zoom video conference in lieu of an in-person meeting due to COVID-19.  The initial SAB 
meeting, focusing on introductions and planning the study (including dissemination), will be held in April, 2019, in 
advance of the 6th annual meeting of the Southeastern School Behavioral Health Community (SSBHC), which was 
established through the PCORI Engagement Award (see www.schoolbehavioralhealth.org). The final SAB meeting, 
focused on dissemination and scaling up of the Partnership intervention will be held in March 2023, at the 10th annual 
Southeastern School Behavioral Health conference, started by the PI and colleagues in 2014. The SAB will be co-directed 
by Dr. Robert Stevens, who has significant experience with PCORI research and Dr. Reverend Al Hathaway, a pastor at 
Union Baptist Church in Baltimore, and a national leader in faith-community collaboration to improve human services. 
Patient engagement will be led by June Greenlaw who is both a parent partner as well as the engagement specialist on 
the previous engagement award. Greenlaw will be the point person for all SAB members and subgroup leads and will 
communicate with them frequently during the proposed study time period.   
     Engagement in Planning the Study. As reviewed above, associated with our prior PCORI engagement award, the SAB 
has been in place since 2015 and has guided our work up to the present, including eight research forums on five priority 
themes (partnerships, school-wide approaches, quality, implementation, cultural competence) and three priority 
populations (youth in child welfare and juvenile justice, and from military families). Systematic qualitative analysis of all 
focus groups confirmed these prioritized themes and the two with the highest frequency represent the two components 
in the Partnership intervention. For both enhancements we have identified evidence-based strategies for 
implementation. The SAB has recently been expanded for this proposal, and has endorsed our study plan and choice of 
comparators, the evidence-based Wellness intervention for 11 comparison schools (6 in SC, 5 in MD), and the 
Partnership intervention for 11 target schools (6 in SC, 5 in MD). The SAB has also provided critical guidance to 
enhancing our engagement and dissemination plans, as well as plans for effectively reaching underserved and minority 
populations.  
     Engagement in Conducting the Study. During project start up, the SAB will identify an informal project name and logo 
that will be used on study materials, a website, ways to most effectively communicate with patients, families and 
stakeholders and ways for them to access information and materials from the study. The SAB will review 
parent/guardian recruitment letters and provide input to assure understandability and sensitivity of the language used, 
and will improve overview materials that will be used when orienting school and mental health staff to study 
procedures/resources.  Throughout implementation, the SAB will be provided with frequent progress updates including 
information on successes and challenges encountered related to recruitment, implementation of both interventions and 
data collection. The SAB will review preliminary (aggregated) data reports and assist in interpreting them to adjust the 
research strategy as needed.  In addition, data will be disaggregated for analysis of service receipt and impacts for 
groups varying on grade level, age, gender, race/ethnicity, and special education status.  If disparities in service receipt 
or outcomes are identified, the SAB will guide the study team in making adjustments to assure responsiveness in 
services. The SAB will also identify additional research questions to be pursued and will be encouraged to participate 
actively in supplementary studies. Additionally, during in-person meetings, we will seek feedback from the SAB on their 
level of engagement in the study and ways to improve engagement.    
    For the Partnership condition, two family advocates serving on the SAB (1 in MD and 1 in SC) will co-lead training for 
clinicians, school staff and selected parent leaders from each of the 11 schools at the beginning of each year, with 
emphasis on family partnerships and cultural humility. Parent leaders will be carefully selected, assuring they are racially 
representative of their school to mitigate barriers of mistrust from underserved and minority populations. Parent 
leaders will be compensated for the time spent to collaborate with fellow parents and school staff to communicate 
about and promote the study, as well as perform outreach to minority populations to increase their access to effective 
SBH services, as well as other programs and services available in the school and community.   

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
5 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
    Engagement in Quality Improvement and Dissemination. The Extension for Community Healthcare Outcomes (ECHO) 
model (25, 26) will be used to create hub and spoke learning networks between the research team and SAB (hub), and 
11schools implementing the Partnership intervention (spokes). Project Manager, Brooke Chehoski, along with an 
additional research specialist (Samantha Reaves) and two Project Family Leaders (Jenah Cason and Susan Tager) will 
attend ECHO training at the University of New Mexico. Guided by the SAB and research team, an ECHO hub team will 
develop training materials and coordinate case studies for the 10 Partnership schools, with each school including the 
clinician, parent leader, school-employed mental health staff (e.g., counselor, psychologist, social worker) and 
educators. The ZOOM platform (www.zoom.us) will be used to facilitate ECHO clinics, which will be scheduled three 
ECHO clinics throughout the school year. These will occur for Partnership schools for the four years of intervention vs. 
comparison (2019-2020, 2020-2021, 2021-2022, and 2022-2023 school years) and participant attendance will be tracked 
as a fidelity measure, with outreach to individuals as indicated. In addition, use of the ECHO model will help to provide 
SBH consumers and providers with rapid information on which interventions work best for particular challenges in a 
variety of subgroup populations. In the academic year following intervention vs. comparison (2024-2025) the ECHO 
program will be expanded to the 11 comparison schools to promote generalization of the study innovation.  Beginning in 
January of 2024, the ECHO program will be used with key stakeholder groups in SC, MD and nationally (e.g., family 
advocates, school psychologists, counselors, educators, physicians) to assist in dissemination. Dr. Eve Fields, a board-
certified psychiatrist with significant experience working with adolescents and a leader in using the ECHO model for 
improving psychiatry services in SC, will guide our use of the ECHO model.   
    The SAB will assist researchers with refining a preliminary dissemination plan during the in person meeting in Year 1 to 
assure that all study activities are conducted in a way that maximizes the potential for disseminating findings to SBH 
consumers and providers, with parent leaders having a significant role in assuring access to this information by diverse 
end users. In Year 5 of the study, SAB efforts will focus on interpreting findings and synthesizing the knowledge gained in 
both traditional ways (academic journals, conference presentations) as well as creative methods that may be more 
accessible for patients and stakeholders such as Infographics to be rapidly distributed in a variety of settings to a large 
audience. All presentations and publications will be identified and prepared using SAB guidance. Through resources of 
large regional/national networks led by PI Weist (SSBHC,  www.schoolbehavioralhealth.org) and Co-I/MD site-PI Hoover 
(CSMH, http://csmh.umaryland.edu) we estimate a reach of study findings and lessons learned (all guided by patients 
and stakeholders) to more than 100,000 leaders and stakeholders.   
 Comparators. At each of the two sites, SC and MD, there will be 5 schools/clinicians randomly assigned to the Wellness 
Group, and 6 schools/clinicians randomly assigned to the Partnership Group for 22 total participating middle schools. 
The grant will support one day per week of clinician time in all study schools. In addition, at each site there will be two 
supervisors/senior trainers, one for Wellness, and one for Partnership. Supervisory assignments will be kept separate for 
each condition, all training events/meetings for the two groups will be held separately and all participants will be asked 
to not share information with the other group. 
1. Clinical Services Support (Wellness) Intervention. The Wellness intervention involves four components:  1) 
Systematic Quality Assessment and Improvement using The SHAPE System (see below), developed by the Co-I/MD site-
PI Hoover and colleagues at the CSMH, 2) a family engagement intervention emphasizing clinicians working with families 
in a collaborative fashion, 3) modular evidence-based practice, involving targeting the most important cognitive-
behavioral skills to address common problems of anxiety, depression, trauma, and conduct problems, and 4) 
implementation support, involving SBH clinicians meeting twice per month for 60-90-minute training sessions for these 
above components.  The evidence base for each of these four components is strong (20, 27-32), with the Clinical 
Services Support representing an integration of them into a package, associated in two prior studies with higher levels of 
each of the four components, and enhanced clinical productivity and service receipt by students/families (18, 19). More 
detail on the Wellness intervention and its four components is provided here: Quality Assessment and Improvement. 
The School Health Assessment and Performance Evaluation (SHAPE) System (see www.theshapesystem.com) is a 
federally funded, public-access, web-based platform to support quality improvement in SBH. Using this system, school 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
6 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
teams are able to self-assess their progress in various domains of quality services using the SBH Quality Assessment Tool 
(SBHQAT; 29, 30) which includes seven domains and 62 items. Mean scores for items within each domain are classified 
as “emerging,” “progressing,” or “mastery” providing guidance to schools and districts on quality improvement 
strategies. Family Engagement. The family engagement intervention is based on research focused on improving 
retention of children and families in community mental health services (32). The program is based on Theories of 
Reasoned Action, and Planned Behavior (33), and includes emphases on problem identification, engagement skills, 
service delivery options, and treatment management skills. Modular Evidence-Based Practice.  To overcome many 
challenges associated with implementing treatment manuals (34,35), modular approaches (27,36) that provide clinicians 
with competency training in core practice elements for particular disorders are increasingly being utilized and showing 
superior impacts on provider preference and on student outcomes (37-39).  As used in the prior NIH-funded RCTs 
(18,19), the PracticeWise system (40) will be used by clinicians to guide their selection and implementation of evidence-
based cognitive behavioral therapy skills to address the most common child and adolescent disorders of anxiety, 
depression, conduct problems, and trauma related problems. Implementation Support.  An extensive literature has 
documented the importance of attending to the implementation of evidence-based practices to attain desired 
intervention outcomes (28,38,39). Based on this literature and guidance from the National Implementation Research 
Network, implementation support will include ongoing training in the three elements of quality improvement, family 
engagement, and modular evidence-based practice in twice monthly, 60-90-minute sessions led by two senior trainers, 
with one assigned to work with the Wellness Group and the other assigned to work with the Partnership Group.  During 
the spring 2020 semester, due to the impact of COVID-19, the Partnership teams in MD and SC began meeting together 
using Zoom technology. This decision was made quickly following decisions in both states to close public schools and to 
require non-essential employees to work from home. Given the early success of this model and in alignment with the 
teleECHO framework, the Partnership teams will continue to meet together for the duration of the study. In addition to 
streamlining preparation time and effort by senior trainers, this shift will align training content and delivery across sites. 
One meeting per month will be facilitated using the teleECHO format, which includes one or more case presentations by 
a clinician and a separate didactic instruction on study elements, facilitated by a content expert. Per prior studies 
(18,19), the implementation support component will follow a Plan-Do-Study-Act framework (41). The senior trainer will 
prioritize a strong relationship with all clinicians, addressing problems constructively and matching training to clinician 
learning styles (28,42).   
 
2. Stakeholder Enhancements (Partnership) Intervention. The Partnership intervention will involve all of the above 
components of the Wellness intervention, and the two additional components identified by patients and stakeholders 
from the recent PCORI award and from relevant research literature. The Wellness intervention will focus on these two 
evidence-based enhancements working together in a school-wide strategy focused on clinicians working actively on 
school teams and assisting in promotion/prevention -Tier 1, early intervention - Tier 2, and treatment -Tier 3 within the 
multi-tiered system of support (MTSS).  Clinicians will actively participate in the MTSS to help improve school culture, 
and enable effective implementation of all intervention elements including SEs.  In each of the 11 Partnership schools, a 
parent leader, and school-employed mental health professional (e.g., school psychologist, counselor) will be assigned to 
work with the clinician and implementation team, one day per week through resources allocated from the grant. The 
two SEs will be: Mental Health Literacy and Stigma Reduction. To enhance mental health literacy and focus purposefully 
on reducing the stigma of seeking SBH services, a program called The Guide, developed in Canada and beginning to be 
used in the US and other nations, will be used (see www.teenmentalhealth.org/curriculum). The Guide is a teacher-
delivered mental health curriculum for middle and high schools. Teachers, as supported by clinicians and parent leaders 
will deliver the program, which includes didactics, interactive classroom activities, and experiential learning. The Guide 
consists of modules focused on stigma and reducing it, understanding mental health, mental health challenges, and 
seeking help/finding support, with positive outcomes of increased mental health knowledge and reduced stigma for 
participating students (43,44). The program is being used throughout Canada, in Washington State, and in a number of 
other countries, and it has strong feasibility and acceptability data, helping to support its growth and sustainability in 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
7 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
Canada and other nations (43, 44). For each of the 11 Partnership schools (5 in MD, 6 in SC) in the summer of the three 
intervention vs. comparison years, developers of The Guide Drs. Stan Kutcher or Yifeng Wei will conduct a one-day 
training involving the SBH clinicians and senior trainers (1 from SC and 1 from MD), as well as with three teacher leaders 
recruited from each of the schools, with one each from 6th, 7th, and 8th grade. For individuals who cannot attend the 
summer training, the research team will host a compensatory training utilizing video segments recorded during the 
summer training. Following one-day training, coaching calls and webinars will be conducted with this group by Kutcher 
or Wei (alternating involvement with each site).  The SBH clinicians and these teachers will be active on the schools’ 
MTSS teams, and will in turn provide training on The Guide to these teams, and clinicians will use materials in sessions 
with students and families. In each school, this group will also use grade-level meetings, MTSS meetings, and other 
avenues to disseminate information on The Guide to other teachers, and will encourage teachers to sign up and go 
through a free on-line course on it.  The online self-study course takes between 4 to 6 hours to complete, and is 
associated with continuing education units and teacher education certificates from the University of British Columbia 
(http://educ.ubc.ca/), which we believe will be an important incentive (e.g., facilitating promotion opportunities).  
Teachers and clinicians involved in implementing The Guide program will complete The Guide Curriculum Assessment, a 
measure with adequate psychometric properties (43,44) at the beginning and end of each school year.  Students 
receiving Tier 3 intervention will also complete this measure (student version) at intake and every three months, 
integrated into the Tier 3 measurement strategy (see below). As result of the COVID-19 crisis in late spring of 2020, 
schools began implementing distance learning for all students. Because of this shift, teachers were temporarily unable to 
facilitate The Guide curriculum in classrooms as planned. One Maryland school teacher and clinician have continued to 
facilitate a live, interactive session using the Blackboard platform, in addition to one South Carolina teacher who is using 
the Google platform. These two schools are able to maintain essential components of the material as articulated by 
consultants Yifeng Wei and Andrew Baxter, and seek additional guidance from the study team and consultants as 
needed. For schools that are unable to facilitate virtual delivery of The Guide curriculum, clinicians will facilitate regular 
meetings with teachers at their respective schools. These meetings are intended to achieve two goals: 1) review The 
Guide content to increase knowledge and confidence and 2) create an implementation plan for delivering The Guide in 
classrooms when schools resume in-person instruction. The study team will track all of these activities via monthly 
reports submitted by clinicians and teachers, including whether sessions were held virtually or in person. With IRB 
approval from all sites, class rosters including school-issued student identification numbers will be collected by the study 
team. This strategy will serve as an additional enrollment strategy, as the team will be able to include variables related 
to classroom participation with The Guide in later analyses of student data (e.g., grades, attendance, discipline referrals) 
to assess any changes that may be associated with the curriculum. All decisions related to The Guide curriculum are 
made through collaboration with Yifeng Wei and associates at teenmentalhealth.org. In addition, Wei will serve as a 
content expert on ECHO calls with teachers and clinicians during years 2 and 3 of the study. 
Enhancing Family-School-Mental Health Partnerships.  The Partners for Change Outcome Management System (PCOMS) 
has been used for more than two decades to enhance patient involvement in mental health services and to improve the 
therapeutic alliance between patients and clinicians (see 45-47). Foundational to the program are two brief measures: 
The Outcome Rating Scale (ORS, 46) and the Session Rating Scale 3.0 (SRS, 47). The ORS measures progress in EB 
symptoms and social relationships (completed at the beginning of the session) and the SRS measures therapeutic 
alliance including quality of the relational bond, and agreement on treatment goals and methods (completed at the end 
of the session). Both scales take less than a minute to complete and score and are available for free online at 
www.talkingcure.com.  The PCOMS program helps to promote dialogue on how therapy is proceeding, increases 
chances for active student/parent input, and helps to assure that both groups are in sync with treatment plans and 
progress.  In addition to the PCOMS system, SBH efforts in Partnership schools will be enhanced by active participation 
of the parents on MTSS teams, which will also receive regular guidance, ideas and resources from the study’s SAB (see 
Engagement Plan). Please note that family-school-mental health partnerships will also be enhanced by ongoing 
involvement and guidance by the family advocate at each site, and parent leader at each of the 11 Partnership schools. 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
8 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
In addition, in each of the 11 Partnership schools, clinicians and MTSS teams will conduct trainings on both Partnership 
Group enhancements, as appropriate. Clinicians and MTSS teams should leverage grade-level meetings to provide 
trainings and information as appropriate for each school environment. 
 
To help to assure the cultural responsiveness of the Partnership intervention, family advocates and parent leaders 
will provide training and ongoing support to SBH clinicians and MTSS teams to: 1) Consider the family’s cultural 
preferences when prioritizing family needs,  2) address culturally specific sources of mental health stigma, and 3) to 
identify obstacles to involvement in services that are unique or particularly relevant to racial/ethnic minority families 
and to develop strategies to overcome these obstacles. Further, Dr. Colleen Halliday-Boykins of the Medical University of 
South Carolina, who possesses expertise in racial/ethnic discrimination and bias in youth as well as racial/ethnic 
disparities in problem behavior, will provide training to clinicians and schools. She will provide on-site unconscious 
racial/ethnic bias trainings to members of school teams in year 1, with additional support via phone and video 
conferencing during additional years. Similar to The Guide training, a compensatory training will be held for individuals 
who are unable to attend the summer training. 
Assuring and Monitoring Intervention Fidelity. In addition to the implementation manual including general and 
specific guidance for all intervention elements, we will monitor intervention implementation to assess fidelity along 
three domains: Adherence, Exposure/Dosage, and Participant Engagement. Based on procedures established for the 
prior RCTs funded by NIH (18,19), fidelity of the Wellness intervention will be assessed by a monthly self-report 
completed by SBH clinicians with self-rated performance on all four dimensions. For clinicians in the Partnership 
condition, with guidance by SAB, items will be added to reflect critical components of both Partnership enhancements. 
Clinical productivity of clinicians in service types of assessment and intervention will also be monitored (see below). For 
the Tier 3 measurement plan (see below), participant engagement will be assessed every three months with the Parent 
Participation Engagement Measure (48), completed by students and parents. The research team will collect and analyze 
fidelity process forms promptly with information going to the SAB for analysis and recommendations back to the 
research team. Intervention feasibility and acceptability will be measured using the Usage Rating Profile – Intervention 
Revised (49), a measure of those factors believed to influence treatment usage at the individual (i.e. Acceptability, 
Understanding), intervention (i.e., Feasibility), and environmental (i.e., Family-School Collaboration, System Climate, 
System Support) levels. Clinicians will complete this measure at the end of each intervention year. Dr. Will Aldridge, 
national leader in implementation science, from the Impact Center at the Frank Porter Graham Child Development 
Institute (in NC) will provide overall guidance on implementation issues encountered in the study. In particular, he will 
provide guidance on implementing the 6-component Partnership intervention in the real world environment of schools, 
with a view toward enhancing the scalability of the intervention, should analyses document expected positive outcomes.  
 
Study Design.  
Measures and Measurement Plan.  All measures chosen for use in the study are in the public domain and have 
strong psychometric properties (e.g., internal consistency, test-retest reliability, concurrent validity). There will be two 
streams of data collection involving students.  The first will be data collected to reflect school-wide impacts of the 
Partnership Condition focusing on academic record variables of students’ grades, attendance, lateness and discipline 
referrals (including office discipline referrals and suspensions), and school climate. School-wide academic record data 
will be collected for all students and de-identified to the research team (it is considerably less burdensome for schools to 
share deidentified data for all students than to share for a sample of them).  School climate data will be collected for a 
sample of about 60 students (3 randomly selected classrooms), and 25-40 staff (requesting all staff complete) at each of 
the 22 schools. School-wide academic record data will be collected once annually for the academic year prior to study 
start (2018-19), and will be collected each year for the three years of intervention vs. comparison, per procedures that 
we are replicating based on the current NIJ-funded RCT.  Selected students and staff will complete the School Climate 
Survey (SCS) a free, online school climate survey from the US Department of Education 
(https://safesupportivelearning.ed.gov/edscls). The SCS is available as a web-based platform and includes a suite of 
school climate surveys for students, instructional staff, and non-instructional staff. Staff and students will be asked to 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
9 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
complete the survey, which assesses the dimensions of engagement, safety, and environment once annually at the end 
of each academic year for the four years of intervention vs. comparison (2019-20, 2020-21, 2021-22, 2022-23). Due to 
the COVID-19 pandemic, the team was unable to collect SCS data from students in 
 schools during the first 
academic year (2019-2020). While this means a compromised set of Year 1 data on school climate for the 
site, 
such measurement is intact for SC, and we will still be able to document trends in time in school climate in
 for 
years 2-4 of the intervention vs. comparison period.  Additionally, the study team will collect rosters of classrooms 
receiving The Guide Curriculum (GC). This will enable two sets of analyses: first, we will document which students 
receiving Tier 3 services and participating fully in the study are exposed to GC and account for this in terms of 
interventions received in analyses. Second, this will significantly expand a second level of enrollment for the study, as we 
will have de-identified academic records for all students (related to recently approved amendments), and will explore 
impacts of the GC on academic record variables (e.g., discipline problems, grades, attendance) for students in 
classrooms in Partnership schools receiving it compared to students in demographically comparable classrooms from 
Wellness schools that do not receive it.   
The second stream of data collection will focus on students receiving Tier 3 services and consenting to be in the 
study.  Clinicians will collect these measures with students and families at intake, every three months, and at case 
closure, using an electronic data portal coordinated by Dr. Jihad Obeid and the Informatics Team at the Medical 
University of South Carolina (MUSC). The measurement strategy for each of the study aims is presented here:   
Aim 1: Evaluate the impact of the Partnership enhancements in increasing the number of students and families receiving 
services and expressing satisfaction with them.  For this primary aim, and per experiences from prior RCTs (18,19), as 
mentioned, we will collect data for each student on the number of sessions conducted by the SBH clinician in categories 
of assessment; individual, group and family therapy; case management; and teacher consultation. In addition, during the 
COVID-19 school closure period we will document which students receive any of these services through telehealth, and 
will document this form of service received. This will enable coding of overall productivity, for types of services, and for 
family involvement in care. To measure satisfaction with services, at the three month or case closure assessment points, 
we will collect the Client Satisfaction Questionairre-8 (CSQ-8) from students and parents. The CSQ-8 is a brief, widely 
used 8-item measure for use with youth aged 11 and older and adults (50).  
Aim 2: Evaluate the impact of the Partnership enhancements on students’ social, emotional/behavioral (EB), and 
academic outcomes. For this primary aim, there will be two sub-aims: Aim 2a will evaluate the impact of the Partnership 
enhancements on school-wide academic record variables of grades, attendance, lateness and discipline referrals. Aim 2b 
will focus on students receiving Tier 3 services and will assess their social and emotional/behavioral (EB) functioning. To 
assess social functioning, the Child and Adolescent Social and Adaptive Functioning Scale (51) will be completed by 
students. The CASAFS includes 24 items measuring students’ functioning in four key social roles: school performance, 
peer relations, family relations, and home duties/self-care. Additionally, both “internalizing” EB problems, which include 
problems of depression, anxiety and exposure to trauma, and “externalizing” EB problems, which include conduct and 
attention problems, will be measured for students receiving Tier 3 services. Student EB functioning will be measured by 
parent and student report forms of the Brief Problem Checklist (52), a 12-item questionnaire that measures externalizing 
and internalizing problems. Derived from the comprehensive Child Behavior Checklist (53), it is designed for repeated 
periodic assessment of clinical progress, yielding both an internalizing and externalizing subscale (52). Please note that 
academic variables collected above school-wide for stream 1 data collection, will also be used for stream 2 data 
collection for students receiving Tier 3 services. Changes in the above variables will be assessed for students for the 
academic quarter before receiving mental health services and the academic quarter after services have been completed. 
     Additional Measures. In addition to measures directly reflecting study aims, secondary aims will be addressed by 
measures reflecting the potential impact of the two Partnership enhancements will be collected at the same time 
intervals for students receiving Tier 3 services. For the enhancement focused on mental health education and stigma 
reduction, students will complete The Guide Curriculum Assessment (GCA) reviewed previously (43,44). The GCA 
includes subscales assessing Mental Health Literacy in three dimensions: Mental Health Knowledge (MHK; 28 items), 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
10 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
Help Seeking (HS; 15 items), and Perceived Stigma (PS; 12 items).  The MHK and PS subscales of the GCA have strong 
psychometric properties and will be used in the study.  The MHK subscale includes 7 items on basic understanding of the 
brain, 9 items on common mental disorders, 7 items on stress and good mental health, and 5 items on causes of mental 
disorders and treatment. The PS subscale includes 5 items reflecting personal stigma (personal attitudes toward people 
contending with mental health problems) and 7 items reflecting social distance (the degree to which people are willing 
to accept people with mental health problems into regular social life). To further assess the reduction in Perceived 
Stigma, we will also collect the PS subscale from the GCA from parents of students receiving Tier 3 services within 
Stream 2 data collection using the teacher/parent version of the measure. 
The enhancement focused on Family-School-Community Partnerships will be measured in three ways. First, the 24 
SBH clinicians will complete the 63-item SBH Quality Assessment Tool (SBHQAT; 29) in August and May in each the four 
intervention versus comparison years (2019-20, 2020-21, 2021-22, 2022-23). Second, clinicians and students receiving 
Tier 3 services will complete the Therapeutic Alliance Scale for Children-revised (TASC; 55), a measure of therapeutic 
alliance across treatment. Third, as mentioned, family participation and engagement in their children’s mental health 
treatment will be measured using the Parent Participation Engagement Measure (PPEM; 48), which assesses parent 
participation behaviors with the therapist as reported by both youth and their parents/guardians. 
     Data Management. The MUSC Informatics Team will work on a robust and secure data management infrastructure 
under the guidance of Dr. Obeid. The team will leverage an existing web-based tool, REDCap, to support data capture 
both in the field and across the collaborating institutions. REDCap (Research Electronic Data Capture) is a software 
toolset and workflow methodology for electronic collection and management of research projects that provides a 
secure, web-based flexible application, and includes real time validation rules with automated data type and range 
checks at the time of data entry. Exports are made available for several statistical packages including SPSS, SAS, STATA, R 
and Microsoft Excel. The underlying database will be hosted at the MUSC data center, a secure environment for data 
systems and servers on campus, and includes redundancy, failover capability, backups and extensive security checks. 
The system has several layers of protection including, user/group account management, data access groups, which allow 
data to be entered by multiple groups in one database with segmented user rights for entered data, audit trails for all 
changes, queries and reports, and Secure Sockets Layer (SSL) encryption. Case report forms (CRF), questionnaires and 
surveys for this project will be implemented in REDCap. The REDCap support team will work closely with the 
investigators to ensure the electronic CRF’s operate correctly, validate error checks, create reports and oversee data 
export. The support team will also test the implementation over various client platforms including mobile devices such 
as iPads and/or Android tablets, and will manage accounts and access at other institutions and remote sites. 
 
Analytic Plan. The proposed project is a cluster-randomized controlled trial (J=22 schools; 11 per treatment arm) 
comparing the efficacy of Wellness vs. Partnership services. As a general analytic plan, we will use hierarchical linear 
modeling (HLM) to evaluate the project aims. Specifically, we will use two-level HLM models to account for students 
being nested within schools. Though one might conceptualize this proposal as a three-level model (i.e., students nested 
within schools, and schools nested within sites), it would be neither feasible nor recommended to model site as a 
multilevel variable given there are only two sites in the design (56). In line with best practice, we will integrate site (i.e., 
SC vs. MD) as a covariate in the statistical models to contain any potential effects of this variable. Based on prior 
research conducted by the team, we expect intraclass correlations (ICCs) to warrant the incorporation of random 
intercepts to account for violations of independence. We will also test independence assumptions for the treatment 
parameters to determine how sensitive treatment effects are to potential violations of independence, and will include 
random slopes to account for nested treatment effects if ICCs exceed .01. For transparency, we will report both fixed 
and random effects (i.e. intercepts and slopes) of the model. Additionally, we will iteratively subset the model using a k-
1 sensitivity analysis to determine the influence of individual sites on parameter estimates. We will estimate statistical 
models using full information maximum likelihood to accommodate any missing data (57). We will use maximum 
likelihood with robust standard errors to estimate models that include the Zero Inflated Poisson link function for count 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
11 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
outcomes (e.g. absences, referrals). This general analytic framework will be used in aims 1 and 2. Details, exceptions, 
and modifications to it are discussed below. 
In addition to primary intent to treat analyses, we plan to use quasi-experimental methods to test dose-response, 
and treatment engagement on a core aspect of the Partnership Condition – the mental health literacy intervention 
through the program, The Guide. In Partnership schools, this intervention has continued to be implemented through all 
phases of the pandemic, going from virtual in some phases to in-person in classrooms. We are able to access student 
academic records (demographics, grades, attendance, discipline) for students in Partnership classrooms receiving The 
Guide, as well as students in comparable wellness classrooms not receiving it. Per below, we have a large sample size to 
pursue these supplementary analyses. In these analyses, we will be overpowered (MDES below .1) to understand the 
impact of this mental health literacy intervention on student functioning, including examining dose of intervention given 
variability in teachers’ delivery of it. We also have broader school climate data to explore the impact of the Partnership 
intervention on this variable (collected fall, 2019; fall, 2020; fall, 2021; spring, 2022; and spring, 2023). 
 
 
 
Model modifications for students receiving tier 3 services. To evaluate the effects of the enhanced treatment on 
students who receive tier 3 clinical services, and to accommodate the fact that there will be a rolling admission into 
clinical services, the treatment parameter will be interacted with a dosage parameter measuring each individual 
student’s weeks of treatment received. This interaction term will provide a treatment effect estimate that is calibrated 
by dosage, as measured by weeks of treatment received. Because we will use an active treatment, we will be able to 
detect treatment-by-dosage differences between the enhanced and control treatments. To accommodate the possibility 
that random assignment does not create exactly equal groups with K=22, and to provide interpretable estimates of 
change in response to the intervention, we will use difference scores as the dependent variable unit of analysis for 
students receiving tier 3 clinical services. This will facilitate the direct examination of how services received over time 
influences youth change. The null hypothesis associated with change scores is that baseline differences remain constant 
and there would thus be no mean change across measurement occasions. Rogosa and colleagues (58-60) highlight the 
utility of using difference scores to evaluate pretest-posttest change in similar studies, indicating that previous work 
critical of this approach has been misleading in that it focused on several restrictive assumptions including high 
correlations of pretest and posttest measures as well as equal variances across measures over time. Given our previous 
research, outcomes, and statistical model, we believe that difference scores are the appropriate dependent variable.   
Given the experiment will examine intent-to-treat effects in a real-world setting, we will conduct analyses to 
investigate variability in implementation and compliance to enhance causal effect interpretation of clients “as treated.” 
Specifically, we will use a Bayesian Additive Regression Tree approach (BART; 61) to delineate the impact of key 
treatment assumptions (e.g., compliance, fidelity, etc.), as well as to estimate the impact of assumption violations on the 
estimation of causal effects in the design. BART is an extension of Classification and Regression Tree (CART) models that 
is based on a recursive partitioning algorithm that explains variation in the outcome through repeated splits of the data 
into homogenous subgroups that are similar on predictor variables. One significant strength of BART is that is capable of 
handling a large number of predictors and incorporates higher-order interactions that may be excluded from parametric 
analysis. Research indicates that BART is insensitive to tuning parameters and produces more accurate estimates of 
treatment effects compared to other methods of analyzing treatment effect heterogeneity (i.e., propensity score 
matching, propensity-weighted estimators, instrumental variable procedures, and regression adjustment; 61). Note that 
this BART enhancement to the randomized, experimental design should not be mistaken for a Bayesian trial design. 
 
Power considerations. The design originally included a planned sample size of approximately N=2800 (1400 per 
arm) for Aims 1 and 2. We previously revised power estimates for these study aims that consider the possibility of 
reduced participant involvement given COVID-19 and recruitment challenges more broadly, with a revised sample size of 
approximately N=1400 (700 per arm). With the research study period concluding in June 2023, we have revised the 
enrollment projection to a conservative estimate of N=1225. With our currently active sites (J=22) and the total number 
of students enrolled in the study (N=1225, including expected spring 2023 enrollment), assuming 30% of variance 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
12 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
explained in the primary outcomes by pre-test covariates, and 80% power, we have a minimum detectable effect size 
(MDES) of .29. If we reach our goal of 1400 students, we will be at .28.  
A priori power analyses indicate that we will be adequately powered to detect any primary outcome that has an 
effect size of d=.29 or greater (i.e., between small [d=.2] and medium [d=.5] in size per Cohen, 1988). Statistical software 
does not differentiate between one variable (e.g., depression) or another (e.g., anxiety) when executing power analysis 
for an outcome. Notably, Ferguson (2009) recommends that "minimum effect size representing a 'practically' significant 
effect for social science data" with difference measures like Cohen's d is .41. 
Variation in cluster size is a typical (and expected) occurrence in cluster-randomized trials, with a priori power 
analyses invoking the projected average sample size per cluster for projected MDES estimates to accommodate 
differences in cluster size. Moreover, methodological literature has shown that any negative influence of cluster 
imbalance is limited to situations where there is a severe Pareto imbalance, defined as “a situation in which 80% of the 
subjects actually belong to only 20% of the clusters” (Guittet, Ravaud, & Giraudeau, 2006). This is not the situation here, 
and thus we expect the observed imbalance to have a negligible impact on analyses. Power analysis for the study 
corresponds to the ITT effect of the intervention, wherein the probability of the intervention’s ability to reject a null-
hypothesis of no intervention effect is considered. Power estimates are not provided for the dose-response effect (i.e., 
the effect of treatment at varying dose levels) as this will be calculated using a Bayesian method (i.e., Bayesian Additive 
Regression Trees, BART).  
Estimating dose-response effects using a conventional, frequentist treatment*dosage interaction would violate the 
strong ignorability assumption, confound effects with varying characteristics of participants and sites, and yield biased 
estimates (e.g., 69,70). The BART models proposed in the current application have demonstrated efficacy in estimating 
treatment effects in quasi-experimental studies that may vary as a function of a continuous variable, such as treatment 
dose (71). The BART approach will provide an estimate of the treatment effect for those who received the intervention 
conditional on baseline covariates (i.e., conditional treatment effect on treated).  We will calculate conditional 
treatment effects on treatment for each individual following the Neyman-Rubin causal model (72,73). The BART models 
will yield an effect size for each varying level of exposure, as well as provide a credible interval (analogous to a 
frequentist confidence interval) for the effect estimate. The BART models will also allow missing data to be included as a 
feature in the predictive model using a “missingness incorporated in attributes” approach. This means that BART will use 
all available data. Kaplener and Bleich (74) demonstrated that this approach is computationally efficient and that results 
are comparable to other advanced missing data approaches (e.g., multiple imputation). 
We will conduct additional analyses to ensure the results from our original models are not due to random noise in the 
sample. Specifically, we will conduct 250 Monte Carlo simulations in which a random binomial variable will be generated 
to represent an instrumental variable such that it will be uncorrelated with covariates or outcomes. Then we will regress 
each of the outcomes on the binomial variable yielding a parameter representing the relationship between the 
instrumental and binomial variable that is unrelated to the outcomes. This analysis will test the sensitivity of our BART 
models on estimated treatment effects and determine how robust our credible intervals are.  
Please note, we will analyze data related to services received and outcomes for students of varying race/ethnicity as 
a supplemental, exploratory aim. Because of increased attention to cultural issues, it is possible that the Partnership 
intervention may have additional benefits in reducing disparities in services and outcomes for youth of color. Should 
findings indeed point to benefits of the Partnership intervention, these will be integrated into our dissemination plan 
and help to fuel critically important future research.    
 
Study Population and Setting. Intervention will begin during the 2019-20 school year, and it will continue through the 
2022-2023 school year. The study sample will include students from 22 middle schools, with 12 schools from SC and 10 
from MD. One mental health center in each state will work with the study, with these mental health centers already 
having established relationships with the middle schools, with clinicians placed in these schools .7 FTE or 3.5 days per 
week.  In MD, the site will be 
 (urban), with one school district and all 10 middle schools coming from this 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
13 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
site.  In SC, the site will be 
 County (suburban and rural). Letters of support from mental health center leaders 
and the two school districts are included, reflecting the strong partnerships already established between the mental 
health centers and these districts.   
 
Utilizing the demographic information available from each site's state Department of Education database, 
participating schools are predominately male (52%) and  White (45%), followed by Black/African American (37%), Asian 
and Multiracial (3%), and less than one percent American Indian and Pacific Islander, with 12% identifying as Hispanic. 
Current prevalence estimates (2,14) predict that 20% of youth will meet criteria for a more serious EB problem. Due to 
random school assignment to condition, we predict these percentages will be evenly split across Wellness and 
Partnership.   
 
As presented above, critical demographic factors will be assessed to investigate whether members of 
underrepresented groups present unique needs for services. Participant subgroups that will be further evaluated will 
include those varying based on mental health diagnosis, gender, race/ethnicity, and sociodemographic characteristics 
including income status, urban-rural-suburban status, and neighborhood levels of crime and violence. Initial analyses will 
be conducted to determine whether there are significant differences in primary outcomes of interest for subgroups 
compared to the overall sample. In instances where significant differences are found for a subgroup, interaction effects 
will be examined within a two-level HLM framework (see above) to further explore subgroup differences.  
Recruitment Plan. School enrollment of each participating school was retrieved at the SC and MD state educational 
databases to determine the average middle school population is 600 across participating schools. In Year 1, around 
12,000 students will attend the 20 study schools with an estimated additional 8,000 students entering study schools as 
6th graders in Years 2 and 3, totaling 20,000 potential participants during the three-year implementation phase.  
There will be two avenues of recruitment (with these strategies endorsed by the USC IRB). First, the Partnership 
enhancement is expected to lead to school-wide impacts and the study involves a systemic intervention involving no 
added risk to students and families; thus, school-wide collection of academic record variable is consistent with the 
research on education systems exemption from parental consent and student assess to collect these data. In terms of 
school-climate, staff completion of the climate measure has been determined to be exempt research with no 
requirement for consent since staff are anonymous and providing no personal information. For the randomly selected 
classrooms of students to complete the climate assessment, an opt-out procedure will be used.  Students/families in the 
randomly selected classrooms will receive information on the study through multiple channels (mailed letters, letters 
sent home with students, emails) and will have multiple ways to opt-out (e.g., sending forms back, emailing the study 
team). All student record data will be de-identified prior to forwarding to the research team. These procedures are 
consistent with those currently being implemented in a large RCT led by the PI, which has had less than a 10% opt-out 
rate and no complaints from families thus far.  
The second avenue of recruitment will be for students receiving Tier 3 treatment services.  These students/parents 
will be consented using electronic consent (e-consent) in REDCap, with the consent form appended at the beginning of 
the survey. Participants and their parents will be required to go through the informed consent process and digitally sign 
the consent form using a mobile tablet device before proceeding into the survey questionnaire. MUSC has successfully 
piloted e-consent in REDCap with other IRB approved research studies. Given challenges with parents’ ability to attend 
meetings at the school to complete the enrollment process, the study team developed protocol to allow for consent to 
be gained via phone and intake to be completed remotely via phone or computer. This flexibility allows enrollment to 
continue, and not be hampered by potential transportation or financial constraints that may otherwise prevent low-
income families from participating. Additionally, as result of the COVID-19 crisis, procedures were developed to allow 
students to assent into the study via telephone or video conference. Students and parents may provide assent/consent 
via phone, after viewing consent materials and having the opportunity to ask questions of the clinician. Intake 
assessments can be completed verbally (via phone or video conference) or a link to complete surveys online may be sent 
via email or directly through the REDCap program by the clinician or research manager.  

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
14 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
Based on current prevalence data, twenty percent (2,14) of students are expected to present with mental health 
challenges (n= 4,000). Of these students, based on the experience in similar studies the research team estimates that 
30% will consent to participate representing the recruitment goal of 1,225 students by the end of the 2022-23 academic 
year. The rate of recruitment will be higher initially due to enrollment of students currently receiving services, however, 
over the course of the study an average of 35 students per month (during the academic year only, or four years times 9 
months for 36 months total) will be enrolled. Note: significant changes to enrollment targets have been made during 
this project largely related to the unanticipated and unprecedented impact of the COVID-19 pandemic which 
temporarily closed school buildings and changed mental health care services starting in March 2020. 
There are essentially six classes of student participants, with groups based on grade level at the start of the 
intervention vs. comparison (IvC) period of the study, which begins in the 2019-20 school year. There is a four-year IvC 
period, 2019-20, 2020-21, 2021-22, and 2022-23.  Group A will be in the 6th grade in 2019-20, with students 
experiencing 1-3 years of services in this time interval. These students could have received services in 6th, 7th and/or 
8th grades during the IvC period. Group B will be in the 7th grade in 2019-20, with students experiencing 1-2 years of 
services in this time interval. These students could have received services in 7th and/or 8th grades during the IvC period. 
Group C will be in the 8th grade in 2019-20, with students experiencing 1 year of services in this time interval.  Group D 
will enter the study in 6th grade in 2020-21. Group E will enter the study in 6th grade in 2021-22. Group F will enter the 
study in 2022-2023 while in 6th grade. 
Identifying eligible participants. Participants already receiving treatment and those newly referred for services will 
be eligible to participate and recruited by their SBH clinician. Throughout the implementation phase (years 1-4) each 
school will use their existing referral process to identify students in need of treatment which will be followed up and 
confirmed by the SBH clinician prior to enrolling them in the study.  
Overcoming barriers to enrollment. In addition to our successful experiences with the opt-out procedure to be used 
for stream 1 data collection, both mental health centers have participated in extensive and similar research with the 
investigators leading this study. The SAB will provide input on the development of consent materials, and methods of 
communication, making sure that they are appealing.  The process of utilizing e-consent will be convenient and 
streamlined for Tier 3 participants. Additionally, participant incentives are proposed, in the amount of $20 per parent 
assessment battery, and a token of appreciation for each student assessment battery to incentivize participation.  
Study retention. Due to documented accessibility advantages of SBH (16) and our past experiences, Tier 3 students 
and parents are likely to remain involved in the study throughout the span of treatment. Additionally, the intervention 
requires no extra effort to participate outside of the assessments, with similar types of assessments routinely used in 
treatment for progress monitoring. In similar studies conducted, the most common reason for study attrition was due to 
students transferring to another school, which has been nominal across school wide populations. However, Tier 3 
students experiencing EB problems are more likely to transfer schools than the general population. 
 Incorporating research activities into routine care. All research activities will be included in routine SBH functions.  
Consent to participate will be integrated into the existing process that clinicians use to obtain consent for treatment. 
Students and their parents may notice that their clinician is utilizing some new strategies, however all within parameters 
of expected clinical care. The same is true in the Partnership condition, will all enhancements those that would logically 
occur in a school embracing innovation and pursuing holistic strategies to remove barriers to student learning.  
 Training and experience of recruitment personnel. For students receiving Tier 3 services, the 20 clinicians will assist 
in study recruitment. Beginning in Year 2, a clinician was added in two 
 schools, one per condition for a total of 
22 clinicians. Beginning in Year 3, two additional schools were added, one per condition, for a total of 22 schools and 24 
clinicians. All clinicians will have a master's degree in a behavioral health discipline (e.g., psychology, counseling, social 
work) and will hold a valid license to provide psychological services or be under the supervision of a licensed clinician. 
Each clinician will participate in human subjects training prior to recruitment efforts and will be trained by the research 
team on recruitment procedures. The Lead Research Manager, Brooke Chehoski, will oversee recruitment efforts with 
the PI, Weist and Co-I/MD Site PI, Hoover, as guided by the SAB and building on experiences from prior RCTs conducted 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
15 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
by the research team, including a large current RCT funded by the National Institute of Justice that includes an opt-out 
procedure. See below for recruitment parameters.  
 
Estimated number of potentially eligible study participants (describe how this number was 
determined [e.g., Electronic Health Record, claims data, clinic logs, administrative data, 
other]): 
20,000 
Total number of study participants expected to be screened: 
20,000 
Total number of study participants expected to be eligible of those screened: 
4,000 
Target sample size (use same number stated in milestones): 
1225 
If applicable, total number of practices or centers that will enroll participants: 
22 
Projected month first participant enrolled (month after project initiation): 
9/2019 
Projected month last participant enrolled (month after project initiation): 
6/30/2023 
Projected rate of enrollment (anticipated number enrolled per month of enrollment 
period): 
42* 
*Rate of enrollment is based on a 9-month time period reflecting the school year in years 1-4 for a total of 36 months. 
Recruiting 39 students per month over 36 months would give a total enrollment of 1,404 students. 
(See appendix for site specific tables) 
Sample Size and Power. Please see Power considerations for the study in the Analytic Plan above (p. 9-10), given we 
discuss these estimates alongside the Bayesian analytic strategy used for part of the analysis in the study. 
Outcomes. We hypothesize that students/families in the Partnership condition will report increased positive outcomes 
compared to those receiving the Wellness intervention alone in the following ways: Improved connection to SBH 
services, longer service duration, and greater satisfaction with service; improved social, EB, and academic outcomes at 
post-testing as and sustained improvements in these dimensions in 9th grade. These outcomes are widely cited in SBH 
literature, and were validated by patients and stakeholders on the SAB and through research forums held throughout SC 
as part of the prior PCORI engagement award.  
 
Site Selection. Proposed study sites were selected based on factors related to feasibility as well as their potential for 
providing a representative patient sample. Both the 
 Mental Health Center (
MHC) and the CSMH, 
have long histories of collaboration with investigators of this study. This existing relationship provides many benefits in 
avoiding common barriers to implementation. The two sites offer complementary demographics with CSMH serving 
youth who are primarily urban and African American, while 
MHC serves a suburban and rural youth population that is 
more diverse racially and ethnically, and both sites serve low and middle income families, allowing us to evaluate for a 
variety of outcomes in subgroup populations. Additionally, incorporating the two interventions into routine SBH 
practices will facilitate findings that are immediately applicable to real world contexts.  
 
 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
16 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
 
Research Team and Environment  
 
Research Team and Environment. This study represents a strong collaboration among faculty and staff from the 
University of South Carolina (USC), University of Maryland Baltimore (UMB), the Medical University of South Carolina 
(MUSC), and collaborating mental health centers and school districts at the two sites. All partners are strongly invested 
in the well-being of students in SC and MD and have the capacity to perform the work proposed in this application. 
Collaborating key personnel are: 
    Mark D. Weist (Principal Investigator, PI). The proposed study builds from a longstanding program of research by the 
PI (Weist) focusing on understanding and measuring quality in school behavioral health and developing and testing 
achievable evidence-based practices in schools, including two RCTs (R01s) funded by the National Institute of Mental 
Health (2003 – 06; 2010-15). In addition, the PI founded and directed the UMB Center for School Mental Health (CSMH) 
focused on advancing effective SBH in communities and states prior to moving to USC. The PI was an investigator on the 
Center for Adolescent Research in Schools (PI, Lee Kern) study funded by the Institute of Education Sciences (IES), an RCT 
that tested evidence-based classroom- and student-focused interventions.  He is currently leading a large scale RCT in 24 
schools funded by the National Institute of Justice (#2015-CK-BX-0018, 2016-2019), which aims to compare the 
effectiveness of 3 intervention conditions. This study has far exceeded its recruitment goals and has collected school 
record data, fidelity data, intervention receipt data, and student and teacher assessment batteries on approximately 
15,000 students thus far.  Dr. Weist is Professor of Clinical-Community, and School Psychology at USC and has directed 
both of these doctoral training programs.  He leads the USC School Behavioral Health Team, which employs 11 staff 
members as well as a cadre of graduate and undergraduate students. The lab is comprised of 10 offices, 15 newer 
computers, a Teleform data entry system, a secure server space, and a variety of statistical software packages.  
    Sharon Hoover (Co-I/MD site-PI). As Co-Director of the CSMH and Associate Professor at UMB, Hoover is a leading 
national scholar on high quality SBH. Dr. Hoover has served as the PI, Site PI, or Co-I on several federally- and state-
funded grants and contracts investigating the translation of children’s evidence-based mental health practice into “real-
world” settings, including: A National Institute of Justice (NIJ) Comprehensive School Safety grant; two National Institute 
of Mental Health R01s on Enhancing SMH Quality; a SAMHSA grant for a National Center for Safe and Supportive 
Learning Environments; two SAMHSA Systems of Care evaluations; two SAMHSA Healthy Transitions evaluations; and 
many more. Through significant local, state and national networks developed since the early 1990s, the CSMH has the 
potential to reach thousands of stakeholders from diverse disciplines with policy and research reports, enhancing the 
impact of the proposed study. The CSMH will utilize the resources made available to it from UMB and its extensive 
network to ensure that the needs of the proposed study are met.   
    Jihad Obeid (Co-I).  Dr. Obeid is the Co-director of the Biomedical Informatics Center at MUSC, which serves the SC 
Translational Research Institute, and Health Sciences South Carolina at a statewide level.  He is a pediatrician, formally 
trained in Informatics at the Division of Health Sciences and Technology, a joint Harvard-MIT fellowship program. He has 
extensive experience in research informatics, starting in 1998 as Director of the Pediatric research unit Informatics core 
at Weill-Cornell Medical College. At MUSC, Dr. Obeid oversees several academic and operational informatics initiatives 
and leads multiple Clinical and Translational Science Award-related Informatics projects such as the Research Data 
Warehouse, REDCap, Profiles research networking system, and many others. He is the division head for Biomedical 
Informatics in the Department of Public Health Sciences at MUSC. Dr. Obeid is a PI, informatics leader or Co-I on several 
NIH funded projects. Given his research background and leadership experience, Dr. Obeid will serve as Co-I and 
Informatics leader on this study. With the research team, he will oversee the development and implementation of a 
robust biomedical informatics infrastructure in support of the aims of the grant and will leverage existing web-based 
tools such as REDCap and data standards to support data capture and e-consent both in the field and on campuses 
across the collaborating institutions.  

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
17 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
    Amanda Fairchild (Co-I). Dr. Fairchild is an Associate Professor of Psychology at USC. As a quantitative psychologist, 
her research focuses on statistical design and analysis. Her methodological expertise focuses on statistical methods to 
optimize prevention and intervention research. The information technology facilities and other resources available to 
Dr. Fairchild and her research team will be made available to the PI throughout the duration of the project period. Dr. 
Fairchild will provide access to a variety of statistical software packages to support data analysis (Mplus, SAS, SPSS and 
R), as well as a multicore Dell applications server to support running software applications as needed.  
    Sam McQuillin (Co-I). Dr. McQuillin is an Assistant Professor at USC. His key contribution to the proposed project 
involves his work in the area of experimental design, quantitative methods, and implementation evaluation. His training 
is in school psychology with a quantitative psychology area of emphasis. As part of his research, he has designed, 
coordinated, and analyzed data from school-based randomized controlled trials, similar to the one in this project. He has 
served as a quantitative methodologist on a broad range of projects and has worked previously with school-based 
cluster randomized trials to be analyzed using hierarchical linear models and Bayesian additive regression designs, which 
are included in the current proposal.  
    Brittany Patterson (Co-I). Dr. Patterson is an Assistant Professor at UMB. She completed a pre-doctoral internship and 
postdoctoral fellowship at the CSMH. Dr. Patterson has provided training for administrators, educators, school police 
officers and student support staff on the impact of trauma on learning, secondary traumatic stress, and evidence-based 
mental health interventions for trauma-exposed youth. In addition, she helped develop and disseminate a toolkit of 
resources from the CSMH on civil unrest, and has worked with school and community partners to foster improved 
school-community-police relations, and to build trauma-informed multi-tiered systems of support in schools.    
    Robert Stevens (Consultant and Co-Chair of SAB). Dr. Stevens is a team member of the Carolinas Collaborative and 
Mid-South Clinical Data Research Network as well as a consultant on the recent PCORI engagement award to Dr. Weist 
and team.  Dr. Steven's background as a former Director of Prevention and Intervention in a local school district has 
given him expertise in this area of research as well as an extensive network of connections with leaders of youth serving 
agencies across the state, which has been invaluable thus far in our patient and stakeholder engagement efforts.  
    June Greenlaw (Stakeholder Engagement Coordinator). Ms. Greenlaw has a graduate degree in community education 
providing excellent experience for her role on this project as an engagement specialist. She is also the manager of the 
Research Consortium on Children and Families which is comprised of 105 faculty members at USC. She served as the 
engagement specialist on the PCORI engagement award. In addition, Ms. Greenlaw is a parent caregiver of a youth 
contending with significant behavioral health issues.  
    Jenah Cason (Family and Youth Advocate). Ms. Cason has a master's in Social Work and serves as the Director of the 
Federation of Families for Children's Mental Health of SC. Ms. Cason has collaborated with the research team since 2010 
when the SSBHC was in development. She is a Co-PI on a current Community Engaged Partners award with Weist's team 
which is evaluating the impact of an intervention to increase Youth Empowerment in SBH.  
    Beverly Griffin (Family and Youth Advocate). Ms. Griffin serves as a Parent Peer Support Provider and Family Advocate 
with the Federation of Families of South Carolina. Ms. Griffin will use her personal experience as a parent of a child with 
a mental health diagnosis and her professional experience to provide support to parent leaders who will be working 
within Partnership schools. 
Christopher Haines (Director of School Mental Health Programs). Mr. Haines serves as the Director of School Mental 
Health Programs at the 
 Mental Health Center. Mr. Haines will assist in overseeing the activities of 
project clinicians at the SC site. Chris Haines, MS, LPC is the Director of School Mental Health Programs at 
 
 Mental Health Center. Chris’s team includes 89 therapists stationed full time in South Carolina’s largest school 
district. Chris is the School Mental Health Therapist at
 School, where he has specialized in the treatment 
of adolescents with severe emotional and behavioral disorders for the past 11 years. Chris is on the South Carolina 
roster of clinicians trained in Trauma-Focused Cognitive Behavioral Therapy, a Master Trainer on Adverse Childhood 
Experiences, and a trauma-informed trainer through the Community Resilience Initiative. Chris speaks across the state 
and the region on trauma-informed school practices and the development and leadership of effective school mental 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
18 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
health programs. With a leading role in school mental health, Chris has been a partner to his school and his District in 
developing multi-tiered systems of support for the social/emotional and behavioral needs of students. 
Jennifer Cox (Program Director, University of Maryland, School Mental Health Program). Ms. Cox is the Program 
Director and a senior Lead Clinician for the University of Maryland, School Mental Health Program (SMHP), a program 
whose mission is to enhance the learning environment of 
 Public schools by removing the barriers to 
learning and actively promoting the social-emotional-behavioral well-being of students. Ms. Cox has over a decade of 
experience providing clinical services, resources, training and consultation across a multi-tiered system of supports. She 
provides administrative and clinical oversight to 25 SMHP schools and has worked to expand the SMHP tele-mental 
health program to increase psychiatry services and access in schools. Ms. Cox has a Certificate in Infant and Early 
Childhood Mental Health, is a board approved supervisor for social workers and is a state-wide trainer for Botvin’s 
LifeSkills.   Her work in schools, specifically with evidence-based practices and parental involvement in school, has been 
published and recognized by the field. She was the recipient of the National Association of Social Workers (NASW) Field 
Instructor of the Year Award and presents at local, state, and national levels. Most importantly, she is the wife to a 
fantastic husband and mother to two amazing boys. 
 
 
 
 
 
 
 
DISSEMINATION AND IMPLEMENTATION POTENTIAL  
 
A. Building on the platform of the ECHO program (25, 26) and guidance by the Stakeholder Advisory Board (SAB), 
dissemination of study findings will begin in Year 5, with comparison schools joining ongoing ECHO clinics held for 
intervention schools (in Years 1-4).  In Year 5, parent leaders and the study team will conduct presentations to each of 
the 22 schools, the sponsoring school districts, and the two mental health centers, reviewing study findings and making 
them accessible through the study website. Signs about the study and its findings will be posted throughout the centers 
so that patients and families receiving treatment within them can learn about the study.  In addition, study notifications 
and consent/assent forms for families will mention the study website and availability of findings as of Spring 2023, and 
our hope is that study participants access these resources to learn about study findings and lessons learned. In addition, 
we will make available as a public domain resource and broadly disseminate the study implementation manual, 
increasing the ability of other sites to implement the six-component Partnership intervention. Further, in Year 5, the 
ECHO platform will be used for sharing of findings and dialogue with professional organizations (e.g., National 
Association of School Psychologists, National Council for Behavioral Health), and state, local and federal policy leaders. 
Our connections to state and national groups with a vested interest in school behavioral health (SBH) will be facilitated 
by networks the PI and Co-I/MD site-PI lead, in particular, the Southeastern School Behavioral Health Community 
(SSBHC, see www.schoolbehavioralhealth.org) and the Center for School Mental Health (CSMH, see 
http://csmh.umaryland.edu ), with an anticipated reach to more than 100,000 stakeholders by study end. More than 
26,000 schools in the nation are implementing MTSS, as in systems such as Positive Behavioral Interventions and 
Supports (PBIS, see www.pbis.org).  Within these efforts, there is increasing priority on effective SBH (17).  The CSMH 
reports a dramatic increase in technical assistance requests for evidence-based mental health practice implementation 
in schools, with a marked uptick following incidents of mass violence in schools. With certainty, national centers for SBH 
and PBIS will use findings from the proposed study in their ongoing work to expand and improve the quality of SBH and 
MTSS in the U.S., and these centers have close working relationships with the professional organizations and networks 
mentioned, and many others. 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
19 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
Please also note that the PI, Co-I/MD site-PI and research team have significant publication records in the SBH area, 
with publications reflecting breadth in outlets of peer-reviewed empirical articles and reviews, books and book chapters, 
monographs, widely disseminated white papers, e-books, research briefs, newsletters and others.  As an example, for 
the recently completed PCORI Engagement Award, the PI and SAB co-chair led a series of 4 issues (the entire 2017 
volume) of the Report on Emotional & Behavioral Disorders in Youth, including 15 articles focused on ways to increase 
the effectiveness of stakeholder driven SBH (75-78).  With significant input from the SAB, user-friendly briefs on study 
findings and lessons learned will be developed and improved after each year, and finalized and broadly disseminated in 
Years 4 and 5 of the study via the SSBHC and CSMH networks and others the study team is connected to (see 
www.carolinanetwork.org, www.pbis.org , www.smhile.com).  
B. Three prominent barriers to disseminating and implementing results from this study in other settings are 
federalism, information management challenges, and discipline and youth-serving system silos. Federalism.  A major 
issue affecting the scaling up of evidence-based strategies to improve student functioning is federalism, including the 
emphasis on states' rights and local control, which can result in highly variable experiences across communities and 
states, and even within them (79). To address this challenge, in our work we have emphasized a community of practice 
approach (80), and effectively convening groups at local and state levels to support the increasing coherence and impact 
of SBH initiatives, as in the work occurring through the SSBHC and CSMH. In particular, states play a crucial role in 
overcoming the challenge of federalism, when they assume liaison roles between federal and national initiatives and 
support, and experiences and resources from other states, and getting information and resources down to 
counties/school districts, and in turn tracking and disseminating information on best practices occurring at local 
levels.  Overcoming challenges of federalism has been a major focus of the CSMH; PI, Weist, and Co-I and Maryland Site 
PI, Hoover. Both the CSMH and the SSBHC offer opportunities for state leaders to present and share innovative 
initiatives during national and regional conferences and webinars, with explicit support for promoting cross-state 
innovation adoption. We are confident that these methods will help to overcome the challenge of federalism, and 
provide an important national exemplar on accomplishing the same. Information Management. Rapidly managing 
diverse sources of information through diverse media is an increasing challenge (81). We will help diverse stakeholders 
use information from the study by developing materials in multiple and user-friendly formats as in the above broad 
array of strategies planned for dissemination, repetitiveness in sharing key messages, and identifying and publicizing the 
experiences of exemplar sites, and their experiences in moving toward greater effectiveness and impact. Two specific 
strategies that have been employed by our team are Playbooks and Infographics.  To date, the CSMH has published two 
Playbooks that succinctly review best practices and tips from the field on critical practice strategies. The two published 
playbooks (School Mental Health Screening, School Mental Health Teaming) shared lessons from the field on how to 
implement tested strategies for practice improvement, with easily understandable text and language and ready-to-use 
tools. For example, the Teaming playbook includes action steps for schools and communities to engage in community-
partnered SBH models, with templates for memoranda of understanding between schools and communities and tools to 
define roles and responsibilities of school- and community-employed support staff. Infographics have also been used by 
the CSMH as a tool for communicating the need for and value of providing mental health services in schools. These types 
of communication tools will be used to communicate findings from the proposed study in a manner that is easily 
digestible and usable by others in the field. Discipline and Youth-Serving System Silos. An unfortunate reality is that 
discipline and youth-serving systems often operate in isolation (16). Since the SBH field is inherently interdisciplinary, 
and operates in the nexus of many systems, efforts may be impeded when disciplines and systems fail to interact 
meaningfully.  As above, to address this challenge, we will connect to the SSBHC, CSMH and other communities of 
practice, and develop a range of interactive information sharing strategies, distilling information down into user-friendly 
briefs and infographics, and using a range of approaches (ECHO video conferencing, professional presentations, smaller 
meetings and discussion forums) to move discussions to dialogue, collaboration, and positive policy change as guided by 
the SAB and PCORI project officers. We look forward to discussions with and active guidance from PCORI project officers 
on building strategies for meaningful interdisciplinary and cross-system collaboration to advance effective SBH initiatives 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
20 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
and to share lessons learned and findings from exemplary initiatives as in the intervention tested in the proposed study. 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
21 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
 
PROTECTION OF HUMAN SUBJECTS.
 
Twenty-two middle schools (12 in South Carolina and 10 in Maryland) will participate in a cluster-randomized controlled 
trial with stratified random assignment used to assign schools to the Wellness Group or the Partnership Group for a 5-
year longitudinal study. These schools will be from two school districts including urban and suburban (MD), and rural 
and rural/suburban (SC) sites. Schools will utilize existing referral processes for identifying students in need of Tier 3 
services. Critical demographic factors will be assessed to investigate whether members of underrepresented groups 
present a unique need for services. It is estimated that over the course of the study, approximately 4,000 
students/families will be eligible to participate, and about 30% will provide assent/consent. Thus, student participants 
will include an estimated 1,225 youth (approximately 612 in each condition) across 12 middle schools in South Carolina 
and 10 middle schools in Maryland. Students may participate in the sixth through eighth grades. The two interventions 
being evaluated are systemic and will be a part of normal educational or therapeutic treatment that the students would 
ordinarily receive. Students receiving Tier 3 services and their parents will be consented to provide assessment batteries 
at in-take and every three months during treatment. For students not receiving treatment with a clinician, researchers 
will implement an opt-out procedure, informing students and their parents that researchers will be collecting de-
identified school records and climate survey on students in the school and that they may choose to opt out if they 
desire. Significant collaboration with the SAB will occur to ensure the protection of participants’ privacy and 
confidentiality.  
Potential Risks. Risks for participants (all students in the 22 participating schools) are minimal, with the greatest risk 
being the release of confidential information, and boredom from completing routine assessments. Students who receive 
individual counseling by a clinician will complete some items of a sensitive nature, and may feel uncomfortable with 
surveys. Strict protections will be in place per procedures of the experienced research team for assuring student 
confidentiality and privacy. There is also a risk that the study will identify students with urgent emotional/behavioral 
issues and need for urgent services, which will be provided by mental health providers in the schools.  
Recruitment and Informed Consent. Participation occurs at two different levels- school wide participation where 
involvement includes typical educational activities (e.g., completing the climate survey) and the release of de-identified 
school record data. As above, academic record data will be collected school-wide without parental consent/student 
assent related to the education systems research exemption for the collection of these data. For the school climate 
measure, an opt-out procedure will be in place where parents receive a letter taken home by students, as well as email 
announcements about the study. Each letter will provide instructions for how to opt-out of the study, including by email 
and calling the study team. Additionally, students receiving therapeutic counseling at school (referrals for counseling will 
follow established mechanisms already in place to refer students for services) and their parents will be informed of the 
study (by their clinician) and will have the option to consent to participate in the study which will be integrated into the 
typical consent for treatment process that is required by mental health providers.  
Protections Against Risk. The study team has significant experience in implementing data collection, management, 
and analyses processes and the physical and personnel resources to carry out these procedures. Per other active studies, 
data will be stored and managed at MUSC and USC and transported through confidential correspondences with research 
personnel collaborating with the participating schools. For each of the research aims, all data will be confidential, with 
strict protections on privacy. All information obtained will be stored in locked file cabinets in a locked room at MUSC and 
USC. Interventions evaluated in this study do not represent more than a minimal risk and therefore monitoring by the PI 
and IRB is presumed sufficient. Should PCORI program staff or the IRB recommend a Data Safety Monitoring Board, one 
will be established prior to study start.  
Potential Benefits of the Proposed Research to Human Subjects and Others. Team member participants are likely to 
benefit from involvement in the study by increased knowledge of utilizing data to inform decision making, significantly 
involving patients and stakeholders in shaping care, and gaining experience with a range of evidence-based practices. 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
24 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
study will operate with the major partner, the 
 Mental Health Center, a leading site for the delivery of 
SBH within the SCDMH system. The SCDMH will commit personnel as outlined in their budget for 12 employees.   

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
25 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
 
REFERENCES CITED  
 
1) Costello, E.J., Foley, D.L., & Angold, A. (2006). 10-year research update review: The epidemiology of child and 
adolescent psychiatric disorders: II. developmental epidemiology. Journal of the American Academy of Child & 
Adolescent Psychiatry, 45(1), 8–25. 
2) Merikangas, K.R., He, J.P., Brody, D., Fisher, P.W., Bourdon, K., & Koretz, D.S. (2010). Prevalence and treatment of 
mental disorders among US children in the 2001-2004 NHANES. Pediatrics, 125(1), 75-81.  
3) Merikangas, K.R., He, J.P., Burstein, M., Swendsen, J., Avenevoli, S., Case, B., … & Olfson, M. (2011). Service 
utilization for lifetime mental disorders in U.S. adolescents: Results of the National Comorbidity Survey Replication—
Adolescent Supplement (NCS-A). Journal of the American Academy of Child & Adolescent Psychiatry, 50(1), 32–45.  
4) U.S. Public Health Service. (2000). Report of the Surgeon General's Conference on Children's Mental Health: A 
National Action Agenda. Washington, DC: Department of Health and Human Services. (Publication No. 017-024-
01659-4). 
5) Lacourse, E., Cŏté, S., Nagin, D. S., Vitaro, F., Brendgen, M., & Tremblay, R. E. (2002). A longitudinal-experimental 
approach to testing theories of antisocial behavior development. Development and Psychopathology, 14(4), 909-
924. 
6) Harpin, V., Mazzone, L., Raynaud, J. P., Kahle, J., & Hodgkins, P. (2016). Long-term outcomes of ADHD: A systematic 
review of self-esteem and social function. Journal of Attention Disorders, 20(4), 295-305.  
7) Colton, C.W., & Manderscheid, R.W. (2006). Congruencies in increased mortality rates, years of potential life lost, 
and causes of death among public mental health clients in eight states. Preventing Chronic Disease, 3(2), 1-14. 
8) Kessler, R.C., Foster, C.L., Saunders, W.B., & Stang, P.E. (1995). Social consequences of psychiatric disorders, I: 
Educational attainment. The American Journal of Psychiatry, 152(7), 1026-1032. 
9) Levinson, D., Lakoma, M.D., Petukhova, M., Schoenbaum, M., Zaslavsky, A.M., Angermeyer, M., ... & Gureje, O. 
(2010). Associations of serious mental illness with earnings: Results from the WHO World Mental Health surveys. 
The British Journal of Psychiatry, 197(2), 114-121. 
10) Vermeiren, R., Jespers, I., & Moffitt, T. (2006). Mental health problems in juvenile justice populations. Child and 
Adolescent Psychiatric Clinics of North America, 15(2), 333-351. 
11) Roeser, R.W., Eccles, J.S., & Sameroff, A. J. (1998). Academic and emotional functioning in early adolescence: 
Longitudinal relations, patterns, and prediction by experience in middle school. Development and Psychopathology, 
10(2), 321-352. 
12) Steinberg, L., & Morris, A. S. (2001). Adolescent development. Annual Review of Psychology, 52(1), 83-110. 
13) Hankin, B. L. (2006). Adolescent depression: Description, causes, and interventions. Epilepsy & Behavior, 8(1), 102-
114. 
14) Merikangas, K.R., He, J.P., Burstein, M., Swanson, S.A., Avenevoli, S., Cui, L., ... & Swendsen, J. (2010). Lifetime 
prevalence of mental disorders in US adolescents: Results from the National Comorbidity Survey Replication–
Adolescent Supplement (NCS-A). Journal of the American Academy of Child & Adolescent Psychiatry, 49(10), 980-
989. 
15) Rones, M., & Hoagwood, K. (2000). School-based mental health services: A research review. Clinical Child and Family 
Psychology Review, 3(4), 223-241. 
16) Weist, M.D., Lever, N.A., Bradshaw, C.P., & Owens, J.S. (2014). Further advancing the field of school mental health. 
In M.D. Weist, N.A. Lever, C.P. Bradshaw, & J.S. Owens (Eds.), Handbook of school mental health: Research, training, 
practice, and policy, 2nd edition  (pp. 1-14). New York: Springer. 
17) Barrett, S., Eber, L., & Weist, M.D. (2013). Advancing education effectiveness: An Interconnected Systems 
Framework for Positive Behavioral Interventions and Supports (PBIS) and school mental health. Center for Positive 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
26 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
Behavioral Interventions and Supports (funded by the Office of Special Education Programs, US Department of 
Education). Eugene, Oregon: University of Oregon Press. 
18) Weist, M.D., Lever, N., Stephan, S., Youngstrom, E.A., Moore, E., Harrison, B., … & Stiegler, K. (2009). Formative 
evaluation of a framework for high quality, evidence-based services in school mental health. School Mental Health. 
1(3), 196-211. 
19) Weist, M.D., Stephan, S., Lever, N., Youngstrom, E.A., McDaniel, H., … & Hoagwood, K. (under review). Impact of a 
package of evidence-based practices tailored to the school context on student psychosocial outcomes: A 
randomized trial. School Mental Health. 
20) Weist, M.D., Garbacz, S.A., Lane, K.L., & Kincaid, D. (2017). Aligning and integrating family engagement in Positive 
Behavioral Interventions and Supports (PBIS): Concepts and strategies for families and schools in key contexts. Center 
for Positive Behavioral Interventions and Supports (funded by the Office of Special Education Programs, U.S. 
Department of Education). Eugene, Oregon: University of Oregon Press. 
21) Chandra, A., & Minkovitz, C. S. (2007). Factors that influence mental health stigma among 8th grade adolescents. 
Journal of Youth and Adolescence, 36(6), 763-774. 
22) Olvera, P., & Olvera, V. I. (2012). Optimizing home-school collaboration: Strategies for school psychologists and 
Latino parent involvement for positive mental health outcomes. Contemporary School Psychology, 16, 77-88.  
23) Graham, J.W., Pettigrew, J., Miller-Day, M., Krieger, J.L., Zhou, J., & Hecht, M.L. (2014). Random assignment of 
schools to groups in the drug resistance strategies rural project: Some new methodological twists. Prevention 
Science, 15(4), 516-525. 
24) Graham, J.W., Flay, B.R., Johnson, C.A., Hansen, W.B., & Collins, L.M. (1984). Group comparability: A multiattribute 
utility measurement approach to the use of random assignment with small numbers of aggregated units. Evaluation 
Review, 8(2), 247-260. 
25) Zhou, C., Crawford, A., Serhal, E., Kurdyak, P., & Sockalingam, S. (2016). The impact of project ECHO on participant 
and patient outcomes: A systematic review. Academic Medicine, 91(10), 1439-1461. 
26) Arora, S., Thornton, K., Komaromy, M., Kalishman, S., Katzman, J., & Duhigg, D. (2014). Demonopolizing medical 
knowledge. Academic Medicine, 89(1), 30-32. 
27) Chorpita, B.F., Becker, K.D., Daleiden, E.L., & Hamilton, J.D. (2007). Understanding the common elements of 
evidence-based practice. Journal of the American Academy of Child & Adolescent Psychiatry, 46(5), 647-652. 
28) Fixsen, D.L., Naoom, S.F., Blase, K.A., Friedman, R.M. & Wallace, F. (2005). Implementation Research: A Synthesis of 
the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National 
Implementation Research Network (FMHI Publication #231). 
29) Connors, E.H., Stephan, S.H., Lever, N., Ereshefsky, S., Mosby, A., & Bohnenkamp, J. (2016). A national initiative to 
advance school mental health performance measurement in the US. Advances in School Mental Health Promotion, 
9(1), 50-69. 
30) Weist, M.D., Sander, M.A., Walrath, C., Link, B., Nabors, L., Adelsheim, S., ... & Carrillo, K. (2005). Developing 
principles for best practice in expanded school mental health. Journal of Youth and Adolescence, 34(1), 7-13. 
31) Nabors, L.A., Lehmkuhl, H.D., & Weist, M.D. (2003). Continuous quality improvement and evaluation of expanded 
school mental health programs. In M.D. Weist, S.W. Evans, & N.A. Lever (Eds.), Handbook of school mental health: 
Advancing practice and research (pp. 275-284). New York, NY: Springer. 
32) McKay, M.M., & Bannon, W.M. (2004). Engaging families in child mental health services. Child and Adolescent 
Psychiatric Clinics of North America, 13(4), 905-921. 
33) Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179-
211. 
34) Evans, S.W., & Weist, M.D. (2004). Implementing empirically supported treatments in the schools: What are we 
asking? Clinical Child and Family Psychology Review, 7(4), 263-267. 
35) Schaeffer, C.M., Bruns, E., Weist, M.D., Stephan, S.H., Goldstein, J., & Simpson, Y. (2005). Overcoming challenges to 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
27 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
using evidence-based interventions in schools. Journal of Youth and Adolescence, 34(1), 15-22. 
36) Chorpita, B.F. (2007). Modular cognitive-behavioral therapy for childhood anxiety disorders. New York: Guilford 
Press. 
37) Weisz, J.R., Chorpita, B.F., Palinkas, L.A., Schoenwald, S.K., Miranda, J., Bearman, S.K., ... & Gray, J. (2012). Testing 
standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A 
randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274-282. 
38) Forman, S.G., Olin, S.S., Hoagwood, K.E., Crowe, M., & Saka, N. (2009). Evidence-based interventions in schools: 
Developers’ views of implementation barriers and facilitators. School Mental Health, 1, 26-36. 
39) Langley, A.K., Nadeem, E., Kataoka, S.H., Stein, B.D., & Jaycox, L.H. (2010). Evidence-based mental health programs 
in schools: Barriers and facilitators of successful implementation. School Mental Health, 2(3), 105-113. 
40) Chorpita, B.F., & Weisz, J.R. (2009). MATCH-ADTC: Modular approach to therapy for children with anxiety, 
depression, trauma, or conduct problems. PracticeWise. 
41) Institute for Healthcare Improvement. (2007). Plan-Do-Study-Act Worksheet. Retrieved February 15, 2017 from 
http://www.ihi.org/IHI/Topics/Improvement/ImprovementMethods/Tools/Plan-Do-Study-
Act+(PDSA)+Worksheet.htm 
42) Walker, J., Koroloff, N., & Schutte, K. (2002). Implementing high-quality team-based individualized services planning: 
Necessary and sufficient conditions. Portland, OR: Research and Training Center on Family Support and Children's 
Mental Health. 
43) Kutcher, S., Wei, Y., & Morgan, C. (2015). Successful application of a Canadian mental health curriculum resource by 
usual classroom teachers in significantly and sustainably improving student mental health literacy. The Canadian 
Journal of Psychiatry, 60(12), 580-586. 
44) Mcluckie, A., Kutcher, S., Wei, Y., & Weaver, C. (2014). Sustained improvements in students’ mental health literacy 
with use of a mental health curriculum in Canadian schools. BMC Psychiatry, 14(1), 379. 
45) Lambert, M.J., Burlingame, G.M., Umphress, V., Hansen, N.B., Vermeersch, D.A., Clouse, G.C., & Yanchar, S.C. (1996). 
The reliability and validity of the Outcome Questionnaire. Clinical Psychology & Psychotherapy, 3(4), 249-258. 
46) Miller, S.D., Duncan, B.L., Brown, J., Sparks, J.A., & Claud, D.A. (2003). The Outcome Rating Scale: A preliminary study 
of the reliability, validity, and feasibility of a brief visual analog measure. Journal of Brief Therapy, 2(2), 91-100. 
47) Miller, S.D., Duncan, B.L., Sorrell, R., & Brown, G.S. (2005). The Partners for Change Outcome Management System. 
Journal of Clinical Psychology, 61(2), 199-208. 
48) Haine-Schlagel, R., Roesch, S.C., Trask, E.V., Fawley-King, K., Ganger, W.C., & Aarons, G.A. (2016). The Parent 
Participation Engagement Measure (PPEM): Reliability and validity in child and adolescent community mental health 
services. Administration and Policy in Mental Health and Mental Health Services Research, 43(5), 813-823. 
49) Briesch, A.M., Chafouleas, S.M., Neugebauer, S.R., & Riley-Tillman, T.C. (2013). Assessing influences on intervention 
implementation: Revision of the Usage Rating Profile-Intervention. Journal of School Psychology, 51(1), 81-96. 
50) Larsen, D.L., Attkisson, C.C., Hargreaves, W.A., and Nguyen, T.D. (1979). Assessment of client/patient satisfaction: 
Development of a general scale. Evaluation and Program Planning, 2, 197-207  
51) Price, C.S., Spence, S.H., Sheffield, J., & Donovan, C. (2002). The development and psychometric properties of a 
measure of social and adaptive functioning for children and adolescents. Journal of Clinical Child and Adolescent 
Psychology, 31(1), 111-122. 
52) Chorpita, B.F., Reise, S., Weisz, J.R., Grubbs, K., Becker, K.D., & Krull, J.L. (2010). Evaluation of the Brief Problem 
Checklist: Child and caregiver interviews to measure clinical progress. Journal of Consulting and Clinical Psychology, 
78(4), 526-536. 
53) Achenbach, T.M., & Rescorla, L. (2001). ASEBA school-age forms & profiles. 
54) Brener, N.D., Collins, J.L., Kann, L., Warren, C.W., & Williams, B.I. (1995). Reliability of the youth risk behavior survey 
questionnaire. American Journal of Epidemiology, 141(6), 575-580.  

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
28 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
55) Abrishami, G.F. & Warren, J.S. (2013). Therapeutic alliance and outcomes in children and adolescents served in a 
community mental health system. Journal of Child & Adolescent Behavior, 1(2), 1-7. 
56) Snijders, T., & Boskers, R. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. 
Thousand Oaks, CA: Sage. 
57) Enders, C.K. (2010). Applied missing data analysis. New York: Guilford Press.  
58) Rogosa, D. (1988). Myths about longitudinal research. In K.W. Schaie, R.T. Campbell, W. Meredith, & S.C. Rawlings 
(Eds.), Methodological issues in aging research (pp. 171-209). New York: Springer. 
59) Rogosa, D., Brandt, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. 
Psychological Bulletin, 92(3), 726-748. 
60) Rogosa, D., & Willett, J.B. (1985). Understanding correlates of change by modeling individual differences in growth. 
Psychometrika, 50(2), 203-228. 
61) Hill, J.L. (2011). Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical 
Statistics, 20(1), 217-240. 
62) Hedges, L.V., & Hedberg, E.C. (2007). Intraclass correlation values for planning group-randomized trials in education. 
Educational Evaluation and Policy Analysis, 29(1), 60-87. 
63) Anker, M.G., Duncan, B.L., & Sparks, J.A. (2009). Using client feedback to improve couple therapy outcomes: a 
randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology, 77(4), 693–704.  
64) Reese, R.J., Norsworthy, L.A., & Rowlands, S.R. (2009). Does a continuous feedback system improve psychotherapy 
outcome? Psychotherapy: Theory, Research, Practice, Training, 46(4), 418-431. 
65) Lambert, M.J., & Shimokawa, K. (2011). Collecting client feedback. Psychotherapy, 48, 72–79. 
66) Slone, N.C., Reese, R.J., Mathews-Duvall, S., & Kodet, J. (2015). Evaluating the efficacy of client feedback in group 
psychotherapy. Group Dynamics: Theory, Research, and Practice, 19(2), 122-136. 
67) Mehta, N., Clement, S., Marcus, E., Stona, A.C., Bezborodovs, N., Evans-Lacko, S., … Thornicroft, G. (2015). Evidence 
for effective interventions to reduce mental health-related stigma and discrimination in the medium and long term: 
Systematic review. The British Journal of Psychiatry, 207(5), 377–384.  
68) Pinto-Foltz, M.D., Logsdon, M.C., & Myers, J.A. (2011). Feasibility, acceptability, and initial efficacy of a knowledge-
contact program to reduce mental illness stigma and improve mental health literacy in adolescents. Social Science & 
Medicine, 72(12), 2011-2019. 
69) Frangakis CE & Rubin DB. (2002). Principal stratification in causal inference. Biometrics, 58, 21–29. 
70) Stuart, EA et al. (2008). Estimating intervention effects of prevention programs: Accounting for noncompliance. 
Prevention Science, 9, 288-298. 
71) Lyons, M., McQuillin, S. (Under review) A reanalysis of the Student Mentoring Program: Estimating treatment effects 
by the quality of the mentoring relationship. Manuscript revised and resubmitted for publication. 
72) Rubin, D. B. (1978) “Bayesian Inference for Causal Effects: The Role of Randomization.” Annals of Statistics 6 (1): 34–
58  
73) Rubin, D.B. (2011) Causal Inference Using Potential Outcomes, Journal of the American Statistical 
Association, 100:469, 322-331. 
74) Kapelner, A., & Bleich, J. (2015). Prediction with missing data via Bayesian Additive Regression Trees. Canadian 
Journal of Statistics, 43(2), 224-239. 
75) Stevens, R., & Weist, M.D. (2017). Partnerships and school behavioral health. Report on Emotional & Behavioral 
Disorders in Youth, 17(2), 29-31. 
76) Stevens, R., & Weist, M.D. (2017). The Patient Centered Outcomes Research Institute and support for school 
behavioral health. Report on Emotional & Behavioral Disorders in Youth, 17(4), 81-85.  
77) Weist, M.D., & Stevens, R. (2017). Advancing school behavioral health. Report on Emotional & Behavioral Disorders 
in Youth, 17(1), 1-5. 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
29 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
78) Weist, M.D., & Stevens, R. (2017). The Importance of School Behavioral Health in 2017. Report on Emotional & 
Behavioral Disorders in Youth, 17(3), 55-57.  
79) Weist, M.D., & Paternite, C.E. (2006). Building an interconnected policy-training-practice-research agenda to 
advance school mental health. Education and Treatment of Children, 29, 173-196. 
80) Wenger, E., McDermott, R.A., & Snyder, W. (2002). Cultivating communities of practice: A guide to managing 
knowledge. Boston: Harvard Business Press. 
81) Alavi, M., & Leidner, D.E. (2001). Knowledge management and knowledge management systems: Conceptual 
foundations and research issues. MIS Quarterly, 25(1), 107-136. 
 
 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
30 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
Appendix 
 
Table 1 
 
Descriptive Statistics of Wellness + Partnership Component  
 
Wellness + Partnership Components 
N 
Min 
Max 
Mean 
SD 
Assuring families are strongly involved in 
services 
92 
2 
6 
5.6 
.79 
School Administrators 
6 
5 
6 
5.8 
.41 
Educators 
12 
2 
6 
5.0 
1.48 
School Counselors 
18 
5 
6 
5.8 
.43 
School-Based Clinicians 
32 
4 
6 
5.7 
.63 
Psychologists and Research Staff 
10 
4 
6 
5.4 
.84 
Community/Family Organizations 
5 
5 
6 
5.6 
.55 
Taking steps to reduce the stigma of services 
91 
2 
6 
5.5 
.85 
School Administrators 
6 
4 
6 
5.5 
.84 
Educators 
12 
2 
6 
5.0 
1.35 
School Counselors 
18 
5 
6 
5.8 
.43 
School-Based Clinicians 
32 
4 
6 
5.6 
.72 
Psychologists and Research Staff 
10 
4 
6 
5.4 
.84 
Community/Family Organizations 
5 
6 
6 
6.0 
0 
Implementing services shown to be effective 
90 
4 
6 
5.7 
.57 
School Administrators 
6 
5 
6 
5.7 
.52 
Educators 
12 
4 
6 
5.5 
.80 
School Counselors 
17 
5 
6 
5.9 
.33 
School-Based Clinicians 
32 
4 
6 
5.6 
.61 
Psychologists and Research Staff 
10 
5 
6 
5.9 
.32 
Community/Family Organizations 
5 
5 
6 
5.8 
.45 
Assuring that services are of high quality 
88 
4 
6 
5.7 
.52 
School Administrators 
6 
6 
6 
6 
0 
Educators 
12 
4 
6 
5.3 
.78 
School Counselors 
18 
5 
6 
5.9 
.32 
School-Based Clinicians 
29 
5 
6 
5.7 
.46 
Psychologists and Research Staff 
10 
5 
6 
5.8 
.42 
Community/Family Organizations 
5 
6 
6 
6.0 
0 
Supporting staff for high quality 
implementation 
89 
4 
6 
5.8 
.48 
School Administrators 
6 
5 
6 
5.7 
.52 
Educators 
12 
4 
6 
5.3 
.65 
School Counselors 
18 
5 
6 
5.9 
.32 
School-Based Clinicians 
3 
5 
6 
5.8 
.38 
Psychologists and Research Staff 
10 
5 
6 
5.9 
.32 
Community/Family Organizations 
5 
6 
6 
6.0 
0 
Promoting school-family partnerships 
90 
2 
6 
5.7 
.69 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
31 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
School Administrators 
6 
5 
6 
5.8 
.41 
Educators 
12 
2 
6 
5.3 
1.23 
School Counselors 
17 
5 
6 
5.9 
.33 
School-Based Clinicians 
32 
4 
6 
5.7 
.65 
Psychologists and Research Staff 
10 
5 
6 
5.7 
.48 
Community/Family Organizations 
5 
5 
6 
5.8 
.45 
TOTAL 6 ITEM Wellness + Partnership SCALE 
AVERAGES  
90 
3.2 
6 
5.7 
.54 
 
Table 2 
 
Descriptive Statistics of Comparison Components 
 
Role 
Number 
Percent of Total 
School Administrators 
81 
15.3% 
Educators 
75 
14.2% 
School Counselors 
143 
27.1% 
Clinicians (Social Workers, Behavioral Specialists, 
DMH school-based clinicians, etc.) 
63 
11.9% 
Psychologists (all non-university employed 
psychologists) 
20 
3.8% 
University Faculty/Staff (includes University 
employed psychologists) 
54 
10.2% 
University Students 
42 
8.0% 
Community/Family Organizations 
42 
8.0% 
Parents 
5 
1% 
School Nurses 
1 
<1% 
School Resource Officers 
2 
<1% 
Total 
528 
100% 
 
 
 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
32 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
Table 3 
 
SAB – SC Team 
 
Name 
Organization 
Subgroup 
Louise Johnson 
SC Department of Mental Health 
MH/Community Partners 
John Payne 
SC Department of Education 
Education 
Lori Chappelle 
Waccamaw Mental Health Clinic 
MH/Community Partners 
Robert Hock 
University of SC- Community Partnerships 
MH/Community Partners 
Leshawn Hiers 
Youth patient 
Patients and Families 
Jenah Cason 
Federation of Families 
Patients and Families 
Beverly Griffin 
Federation of Families 
Patients and Families 
June Headley-Greenlaw 
Patient caregiver/Engagement Specialist 
Patients and Families 
Bev McCarty 
Patient caregiver 
Patients and Families 
Erika Kirby 
Blue Cross Blue Shield Foundation of SC 
 
Pete Liggett 
SC Department of Health and Human 
Services 
 
Rob Rhodes 
SC school staff in study 
Education 
Alice Matthews 
SC parent of a child in study 
Patients and Families 
 
 
Table 4 
 
SAB – MD Team 
 
Name 
Organization 
Subgroup 
Rev. Al Hathaway 
MH/Community 
MH/Community Partners 
Shanda Crowder 
Family League of Baltimore 
Patients and Families 
Deborah Badawi 
University of Maryland Pediatrics 
MH/Community Partners 
Felicia Taylor 
Patient caregiver 
Patients and Families 
Susan Tager 
Patient caregiver 
Patients and Families 
Michael Wills 
New Lens 
MH/Community Partners 
Terry Patton 
Retired MD teacher 
Education 
Nisha Sachdev 
Bainum Foundation 
 
Valerie Conaway 
MD parent of a child in study 
Patients and Families 
James Padden 
 Schools 
 
 

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
33 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        

 
 
 
PRINCIPAL INVESTIGATOR: Weist, Mark D. 
 
 
 
35 
PCORI Cycle 1 2018 Broad PFA: Research Plan Template                                                                        
Preservation of notes regarding Study Aim 3, removed in the May 2023 contract modification: 
 
The original research plan included a third study aim: Evaluate the follow-up effects of the intervention on social, EB, academic and health outcomes in a sample 
of students followed into high school. This aim was removed in a subsequent revision of the research plan (May 2023) due to significant challenges in contacting 
students who have been transferred to a new therapist in high school or who have been discharged from services. Additional references to Aim 3 have been 
removed from this plan to reduce confusion, but will be preserved in the appendix.  
 
Year 5 of the project (2022-2023) included plans to collect follow-up data from students who previously enrolled during middle school but had since moved on 
to 9th or 10th grades. The purpose of this follow-up timepoint was to discern whether students who were part of the Partnership condition sustained any 
improvements in social, emotional/behavioral, and academic functioning after transitioning to high school. Measures collected during follow-up in middle school 
would be used for follow-up for eligible students in 9th or 10th grade during the 2022-2023 school year.