Working to Implement and Sustain Digital Outcome Measures

Study Overview
=================
Brief Summary
-----------------
This study will investigate the effects of an implementation strategy called Leadership and Organizational Change for Implementation (LOCI), relative to training and technical assistance only (also known as 'implementation as usual' or IAU), on fidelity to, and youth service outcomes of, a well-established digital measurement-based care intervention called the Outcomes Questionnaire-Analyst in outpatient community mental health clinics.

Detailed Description
-----------------
This study will investigate the effects of LOCI on the fidelity and and clinical outcomes of a digital measurement-based care (MBC) system called the Outcomes Questionnaire Analyst (OQ-A). Over two phases, the investigators will recruit a total of 20 clinics with up to 2 first level leaders per clinic (max N of 40 total) and a minimum of 3 clinicians per site (60 total). In each phase, a unique cohort of 360 caregiver-youth dyads will be sampled from the participating clinics. The total caregiver-youth enrollment for two phases will be 720 (360*2). Clinics will be randomly assigned, using covariate constrained randomization, to the control condition (training and technical assistance to implement the OQ-A system) or the intervention condition (training and technical assistance in the OQ-A plus the LOCI implementation strategy). The IAU condition will include training and technical assistance in the OQ-A system only. The intervention condition will include training and technical assistance in the OQ-A system plus the LOCI strategy.

Official Title
-----------------
Randomized Trial of a Leadership and Organizational Change Strategy to Improve the Implementation and Sustainment of Digital Measurement-based Care in Youth Mental Health Services

Conditions
-----------------
Implementation, Behavioral Symptoms

Intervention / Treatment
-----------------
* Behavioral: Leadership for Organization Change and Implementation (LOCI)
* Behavioral: Training and Technical Assistance Only


Participation Criteria
=================
Eligibility Criteria
-----------------
Participants for this project will fall into four groups: 1) clinic executives and upper leaders (i.e., Chief Executive Officer, Executive Directors, program administrators, n=20), 2) clinic first-level leaders (i.e. clinical supervisors, n=20), 3) clinicians serving children with emotional and behavioral disorders (n=120), and 4) caregivers of children with emotional and behavioral disorders (n=720). Caregivers of children with emotional and behavioral disorders will be recruited in two distinct cohorts of 360 each. Inclusion criteria for all groups of participants are intentionally broad to ensure robust and valid analyses. Inclusion Criteria for Clinics Provide outpatient psychotherapy services to children ages 4 to 18 who have emotional and/ or behavioral disorders and their families Has at least 3 full time equivalent clinicians on staff Not currently implementing a digital measurement-based care system clinic wide Inclusion Criteria for Executives and Upper Leaders 1. Identified as Chief Executive Officer, Executive Director, or high-level administrator at an enrolled clinic Inclusion Criteria for first-level leaders 1. Identified as a clinical supervisor or clinical work-group supervisor/ leader at an enrolled clinic Inclusion Criteria for Clinicians Employed as staff at participating clinic Provides services to 3 or more eligible youth clients Inclusion Criteria for Child/ Caregiver Dyads Child is ages 4 to 18 years Child has been diagnosed with an emotional or behavioral disorder by clinic staff Clinical staff at the site determined that the youth's treatment needs can be appropriately addressed by the clinic

Ages Eligible for Study
-----------------
Minimum Age: 4 Years

Sexes Eligible for Study
-----------------
All

Accepts Healthy Volunteers
-----------------
Accepts Healthy Volunteers

Study Plan
=================
How is the study designed?
-----------------

Design Details

Primary Purpose: Health Services Research
Allocation: Randomized
Intervention Model: Parallel Assignment
Masking: Double


Arms and Interventions

| Participant Group/Arm | Intervention/Treatment |
| --- | --- |
| Experimental: LOCI (Intervention)<br>In clinics assigned to the LOCI condition, executives and first-level leaders will receive leadership training and coaching to support implementation of the OQ-A system. In addition, leaders and clinicians in this condition will receive training and technical assistance to implement the OQ-A measurement-based care system. | Behavioral: Leadership for Organization Change and Implementation (LOCI)<br>* LOCI is a multicomponent implementation strategy that engages organizational executives and first-level leaders (i.e., those who administratively supervise clinicians) to build an organizational climate to support the implementation of a focal evidence-based practice (EBP) with fidelity. In this study, the focal EBP is the OQ-A system. LOCI includes two overarching components: (1) monthly organizational strategy meetings between executives and LOCI consultants/trainers to develop and embed policies, procedures, and practices that support implementation of a focal EBP, and (2) training and coaching for first-level leaders, to develop their skills in leading implementation. The aim of these components is to develop an organizational implementation climate in which clinicians' perceive that use of the OQ-A with high fidelity is expected, supported, and rewarded.<br>Behavioral: Training and Technical Assistance Only<br>* All leaders and clinicians in participating clinics will receive standardized OQ-A training and technical assistance provided by the OQ-A purveyor organization. This includes an initial, 6-hr, in-person OQ-A training; two, live, virtual, 1-hr booster trainings, offered 3 and 5 months after the initial training; and, year-round technical assistance from the OQ-A purveyor organization. Technical assistance includes virtual training sessions, online library of training videos, and customer care representative for technical support. In addition, to encourage participation in the study, a set of four 1-hr, web-based general leadership seminars will be offered to leaders in the control condition. These will cover topics ranging from effective leadership, to giving effective feedback.<br>|
| Active Comparator: Training and Technical Assistance only (Control)<br>In clinics assigned to the control group, leaders and clinicians will receive training and technical assistance to implement the OQ-A measurement-based care system. In addition, to support enrollment in this condition, leaders in this condition will be offered access to general web-based leadership seminars. | Behavioral: Training and Technical Assistance Only<br>* All leaders and clinicians in participating clinics will receive standardized OQ-A training and technical assistance provided by the OQ-A purveyor organization. This includes an initial, 6-hr, in-person OQ-A training; two, live, virtual, 1-hr booster trainings, offered 3 and 5 months after the initial training; and, year-round technical assistance from the OQ-A purveyor organization. Technical assistance includes virtual training sessions, online library of training videos, and customer care representative for technical support. In addition, to encourage participation in the study, a set of four 1-hr, web-based general leadership seminars will be offered to leaders in the control condition. These will cover topics ranging from effective leadership, to giving effective feedback.<br>|

What is the study measuring?
-----------------
Primary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| MBC fidelity index | Fidelity to the OQ-A will be measured by using electronic meta-data from the OQ-A system combined with caregiver reported information on the number of sessions the youth attended. For each youth, a fidelity index will be generated that represents the product of two quantities: (a) the youth's completion rate (i.e., number of measures administered relative to the number of sessions attended within the 6-month observation period), and (b) the youth's viewing rate (i.e., the number of feedback reports viewed by the clinician relative to the number of measures administered). Note that this product is equivalent to the ratio of viewed feedback reports to total sessions; it represents an events/trials proportion. MBC fidelity index scores summarize the level of MBC fidelity experienced by each youth (range=0-1). | 0-6 months after youth's baseline/ entry into treatment | 
| Change from Baseline to 6-months in Youth Total Problems Score on the Short form Assessment for Children (SAC) | The SAC Total Problem Score is a 48-item measure of youth internalizing (e.g., anxious, depressed) and externalizing (e.g., aggressive, noncompliant, overactive) behaviors (range=0-96). Caregivers will complete the SAC at the youth's baseline (entry into treatment) and monthly thereafter for six months regardless of the youth's continued participation in treatment. Higher SAC scores indicate more problems. | 0-6 months after youth's baseline/ entry into treatment | 

Secondary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Change from Baseline in First-level Leaders' Implementation Leadership (ILS) | Clinicians will assess the extent to which their first-level leaders (e.g., clinical program manager) exhibit implementation leadership behaviors with regard to the OQ-A using the 12-item Implementation Leadership Scale (ILS). Responses are made on a 0 (not at all) to 4 (very great extent) scale. Total scores are calculated as the mean of all items. Higher scores indicate more extensive demonstration of implementation leadership behaviors. This outcome will be assessed at baseline, 4-, 8-, 12-, and 18-months. Clinician responses will be aggregated to the clinic level as appropriate. | 0-18 months | 
| Change from Baseline in First-level Leaders' Transformational Leadership (MLQ) | Clinicians will assess the extent to which their first-level leaders exhibit transformational leadership behaviors using the Multifactor Leadership Questionnaire (MLQ). Responses are made on a 5-point scale (Not at all to Frequently, if not always). Consistent with prior studies, we will use the 20-item transformational leadership total score, calculated as the mean of four subscales: idealized influence, inspirational motivation, intellectual stimulation, and individual consideration. This outcome will be assessed at baseline, 4-, 8-, 12-, and 18-months. Clinician responses will be aggregated to the clinic level as appropriate. | 0-18 months | 
| Change from Baseline in Clinic Implementation Climate for OQ-A | Clinicians' perceptions of their clinics' implementation climate for OQ-A will be measured using the 18-item Implementation Climate Scale (ICS). Responses are made on a 0 (Not at All) to 4 (A Very Great Extent) scale with the total score calculated as the mean of all items. Higher scores indicate higher levels of implementation climate for OQ-A. This outcome will be assessed at baseline, 4-, 8-, 12-, and 18-months. Clinician responses will be aggregated to the clinic level as appropriate. | 0-18 months | 

 Terms related to the study
=================
Keywords Provided by Centre Hospitalier Valida
-----------------
LOCI, measurement-based care, evidence-based practice, behavioral health, youth mental health, leadership, implementation science