Measurement Training and Feedback System: Family-Based Services
Condition: Substance Use Disorders
MeSH terms: Substance-Related Disorders
Study type: Interventional

Summary: 
      National mandates for increasing accountability in behavioral healthcare have created an
      urgent need for effective quality assurance (QA) procedures featuring pragmatic measures of
      treatment implementation quality in usual care (UC). The most practical method for assessing
      treatment quality in UC is therapist self-report; unfortunately, therapist-report measures of
      fidelity to evidence-based practices (EBPs) have demonstrated weak validity to date. This
      study addresses this need for effective QA procedures by first developing treatment quality
      assurance procedures designed to increase the adoption and quality of empirically supported
      family-based services (FBS) for adolescent substance use (ASU) in usual care, and then
      testing two system-level implementation strategies for installing the new quality procedures
      in ASU treatment sites.

      FBS have achieved the strongest evidence base for treating ASU and are a prime candidate for
      upgrading the quality of ASU services in various systems of care. FBS comprise both family
      participation in services, the systemic parameters wherein family members are included in
      assessment and treatment activities; and family therapy techniques, the specific
      interventions that clinicians use to directly target family members and family functioning
      for change. For FBS to fulfill their potential to enhance ASU treatment systems, FBS
      implementation must be supported by effective quality assurance procedures designed to ensure
      that FBS are delivered with fidelity. The proposed study will leverage a strong
      research-government partnership between the applicant organization and the New York
      single-state agency for SU services.

      The study will first develop innovative quality procedures (Aim 1) that use existing FBS
      quality metrics to promote high-fidelity FBS: Measurement Training and Feedback System for
      Implementation (MTFS-I), a pragmatic evidence-based method for increasing FBS quality by
      providing monthly feedback on therapist-reported FBS delivery along with brief online FBS
      training modules. The study will then experimentally compare two system-level implementation
      strategies designed to foster MTFS-I utilization in usual care for ASU. Core Training Only
      will contain two 3-hour training sessions: Mapping existing FBS and identifying site goals
      for FBS improvement; and Installing and sustaining the MTFS-I. Core + Facilitation is an
      additive strategy that will begin with the Core Training sessions and then continue with
      monthly facilitation meetings for one year to promote MTFS-I use and progress toward FBS
      improvement.

      The study will feature a three-group cluster randomized trial testing Core Training Only
      versus Core + Facilitation versus no-intervention Control in 9 ASU clinics across New York
      State. MTFS-I utilization data will be collected from Core Training and Core + Facilitation
      sites for one-year follow-up (after initial Core Training); FBS quality data on family
      participation and family therapy technique use will be collected from all sites over one-year
      follow-up; observational coding methods will verify therapist reports of FBS quality; and
      client outcome data for all sites will be retrieved from administrative data warehouses over
      one-year baseline and one-year follow-up. These data will enable between-condition
      comparisons of FBS delivery (Aim 2: MTFS-I utilization, family participation, family therapy
      technique use) and client outcomes (Aim 3: therapeutic goal achievement, substance use
      change). If study aims are achieved, investigators would be positioned to mount a fully
      powered RCT to test the effectiveness of these FBS quality improvement procedures at scale.
    

Detailed description: 
      Discovering and disseminating effective methods to improve the quality of treatment services
      for adolescent substance use (ASU) within the national behavioral healthcare system is an
      urgent public health priority. Despite the strong legislative and policy focus on quality of
      care evident in the Children's Health Insurance Program Reauthorization Act of 2009 and the
      Affordable Care Act of 2010, recent comprehensive reports underscore the enduring "quality
      chasm" between behavioral treatments proven in controlled research versus those commonly
      practiced in usual care. ASU treatment quality in particular is considered mediocre to
      inadequate, due to a host of factors headlined by the absence or modest quality of
      evidence-based services, along with inadequate provider training, little quality monitoring,
      inattention to data-driven decision-making, and ineffective system-level policies for
      promoting existing quality mandates.

      Family-based services (FBS) are a prime candidate for upgrading the quality of the ASU
      treatment system. FBS comprise both family participation in services, the systemic parameters
      wherein family members are included in assessment and treatment activities; and family
      therapy techniques, the specific interventions that clinicians use to directly target family
      members and family functioning for change. FBS have reached the highest levels of empirical
      validation for ASU, posting an exemplary record of success in comparison to alternative
      evidence-based treatments as well as usual care, and FBS produce the largest average effect
      sizes by a large margin. Due in large part to this extensive evidence base, FBS have long
      been strongly endorsed by federal agencies, national associations, and policy-making groups.
      There is also incentive from ASU clinical providers and payers to deliver FBS, which are now
      approved for treating ASU and disruptive behavior disorders by federal and private insurance
      plans and regulatory agencies that govern licensed treatment providers. FBS are also widely
      endorsed by therapists treating youth in routine care. Thus there is impetus from all corners
      to expand FBS implementation in usual care for ASU.

      There are several well-documented barriers to dissemination of manualized FBS models,
      including the cost and complexity of delivering these models in everyday settings. In
      addition, for FBS to fulfill their potential to enhance the quality of ASU treatment in usual
      care, FBS implementation must be supported by effective quality assurance (QA) procedures
      designed to ensure that FBS are delivered with fidelity, that is, to the target population,
      by appropriately trained providers, and in accord with specified procedures. The time is
      propitious for developing such procedures in the current healthcare market, which is
      incentivized to establish reliable standards for quality care. First, there is growing demand
      for innovative quality indicators of behavioral treatment that assess appropriateness and
      potential effectiveness of care. Conventional quality indicators capture broad principles of
      behavioral care such as treatment assignment, retention and follow-up rates, referrals for
      ancillary care, and client safety. However, the emerging quality-of-care implementation
      framework advocates that fidelity to evidence-based treatments itself be considered a quality
      indicator. Second, in order to properly monitor treatment fidelity in usual care, there is
      urgent need to develop quality metrics that can reliably and pragmatically measure fidelity
      in front-line treatment settings. The implications of emerging scientific and policy mandates
      for improving treatment quality are clear with regard to FBS for ASU: Pragmatic QA procedures
      for ensuring high-fidelity FBS need to be developed, and these procedures need to be anchored
      by reliable FBS fidelity metrics.

      This study will develop pragmatic QA procedures designed to promote FBS adoption and quality
      in ASU treatment systems using a measurement feedback system (MFS). MFS is a performance
      feedback loop in which a given quality indicator is continuously monitored by the clinician
      to gauge case progress and support clinical decision-making. MFS feedback loops usually take
      the form of easy-to-digest data reports that provide summary appraisals of individual client
      progress on selected indicators in comparison to a desired benchmark. To date MFS has been
      used in mental health care to enable monitoring of client outcomes primarily-for example,
      therapists tracking weekly client-report depression scores compared to age-adjusted norms on
      a depression inventory. With adults, utilizing MFS has led to impressive gains in outcomes
      with diverse samples: preventing early treatment failure, reversing symptom deterioration,
      and enhancing overall outcomes. MFS research on youth is in its early stages but growing
      rapidly, with strong enthusiasm about reaping comparable benefits. Importantly, clinicians
      trained in MFS develop positive attitudes toward it.

      MFS successes for client outcomes have generated enthusiastic support for the value of
      developing complementary MFS procedures that provide measurement Training along with
      data-based Feedback on implementation: MTFS-I. Due to streamlined administration procedures
      and emerging evidence, MTFS-I has been labeled a promising QA strategy with broad
      dissemination potential for youth. MTFS-I has already been incorporated into manualized QA
      procedures to bolster fidelity to several varieties of manualized treatment; in a few of
      these instances, feedback reports contain data input from therapists on fidelity indicators.
      One study found that MTFS-I for clinician-reported session content during youth therapy
      increased the likelihood and rapidity of addressing that same content in future sessions. In
      keeping with primary goals of the R34 mechanism, this study will test a pragmatic MTFS-I
      designed to improve the quality of FBS implementation in ASU treatment sites. The MTFS-I will
      draw on technology from existing feedback systems, but also, be the first to target
      system-level FBS delivery for ASU. The four MTFS-I components, described in C4, are designed
      to increase the extensiveness (i.e., amount and frequency of) FBS implementation.
    

Intervention name(s): Core Training: FBS System Mapping and MTFS-I Installation; Core Training & Facilitation: FBS System Mapping and MTFS-I Installation plus monthly on-site meetings
Intervention type(s): Behavioral; Behavioral
Intervention description(s): Core Training will be a blended strategy wherein multiple interventions targeting multiple agency roles are combined to help ASU treatment providers increase their FBS quality. Core Training will be conducted with administrators, supervisors, data managers, and clinical staff. It will contain two 3-hour sessions focusing on FBS System Mapping and MTFS-I Installation. FBS System Mapping will focus on process mapping of existing site policies and practices for assessing family functioning at intake, educating families about intervention strategies and activities, including family members in treatment sessions, and hiring and supporting clinicians trained in and routinely delivering FBS. MTFS-Installation will focus on introducing MTFS-I components and integrating principles of DDDM.; Facilitation will consist of monthly on-site meetings to review progress toward FBS improvements, discuss MTFS-I feedback reports and online resource use, and provide intensive technical assistance for progressing toward site-selected FBS improvement goals. Meetings will involve a subset of site-delegated participants from Core Training sessions to serve as a local change team (quality improvement team). The efficacy of facilitation-guided change teams has been established. Although there are several models for creating change teams, at least three features are common: (1) members represent multiple organizational roles (supervisors, data managers, line staff); (2) focus on data-driven decisions; and (3) use of external expert advisors.
Allocation: Randomized
Intervention model: Parallel Assignment
Intervention model description: This study is a cluster randomized trial testing two additive implementation strategies for installing FBS quality improvement procedures in ASU outpatient treatment. Nine ASU treatment sites will be yoked and randomized into three conditions to enhance ecological validity; 1) Core Training Only; 2) Core Training and Facilitation; and 3) Control (no intervention). The implementation strategies (i.e., Core Training Only, Core + Facilitation), are system-level strategies for translating evidence-based interventions into multi-site service settings.
Masking: None (Open Label)
Primary purpose: Treatment
Keywords: Adolescents; Substance Use Disorders