Building Outcomes With Observation-Based Supervision: An FFT Effectiveness Trial
Condition: Delinquency; Substance Abuse; Self Destructive Behavior
MeSH terms: Substance-Related Disorders; Self-Injurious Behavior
Study type: Interventional
Phase: Phase 2

Summary: 
      The proposed effectiveness study examines differences in treatment outcomes of an
      observation-based supervision (BOOST) versus supervision as usual (SAU). The study will be
      implemented within 16 teams delivering FFT services at 11 sites in the California Institute
      of Mental Health (CIMH) system. The 16 FFT therapist teams will be randomly assigned either
      to BOOST or SAU. Each team will have 3 therapists who will treat 6 families for a total of 18
      families per team. Thus, each condition will include 24 therapists who will treat 144
      families. The project will be implemented in four staggered waves to establish a more even
      rate of data collection and treatment implementation to enhance the feasibility of the study
      by keeping staffing and project costs more constant across the 5-year project. Each wave will
      involve 4 FFT teams, 2 receiving BOOST and 2 receiving SAU. Teams will be randomized to
      supervision conditions. Outcome assessments of parents and adolescents will be conducted at
      baseline and at 4 months and 16 months after treatment initiation.
    

Detailed description: 
      The proposed study is designed to evaluate two distinct approaches to therapist supervision
      in the implementation of Functional Family Therapy (FFT), an EBT for adolescent substance
      abuse and delinquent/behavior problems. The impetus for the proposed research emerged from a
      large scale dissemination study conducted by the Washington State Institute for Public
      Policy. The study identified two key findings: (1) Only half the therapists were actually
      implementing the FFT model in a competent manner, and (2) positive outcomes, i.e. reductions
      in felony recidivism, were evident only for therapists implementing the model competently.
      For the subset of competent FFT therapists, the financial benefits of the dissemination were
      estimated at $7.50 for each dollar of program cost. Based on the accumulated evidence
      supporting FFT, interest in disseminating the FFT treatment model has skyrocketed. Currently,
      the dissemination organization, FFT LLC, has trained more than 270 local, state, national and
      international organizations. Over 1100 FFT therapists have served over 12,000 families around
      the globe, with large scale multi-site implementations ongoing in California, Washington,
      Pennsylvania, New York, Florida, and the Netherlands.

      To address the key issue of enhancing treatment competence, FFT LLC recently developed and
      implemented a sophisticated web-based application designed to monitor highly structured FFT
      therapist progress notes, as well as supervisor and client ratings of therapist competence.
      The process helps to maximize sustainability for community programs by limiting costs. Hence,
      the supervision process involves feedback to therapists based on the progress notes and
      therapist-supervisor discussions of therapist performance. The impact of this supervision
      process on therapist competence and treatment outcomes is unknown. By contrast, Miller and
      his colleagues found that supervision involving active feedback and/or coaching based on
      supervisor review of therapy sessions to observe therapist behaviors directly results in
      improved model fidelity. Such observation-based supervision has been the hallmark in the
      development of family therapy models and in fidelity monitoring processes in efficacy trials
      evaluating family therapy and addictions treatments. By eliminating this practice, it is
      possible that FFT dissemination efforts may unwittingly be omitting one of the most important
      components needed for effective transfer of treatment.

      During this early period of research on dissemination, it is vital to identify the specific
      procedures most likely to maximize effective translation of efficacious treatments into
      community settings. Observation-based supervision of family therapy has played a critical
      role both in the development of clinical practices and in efficacy trials which have
      established positive effects on youth outcomes. Hence, a key first step in effectiveness
      research is to examine the extent to which supervision practices involving therapist report
      and observation of therapist behaviors during sessions influence therapist competence and
      subsequent treatment outcomes. The costs associated with supervision based on a direct review
      of therapy session recordings are often viewed as prohibitive for community implementation of
      EBTs. However, the potential sustainability of community implementations must be considered
      in the context of empirical evidence for the procedures in relation to outcomes achieved. If
      more intensive supervision practices were to enhance competence and produce superior
      outcomes, then economic analyses evaluating supervision cost, cost effectiveness, and
      cost-benefit would be essential. Such analyses would ultimately drive policy considerations
      with respect to funding EBTs at the community level to maximize the long-term financial and
      societal benefits.

      Study Aims

      The purpose of the proposed study is to examine the effects of observation-based supervision
      (BOOST) versus the standard supervision as usual (SAU) approach currently used by FFT LLC on
      therapist competence and adolescent outcomes in a community-based sample of adolescents
      receiving FFT. The study will be implemented with 16 FFT teams affiliated with the California
      Institute of Mental Health (CIMH), the coordinating center for stakeholders implementing FFT
      in community agencies in California. Of these 16 teams, 14 teams will be from 10
      community-based agencies affiliated with CIMH and 2 teams will be FFT teams within Los
      Angeles County Probation. Participants will include three therapists from each of the 16 FFT
      teams (n=48) and 6 adolescents/families per therapist (n=288). All of the adolescents
      referred to participate in the study by the FFT teams within Los Angeles County Probation
      will be youth who are "home on probation." In addition, some of the youth referred to the
      study by community-based organizations will include "home on probation" youths (e.g., Star
      View Behavioral Health) and some will not (e.g., Long Beach Adolescent Program). Similarly,
      youths referred by some community-based organizations to participate in the study may have
      child welfare involvement. Therapist teams will be randomly assigned to either BOOST or SAU.
      The aims of the study are as follows:

      Specific Aim 1: To examine differences between BOOST and SAU in improving treatment outcomes.
      Specifically, we hypothesize that BOOST will be associated with significantly greater
      reductions in adolescent substance use (Aim 1a) and conduct disturbance/delinquent behaviors
      (Aim 1b) than SAU.

      Specific Aim 2: To examine differences between BOOST and SAU in improving engagement and
      retention in treatment. Specifically, we hypothesize that BOOST will be associated with
      significantly higher rates of engagement and retention in treatment.

      Family therapy interventions are based largely on the premise that dysfunctional family
      processes play a central role in the development of adolescent problem behaviors such as
      substance use and delinquency. Family therapy is aimed at mitigating the dysfunctional
      processes. The success of family therapy in improving family functioning and youth outcomes,
      as noted above, depends on the fidelity of treatment implementation in general and on
      therapist competent adherence to the treatment model in particular. Thus:

      Specific Aim 3: To examine differences between BOOST and SAU in improving therapist
      competence and family functioning. Specifically, we hypothesize that BOOST will be associated
      with significantly greater rates of therapist competence (Aim 3a) and improvements in family
      functioning (Aim 3b) than SAU.

      Exploratory Aim 4: To explore potential mediating effects of therapist competence and family
      functioning on treatment outcomes. Specifically, we will explore whether therapist competence
      mediates (Aim 4a) the association between supervision condition and youth outcomes (substance
      use, conduct/delinquent behaviors), improvements in family functioning, and greater rates of
      engagement and retention in treatment. We also will explore whether improvements in family
      functioning (Aim 4b) mediate the link between supervision condition and youth outcomes
      (substance use, conduct/delinquent behaviors).
    

Intervention name(s): Supervision; Observational Supervision
Intervention type(s): Behavioral; Behavioral
Intervention description(s): SAU consists of a 1-hour weekly group supervision session with the clinical team (typically 3-6 therapists). The on-site supervisor also conducts individual supervision sessions with each member of the clinical team. As such, the usual FFT supervision practice involves each therapist receiving two hours of supervision per week (one hour of group, one hour of individual). Supervision sessions are conducted by an on-site supervisor who has received intensive training in FFT supervision. The on-site supervisor also continues to receive feedback and guidance from a national consultant (FFT LLC) for one hour per month.; One intent of BOOST is to provide ongoing development of therapist clinical skills, and identify novel approaches to unique family circumstances that are consistent with FFT treatment. BOOST involves the BOOST supervisor reviewing the recorded therapy sessions prior to supervision sessions with the therapist and on-site FFT supervisor and weekly group and/or individual supervision meetings during which the BOOST supervisor provides feedback and coaching to the therapist. The goal of observation-based supervision is to ensure feedback to therapists is based on the supervisors' direct judgments about therapist behaviors, model adherence, and delivery of FFT services rather than relying solely on therapist self-report of session activities and interactions.
Other intervention name(s): SAU; BOOST
Allocation: Randomized
Intervention model: Parallel Assignment
Masking: Double (Participant, Investigator)
Primary purpose: Treatment
Keywords: Recording; Supervision; Observation