# Presentation: 496671

## Four Views of a Depression Quality Improvement (QI) Intervention: 

- Qualitative Description, QI Findings, Economic Data and Trial Results
- QUERI National Meeting, Dec 2008

## What is TIDES?(Translating Initiatives in Depression into Effective Solutions)

- 1990’s:  Researchers tested depression care improvement models
- Collaborative care with care management necessary and sufficient (over 36 high quality randomized trials)
- 2000:  TIDES = Can VA implement collaborative care as part of routine care?
- VISN leadership were decision-makers
- 2006:  TIDES part of national VA rollout

## TIDES is An Evidence-Based Quality Improvement Project

- Design based on successive Plan-Do-Study-Act Cycles (PDSA)
- Starting point for PDSA’s is evidence from the literature
- VISN leaders decided on general program features through an expert panel process
- Regional and local team-based quality improvement process was flexible and variable

## Qualitative and Quantitative Evaluations of TIDES

- A series of projects with PI’s/CoPIs including
- Ed Chaney, Fen Liu (Seattle)
- JoAnn Kirchner, Rick Owen, Mona Ritchie (Little Rock)
- John Williams (Durham)
- Lisa Rubenstein, Becky Yano, Jackie Fickel, Susan Vivell (GLA)
- Clinical Leaders Randy Petzel, Mike Davies, Cathy Henderson, Clyde Parkis, Susan McCutcheon, Skye McDougal, Ken Clark, Ron Norby, and many others

## QUERI Water PIPELINE

- PIC, MHAP,
- MOOD,IMPACT,
- RESPECT, TEAM

**HSR**

**Implementation Science**

**Implementation Policy**

**A**
**C**
**C**
**O**
**U**
**N**
**T**
**A**
**B**
**I**
**L**
**I**
**T**
**Y**

**Depression**
**Care **
**Guidelines**

**Depression**
**Collaborative**
**Care Model**

**Revise Guidelines**

## This Session

- Presentations:
- Jackie Fickel—Initial TIDES sites at baseline
- Fen Liu---Key aspects of implementation and their costs
- Ed Chaney—The story of a randomized trial
- Lisa Rubenstein—TIDES QI data
- Discussion:
- Do different views of the elephant enrich understanding?
- Implications of TIDES for VA and for your work?

## Clinic-level Usual Care for Depression in Primary Care Settings

- Jacqueline Fickel, PhD, Elizabeth Yano, PhD, Louise Parker, PhD, Lisa Rubenstein, MD
- --VA Sepulveda HSR&D Center of Excellence
- QUERI National Meeting, December 11, 2008

## Clinic-level Assessment

- Usual care process
- Concordance with evidence-based guidelines
- Barriers

**Notes:**
Multi-component models target  clinic-level care processes 
provider education, 
patient education, 
patient self-management support,
IT/provider decision-support tools, 
mechanisms for PC-MH collaboration
care management

But, there is little documentation of clinic-level usual care—
Research studies usually describe aspects of the care process between individual patients and providers, or report barriers to care

Would be important to know usual care--  
Baseline for process evaluation
Comparison between QI intervention sites and “usual care” sites
Which aspects of concordance of usual care with guidelines are in greatest need of improvement?

## Methods

- Cross-sectional observation prior to a multi-site intervention:  collaborative care model for depression in primary care clinics
- Case comparison strategy
- 10 VA primary care clinics, from 3 networks
- In-depth, semi-structured interviews with 22 clinical leaders
- Supplemental descriptive, organizational structure data, for context

**Notes:**
Comparative case study:  
Common patterns across sites
Particularities

3 VISNs, across 5 states: 
SE/Gulf Coast, Upper Midwest/Great Lakes, Northern Great Plains
Large (10,000-13,000) & small (3900-7700)
Metro (6) & rural (4)

Interview Data:  
Qualitative descriptions of the care process from key informant interviews with clinical leaders from each clinic (10 PC and 12 MH)  (November 2001-May 2003)

Content analysis—categorize responses and describe key processes of care  
(Crabtree & Miller 1999, Ryan & Bernard 2003)

Matrix analysis—summarize descriptors and synthesize into case profiles for each clinic
(Miles & Huberman 1994)

## Results: General similarities across sites, but also particular processes specific to each clinic

- Usual care processes
- 3 clinics rarely diagnosed or treated depression
- Differences from guidelines
- Assessment limited, informal diagnostic methods and decisions for locus of care, treatment options mostly pharmacologic
- Barriers to appropriate management of patients with depression in PC
- Zero to six barriers cited per clinic

**Notes:**
General similarities across sites:
Screening regularly; diagnosing informally; treating in PC according to pt severity and provider comfort level; consultation and referral via electronic consult system, paper & phone

Specific variations were seen in all steps of care: screening, assessment, diagnosis, treatment, and follow/up         (“usual care” is not a defined standard)

Major exceptions to typical care process were the clinics that were rarely diagnosing (3) or treating (4)

Varied from guidelines to different degrees in different clinics

Most common barriers:  inadequate time/# of PCPs and inadequate PCP training in MH

Conformance to practice guidelines could be viewed as a continuum, rather than an all-or-nothing situation

[Schulberg et al 1998. Update of AHCPR Practice Guidelines.   VHA/DOD 2000 Clinical Practice Guidelines2.0]

## How research team can work with clinic staff

- Care process, especially whether already diagnosing and/or treating depression in primary care
- Awareness of barriers
- Nature of barriers

**Notes:**
Results suggest that individual practices will have specific needs for targeting QI activities and tailoring interventions to local context

## Closest of typical care process to guidelines + Typical barriers

| Clinic | Differences from Guidelines | Barriers reported |
| --- | --- | --- |
| C1 | PCP may do some limited assessment
Informal diagnosis
Informal decision to treat in PC
Pharmacotherapy, with informal supportive counseling
 | Inadequate time/number of PCPs
Inadequate MH training for PCPs
Patient reluctance/stigma
 |

**Notes:**
Closest to guidelines of usual process described:     (two other “typical” clinics, A3, C2)

C1 (C2a)      Screening regularly, CPRS-based process for limited assessment, assigned provisional diagnosis, did limited tx planning, offered informal supportive counseling as well as meds, follow-ed up to check symptoms & adjust tx prn, and communicated with MH for continuation & maintenance.
Reported 3 common barriers.

How researchers can work with clinic staff:  A multi-faceted, evidence-based strategy like collaborative care could help address barriers via DCM or co-located MH provider, and PCP depression education

## Most variations from typical care process + Unusual barriers

| Clinic | Differences from Guidelines | Barriers reported |
| --- | --- | --- |
| A1
 | Unclear process for clinical assessment
Rarely diagnose depression
PCPs rarely treat
 | Inadequate time/number of PCPs
Institutional policy barriers
Local culture/turf issues
 |

**Notes:**
Most distant from guidelines: 
3 clinics rarely diagnosed or treated depression  [A1 (A1a), B3 (B1c), C3 (C1a)]

Each with different profile of barriers:
A1 (A1a)    included barriers related to institutional policy within the clinic & HCS, plus legacy of local culture & turf issues.
How researchers can work with clinic staff:  Simply trying to implement a collaborative care model here without addressing the systemic issues probably wouldn’t move the practice a great deal closer to guideline recommendations

## Identified no barriers to PC management of depression

| Clinic | Differences from Guidelines | Barriers reported |
| --- | --- | --- |
| B1 | PCP may do some limited assessment
Informal diagnosis
PCPs rarely treat | none
 |
| C2
 | Unclear process for clinical assessment
Informal diagnosis
Informal decision to treat in PC
Pharmacotherapy, with informal supportive counseling
Little mention of monitoring, continuation or maintenance | none
 |

**Notes:**
2 clinics with a number of variations from guideline-concordant practice, yet  reported no barriers:
B1 (B1b), C2 (C1b)

How researchers can work with clinic staff:   May lack awareness of the need for change, and need further pre-implementation assessment, education on guidelines, or other actions to better understand the appropriate strategy(ies) for improving concordance.

## Implications

- Define individual clinics’ situations prior to QI interventions
- Insights regarding how to tailor interventions
- Keep in mind for insights on interpretation of evaluation results

**Notes:**
Evidence indicates that quality of care improvement for depression requires changes in care delivery at clinic or practice group level.

Quality of care evaluation has traditionally looked at the individual patient or provider level.

Can miss important aspects of organization structures and relationships that can influence success of practice-level interventions

Local autonomy in adapting care models and guidelines, versus disseminating standardized, evidence-based models
     balance central guidance for local-level stakeholders in tailoring a collaborative care model to local needs

Assume that site-specific variations are present, and consider how to respond.  Not treating usual care as a black box

## Organizational Cost of Quality Improvement for Depression 

- Chuan-Fen Liu, PhD MPH
- VA Norwest HSR&D Center of Excellence
- December 11, 2008

## Objectives

- Document implementation timelines
- Estimate organization cost of implementing TIDES models

## Study Design

- Cross-sectional descriptive analysis from April 2000  to June 2004
- Study Sites: 7 TIDES intervention clinics
- QI participants: clinical and technical expert support teams
- Activities included: communications, meetings, protocols and materials development, training, and other activities to facilitate the process
- Activities excluded: usual patient care, evaluation and research, Human Subjects Review
- Data Sources: project records and surveys

## Slide 19

## Implementation Process by Region and Practice

## Preparation and Design Phases

| Phase | Months | Range |
| --- | --- | --- |
| Preparation | 9 |   6-11 |
| Design | 17 | 12-30 |
|     Basic design |   3 |     1-7 |
|     Practice engagement | 14 |   6-21 |
| Preparation + Design | 27 | 23-34 |

-  The length of design phase decreased with the order of initiation across VISNs: 20 months; 19 months; 12 months

## QI Participants, Person Hours and  Costs

| 
Participants | 
Person Counts | 
Person Hours | 
Total Costs |
| --- | --- | --- | --- |
| Technical Expert Team | 
42 (33%) | 
2,147 (66%) | 
$197,787 (70%) |
| Clinical Partners | 
86 (67%) | 
1,086 (34%)
 | 
$84,438 (30%) |
| 
All Participants | 
128 (100%) | 
3,233 (100%) | 
$282,225 (100%) |

## Person Hours and Cost by Phase

|   | Clinical Team |  | Technical Expert Team |  |
| --- | --- | --- | --- | --- |
| Project Phase | Person Hours
(1,085) | Cost
($84,439) | Person Hours
(2,147) | Cost
($197,787 ) |
| Preparation and design  | 68% | 73% | 76% | 89% |
| Implementation  |  |  |  |  |
|     PDSA cycle | 17% | 14% | 13% | 6% |
|     Routine care  | 15% | 13% | 11% | 5% |

## Conclusions

- Clinical partners contributed more people, but less time
- As planned, clinical partners served as decision makers, the technical expert team carried out the majority of the labor
- Creation of a successful depression care model required significant resources

## Depression Quality Improvement Through the Lens of a Randomized Clinical Trial 

- Intent to Treat Versus Treating the IntentEdmund Chaney PhD Northwest HSR&D COE
- QUERI National Meeting, Dec 2008

## What was WAVES?(Well-being Among Veterans Enhancement Study)

- Site level randomized trial of TIDES
- In each of 3 early adopter VISNs two small to medium sized CBOCs were randomized to TIDES and one to usual care.
- Subject Recruitment:
- Population-based research telephone depression screening
- Identified consenting depressed patients were  referred to DCM at TIDES sites
- TIDES was implemented as QI at each site and was stable prior to WAVES referrals

## WAVES Design

- Designed to test treatment adherence
- Antidepressant use and psychotherapy visits assessed based on administrative data at 7 months
- Health outcomes were assessed by survey at 7 months (72% completion)

## WAVES Research Patient Sampling

## WAVES Results

- Significantly increased antidepressant use, adherence
- Non-significant improvement in overall appropriate care
- Adherent to medications (MPR > .8) or at least 4 MHS visits
- No significant improvement in depression symptoms or functional status

## Effect of Implementing TIDESon Treatment at 7 months

## What Happened?  Trials vs. QI

- Delayed start due to IRB: TIDES close to capacity when WAVES started
- Plan: TIDES Plan-Do-Study-Act cycles with no more than 50 patients prior to WAVES start
- Primary care provider-referred patients “competed” with research nurse-referred patients
- Research patients received lower priority
- Primary care provider experience level affected effectiveness of the model
- Few WAVES PCP’s were experienced

## WAVES Referral Outcome (Triage) 

- No Assessment38%(n=141)

**Refused****treatment****41%****(n=58)**

**Mean DCM ****Contacts****1.3****57%****(n=80)****SD=0.5**

**MH ****Mean DCM ****Contacts****1.8**** (n=80)****SD=0.7**

- Note: MH patients are those who were in Mental Health Care when referred to TIDES.

**WAVES Referral Outcome (Triage)**

## Treating the Intent: Primary Care Provider TIDES Experience

## What Happened:  Regional Differences

- One of three VISNs had a weaker program
- Patient health and adherence outcomes significantly worse for this VISN (p < .05)
- Little VISN Director level back-up
- Care manager turnover with inadequate replacement

## Lessons from WAVES: Trials May Not Reflect Real World Implementation 

- Real world interventions are implemented through some type of quality improvement method
- Interventions only sustain if integrated into organization’s real world activities
- Difficult to use trials to study QI implementation without distortion
- Trial can’t capture some strong determinants of real world program functioning
- E.g., primary care provider preferences and experience determine use of the treatment model

## Lessons from WAVES:  Trials Reflect Outcome Determinants 

- The outcome impacts of weakness in one of three VISNS
- Shows program is sensitive to leadership &  turnover
- Effects of TIDES on a representative sample
- Shows importance of assessing who is getting the program
- Can enable fine-tuning of factors such as  provider engagement

## What, When, Why and How are Essential for Implementation Research

- SQUIRE criteria codify reporting
- Multiple methods are necessary
- E.g., TIDES QI Registry, site readiness data helped to interpret WAVES outcome
- Distinctions between QI Implementation and effectiveness research need to be better understood

## TIDES Quality Improvement Findings: Patient Care and Outcomes

- Lisa V. Rubenstein, MD, MSPH
- QUERI National Meeting, Dec 2008

## Purpose of TIDES QI Data

- Assess
- Patient safety and effectiveness for those referred to the model
- Adherence to evidence-based DCM model
- Focus for improvement
- Assess quality variations across clinics
- Develop TIDES performance measures

## What is This Data?

- Many possible names
- Formative evaluation
- Quality improvement
- Patient safety
- What it isn’t:  Comparative Effectiveness

## Methods

- Care managers trained to
- Administer PHQ-9 in standardized way
- Record panel management & results on a log
- Transfer log data to Excel
- Excel spreadsheets (de-identified) sent to research team for analysis
- Spans 1/2002 to 9/2004 (21 months)
- Crosses many PDSA cycles

## Results #1:  Enrollment & F/U

- Among 858 pts referred to TIDES, 738 (87%) were eligible for initial assessment
- 24 refused (2.8%)
- 86 could not be contacted (10%)
- 10 misc. (e.g., too sick) (1%)
- Of 738 eligibles, 86% (636) got a full initial assessment by the DCM
- 81% of these (516) completed 6 mos.PHQ f/u

## #2: Depression Symptom Outcomes 

- PHQ 9 _>_ 10 = probable major depression
- PHQ 9 < 5 = probably no significant depression
- Baseline PHQ 9 mean = 12.2
- F/u PHQ 9 at 24 weeks = 7.4
- ___________________________________
- Mean pre-post drop in PHQ 9 = 5.8 points

## #3: Mean PHQ Change: Referral Type (TIDES vs WAVES)

|  | Baseline
Mean PHQ | 6 Mos 
Mean PHQ |
| --- | --- | --- |
| TIDES (n = 310) | 12.1 | 5.8 |
| WAVES (n = 206) | 14.0 | 9.6 |

- P=.002, controlling for baseline PHQ, VISN, pt complexity, interactions
- Interactions not significant (VISN, pt complexity)

## #4:  Mean PHQ Change: VISNs (includes TIDES and WAVES pts)

| VISN | Baseline
Mean PHQ | 6 Mos 
Mean PHQ |
| --- | --- | --- |
| #1 | 12.6 | 6.1 |
| #2 | 12.7 | 7.5 |
| #3 | 13.4 | 9.17 |

- P=.02, controlling for referral type, pt complexity, int.
- No significant interactions (referral type, pt complexity)

## #5:Contact New Referrals Promptly

- 81% enrolled within 1 month (48% in 1 wk)
- TIDES referral: 	81 % enrolled in 1 month
- WAVES referral: 	74% enrolled in 1 month
- VISN #1: 91%
- VISN #2:  81%
- VISN #3: 78%

## #6: Enrollment and f/u calls

- Mean enrollment calls/pt = 3.7
- TIDES and WAVES were equivalent
- Proportion of pts completing adequate TIDES care (at least 4 treatment contacts) was significantly lower for WAVES-referred pts (p = .01)
- Pts receiving at least 4 calls had significantly better outcomes (p <  .05)

## TIDES QI Formative Findings: Reach

- Reach:
- Clinics varied from 46% to 100% of clinicians who referred to TIDES at least once; 0% to 36% who referred at least 10 pts
- Sustainability:
- Least ready site performed, but did not sustain (in a favorable VISN)
- Most ready site did not perform or sustain (in an unfavorable VISN)

## Could We Have Predicted Success?

- VISN effects seemed to trump individual site effects (positive and negative)
- “Most ready” site from Fickel data was in the lowest performing VISN
- “Least ready” site was in the best performing VISN
- Care managers were at the VISN level
- Lowest VISN leadership support linked to lowest care manager support

## Conclusions

- TIDES was safe and effective for the majority of those referred to it
- TIDES model performance echoes literature
- TIDES QI measures show progress and room for improvement
- QI measures for evidence-based depression care models is critical
- Assessment of site and VISN readiness may help future TIDES-Type implementation

## QUESTIONS FOR DISCUSSION

- Do these four different views of TIDES enrich your understanding of the program?
- What are the implications of what you’ve heard for your own work