VA Ann Arbor Center for Clinical Management Res

& VA Quality Enhancement Research Initiativ

Comparative Effectiveness of

Collaborative Collaborative Chronic Chronic Care Model

Mental Health Conditions: System

Review, Meta‐Analysis &

Implementation Strategies

Mark S. Bauer, MD

VA Center for Organization, Leadership, & Manage

Research (COLMR) (COLMR)

Amy M. Kilbourne, PhD, MPH
Disclosures

Royalties (~$300/year):Springer Publishing (MSB)Structured Group Psychotherapy for Bipolar Disorder:The Life Goals Program[therapist manual]New Harbinger Harbinger (MSB, AMK)Overcoming Bipolar Disorders[patient [patient self‐guided workbook] workbook]
Cyberseminar Outline

1. Meta‐Analysis of Collaborative Chronic Care

Models (CCMs) for Mental Health Conditio

2. The Bipolar CCM: Implementation Strategie

3. Audience Discussion*Woltmann E, Grogan‐Kaylor A, Perron B, Georges H, Kilbourne A

MS. Comparative effectiveness of collaborative chronic care 

mental health conditions conditions across primary, primary, specialty, specialty, and beha

beha

health settings. Systematic review and meta‐analysis. Amer 

Psychiatry 2012; 169:790‐804
-

t t t

Defining

Collaborative Collaborative Chronic Chronic Care Mo

• Goal: Evidence Evidence -based, anticipatory, based, anticipatory, 

continuous, collaborative care 

• CCM Elements CCM Elements: 

• Practice redesign• Patient self Patient self -management support management support • Expert systems (on-site, guidelines) • Information systems Information systems • Community linkages • L d ea ershi p /organ izational commitment 

Wagner & Von Korff 1996&1997; C
CCM Elements

Evidence‐Based, Planned Planned Care via

PracticeRedesign

PatientEducation

ExpertSystems

InformationMgt.

•Scheduling•Work Roles•Follow‐Up

•Self‐Mgt•Beh’l Change•Collaboration•SharedDecision‐Making

•Providereducation•Decision

Support•Specialty

Consultation

•Registry•Reminders•Outcome

tracking•Feedback

•Care Plans

Key Concepts: Anticipatory, Population-Based, Partnere

(after Wagner 1996, Von Korff 1997; Bode
CCMs: The RCT Evidence

• Diabetes • A th s ma 

• Arthritis • Congestive heart failure 

• F il ld l Frail elderly 

• Depression treated in primary ca• Growing evidence base in broade

groups, e.g., bipolar disorder, angroups, e.g., bipolar disorder, an

disorders 
Systematic Review & Meta‐ana

Goals & Ch ll a enges

• Central Question Central Question: How broad & : How broad & 

consistent are CCM effects on me

health outcomes? health outcomes? 

• Challenges: 

• Defining CCMs• Multiple populations, multiple outcome d

multiple articles from some trials, multiplanalyses in single article

• Long‐term trials with chronic outcomes 

complex, adjusted/modeled analyses
excluded

CCM Definition

• > 3 of the 6 Wagner‐Von Korff criteria criteria...

...whether or not “W‐VK lineage” is cit

• Mobile treatment team interventions

excluded

• Inter‐rater reliability:

• CCM yes/no: Kappa = 1.00

• CCM component count: ICC = 0.93
-

t t t

• CCM Elements: CCM Elements: 

• Practice redesign • Patient self Patient self -management support management support 

• Expert systems (on-site, guidelines) 

• Information systems Information systems 

• Community linkages 

• L d hi / i i l i Leadership / organizational commitment 

Defining

Collaborative Collaborative Chronic Chronic Care Mo

• Goal: Evidence Evidence -based, anticipatory, based, anticipatory, 

continuous, collaborative care 

Wagner & Von Korff 1996&1997; C
For included RCTs also extract:

Hierarchical review schema

Outcome Domains

• MH symptoms symptoms (depression (depression, mania, a

a

suicidality, etc.)

• Mental Quality of Life

• For included RCTs, also extract:

• Overall Quality of Life

• Social role function

• Physical Quality of Life

• Costs
Hierarchical Review Schema

Specifying Specifying Outcome Outcome Analyses Analyses to A

A

• One analysis analysis per outcome outcome domain• Whole sample > subsample• Only most global outcome (e.g.: depressio• Longest follow‐up interval

• Continuous > categorical variable

• Unadjusted Unadjusted > adjusted adjusted analyses analyses

• Whether or not primary outcome variable
Consolidating longitudinal data i

single meta‐analytic value

Simon et al, Arch Ge
Systematic Review

Summary of PRISMA Diagram

Articles Reviewed: 2,114

Full‐Text Reviews: 272

Disorder RCTs A ti r cles 

Systematic Review Totals 57 78 

Meta‐Analysis Totals 30 28 

Depressive Disorders 40 60 

Bipolar Disorders 4 4 

Anxiety Disorders 3 4 

Other/Multiple C diti on ons 10 10 
ID ES (95% (95% CI) 

Unutzer (2002) 0.60 (0.50, 0.69) 0.60 (0.50, 0.69) Liu (2003) Swindle (2003) Datto (2003) Oslin (2003) 

0.07 (-0.14, 0.28)0.19 (-0.08, 0.45)0.42 (-0.14, 0.98)0.63 (0.23, 1.04)0.07 (-0.14, 0.28) 0.19 (-0.08, 0.45) 0.42 (-0.14, 0.98) 

0.63 (0.23, 1.04) 

Smith (2006) Bruce (2004) Asarnow (2005) 

( ) 

Callahan (2006) 

0 15 ( 0 13 0 42)0.08 (-0.11, 0.28)0.19 (-0.02, 0.40)

( , )

0.50 (0.18, 0.83)

0 15 ( 0 13 0 42) 0.08 (-0.11, 0.28) 0.19 (-0.02, 0.40) 

( , ) 

0.50 (0.18, 0.83) 

Richards (2007) Smith (2006) Kilbourne (2008) Ross (2008) 

0.20 (-0.27, 0.68)0.15 (-0.13, 0.42)0.00 (-0.52, 0.52)0.06 (-0.25, 0.37)

0.20 (-0.27, 0.68) 0.15 (-0.13, 0.42) 0.00 (-0.52, 0.52) 0.06 (-0.25, 0.37) 

NOTE: Weights are from random effects analysis Overall (I-squared = 79.4%, p = 0.000) Davidson (2010) Kroenke (2009) 

0.31 (0.16, 0.47)0.48 (0.17, 0.80)0.77 (0.51, 1.03)

0.31 (0.16, 0.47) 0.48 (0.17, 0.80) 0.77 (0.51, 1.03) 

NOTE: Weights are from random effects analysis 

-1 -.5 0 .5 1

Comparison is better CCM is better 

Meta-Analysis of Reductions in Depress

Study 
ID ES (95% CC I) 

A (2005) 0 15 ( 0 04 0 34) 0 15 ( 0 04 0 34) Kilbourne (2008) Asarnow (2005) Richards (2007) 

0.40 (-0.12, 0.92)0.15 (-0.04, 0.34)

0.40 (-0.08, 0.88)

0.40 (-0.12, 0.92) 0.15 (-0.04, 0.34) 0.40 (-0.08, 0.88) 

Overall ( I-squared = 0.0%, p = 0.610) Ross (2008) 

0.18 (0.03, 0.33)0.10 (-0.21, 0.41)0.18 (0.03, 0.33) 0.10 (-0.21, 0.41) 

NOTE: Weights are from random effects analysis 

-1 -.5 0 .5 1 

Comparison is better CCM is better 

Meta Analysis of Mental QOL Meta-Analysis of Mental QOL 

Study 
CI

Druss (2001) Richards (2007) 

0.66 (0.29, 1.02)0.03 (-0.45, 0.510.66 (0.29, 1.02) 0.03 (-0.45, 0.51

K k (2009) Kilbourne (2008) Ross (2008) 

0 46 (0 21 0 1)0.25 (-0.27, 0.770.30 (-0.01, 0.610 46 (0 21 0 1) 0.25 (-0.27, 0.770.30 (-0.01, 0.61

Overall (I-squared = 31.0%, p = 0.203) Kroenke (2009) 

Druss (2010) 

0.33 (0.17, 0.49)0.46 (0.21, 0.71)

0.21 (0.01, 0.40)0.33 (0.17, 0.49) 0.46 (0.21, 0.71) 0.21 (0.01, 0.40) 

NOTE: Weights are from random effects analysis 

-1 -.5 0 .5 1

Comparison is better CCM is better 

Meta-Analysis of Physical QOL

Study ID ES (95% CI) 
Van Orden (2009) 

Unutzer (2002) 

0.00 (-0.38, 0.38)0.26 (0.17, 0.36)0.00 (-0.38, 0.38) 0.26 (0.17, 0.36) 

Overall (I-squared = 41.8%, p = 0.190) 

( ) 

0.20 (-0.02, 0.42)

( , )

0.20 (-0.02, 0.42) 

( , ) 

NOTE: Weights are from random effects analysis 

-1 -.5 0 .5 1 

Comparison is better CCM is better 

Meta Analysis of Overall QOL Meta-Analysis of Overall QOL 

Study 

ID ES (9(9 5% CI CI ) 
Druss (2001) 0.11 (-0.25, 0.47) 

0.11 (-0.25, 0.47) 

Overall (I squared = 0 0% p = 0 889) Van Orden (2009) 

0 09 ( 0 17 0 35)0.07 (-0.30, 0.45)0 09 ( 0 17 0 35) 0.07 (-0.30, 0.45) 

NOTE: Weights are from random effects analysis Overall (I-squared = 0.0%, p = 0.889) 0.09(-0.17, 0.35) 

35) 

-1 -.5 0 .5 1

Comparison is better CCM is better

Meta-Analysis of Global Mental Health

Study ID ES (9(9 5% CC I) 
Kroenke-2009 () 0.21 (-0.04, 0.45

0.21 (-0.04, 0.45

Kilbourne (2008) Unutzer (2002) 

-0.18 (-0.70, 0.330.34 (0.25, 0.43)-0.18 (-0.70, 0.330.34 (0.25, 0.43) 

NOTE: Weights are from random effects analysis Overall (I-squared = 56.2%, p = 0.102) 0.23 (0.02, 0.44) 

0.23 (0.02, 0.44) 

NOTE: Weights are from random effects analysis 

-1 -.5 0 .5 1

Comparison is better CCM is better 

Meta-Analysis of Social Role Function

Study ID ES (9(9 5% CI) 
t t

Meta-Analysis of Clinical Outcomes

Study %ID ES (9(9 5% CI) WeightReductions in Depression Unutzer (2002) Swindle (2003) Liu (2003) Oslin (2003) Datto (2003) Bruce (2004) Asarnow (2005) Callahan (2006) Smith (2006) 

0.60 (0.50, 0.69)0.19 (-0.08, 0.45)0.07 (-0.14, 0.28)0.63 (0.23, 1.04)0.42 (-0.14, 0.98)0.08 (-0.11, 0.28)0.19 (-0.02, 0.40)0.50 (0.18, 0.83)0 15 ( 0 13 0 42)

9.757.768.535.884.298.728.496.987 65

0.60 (0.50, 0.69) 0.19 (-0.08, 0.45) 0.07 (-0.14, 0.28) 0.63 (0.23, 1.04) 0.42 (-0.14, 0.98) 0.08 (-0.11, 0.28) 0.19 (-0.02, 0.40) 0.50 (0.18, 0.83) 0 15 ( 0 13 0 42) 

9.75 7.76 8.53 5.88 4.29 8.72 8.49 6.98 7 65 

. Smith (2006) Richards (2007) Kilbourne (2008) Ross (2008) Kroenke (2009) Davidson (2010) Subtotal (I-squared = 79.4%, p = 0.000) Mental QOL Druss (2001) Asarnow (2005) 

0.15 (-0.13, 0.42)

0.20 (-0.27, 0.68)0.00 (-0.52, 0.52)0.06 (-0.25, 0.37)0.77 (0.51, 1.03)0.48 (0.17, 0.80)0.31 (0.16, 0.47)-0.15 (-0.51, 0.21)0 15 ( 0 04 0 34)

7.655.154.717.167.887.06100.0013.6226 96

0.15 (-0.13, 0.42) 

0.20 (-0.27, 0.68) 0.00 (-0.52, 0.52) 0.06 (-0.25, 0.37) 0.77 (0.51, 1.03) 0.48 (0.17, 0.80) 0.31 (0.16, 0.47) -0.15 (-0.51, 0.21) 0 15 ( 0 04 0 34) 

7.65 5.15 4.71 7.16 7.88 7.06 100.00 13.62 26 96 

. Asarnow (2005) Richards (2007) Ross (2008) Kilbourne (2008) Druss (2010) Subtotal (I-squared = 40.1%, p = 0.138) Physical QOL Druss (2001) Richards (2007) Ross (2008) 

0.15 (-0.04, 0.34)

0.40 (-0.08, 0.88)0.10 (-0.21, 0.41)0.40 (-0.12, 0.92)0.37 (0.17, 0.57)0.20 (0.04, 0.36)0.66 (0.29, 1.02)0.03 (-0.45, 0.51)0 30 ( 0 01 0 61)

26.968.6716.537.6826.53100.0013.598.9017 29

0.15 (-0.04, 0.34) 

0.40 (-0.08, 0.88) 0.10 (-0.21, 0.41) 0.40 (-0.12, 0.92) 0.37 (0.17, 0.57) 0.20 (0.04, 0.36) 0.66 (0.29, 1.02) 0.03 (-0.45, 0.51) 0 30 ( 0 01 0 61) 

26.96 8.67 16.53 7.68 26.53 100.00 13.59 8.90 17 29 

. Ross (2008) Kilbourne (2008) Kroenke (2009) Druss (2010) Subtotal (I-squared = 31.0%, p = 0.203) Overall QOL Unutzer (2002) Van Orden ( 2009) Subtotal (I-squared = 41.8%, p = 0.190) 

0.30 (-0.01, 0.61)

0.25 (-0.27, 0.77)0.46 (0.21, 0.71)0.21 (0.01, 0.40)0.33 (0.17, 0.49)0.26 (0.17, 0.36)0.00 (-0.38, 0.38)0.20 (-0.02, 0.42)

17.297.7922.7329.71100.0075.7424.26100.00

0.30 (-0.01, 0.61) 

0.25 (-0.27, 0.77) 0.46 (0.21, 0.71) 0.21 (0.01, 0.40) 0.33 (0.17, 0.49) 0.26 (0.17, 0.36) 0.00 (-0.38, 0.38) 0.20 (-0.02, 0.42) 

17.29 7.79 22.73 29.71 100.00 75.74 24.26 100.00 

. . Global Mental Health Druss (2001) Van Orden (2009) Subtotal (I-squared = 0.0%, p = 0.889) Social Role Function Kroenke-2009 () Unutzer (2002) Kilbourne (2008) S b l (I d 56 2% 0 102) 

0.11 (-0.25, 0.47)0.07 (-0.30, 0.45)0.09 (-0.17, 0.35)0.21 (-0.04, 0.45)0.34 (0.25, 0.43)-0.18 (-0.70, 0.33)0 23 (0 02 0 44)

52.8147.19100.0032.9154.0213.07100 00

0.11 (-0.25, 0.47) 0.07 (-0.30, 0.45) 0.09 (-0.17, 0.35) 0.21 (-0.04, 0.45) 0.34 (0.25, 0.43) -0.18 (-0.70, 0.33) 0 23 (0 02 0 44) 

52.81 47.19 100.00 32.91 54.02 13.07 100 00 

NOTE: Weights are from random effects analysis . Subtotal (I-squared = 56.2%, p = 0.102) 0.23 (0.02, 0.44) 100.00 

-1 -.5 0 .5 1

Comparison is better CCM is better
ID ES (( 95% CI) 

Simon (2000) 0 03 ( 0 18 0 23) 

0 03 ( 0 18 0 23) 

Simon (2001) Katon (2002) Simon (2000) Druss (2001) 

0.22 (0.01, 0.42)0.25 (-0.04, 0.54)0.03 (-0.18, 0.23)

-0.10 (-0.46, 0.25)0.22 (0.01, 0.42) 0.25 (-0.04, 0.54) 0.03 (-0.18, 0.23) 

-0.10 (-0.46, 0.25) 

Bauer (2006) Pyne (2003) Swindle (2003) Simon (2002) 

-0.05 (-0.27, 0.17)0.16 (-0.05, 0.36)0.10 (-0.16, 0.37)0.02 (-0.18, 0.23)-0.05 (-0.27, 0.17) 0.16 (-0.05, 0.36) 0.10 (-0.16, 0.37) 0.02 (-0.18, 0.23) 

Simon (2007) Katon (2006) Simon (2006) Unutzer (2008) 

-0.03 (-0.26, 0.19)-0.13 (-0.39, 0.13)0.20 (-0.01, 0.42)-0.13 (-0.35, 0.10)

-0.03 (-0.26, 0.19) -0.13 (-0.39, 0.13) 0.20 (-0.01, 0.42) -0.13 (-0.35, 0.10) 

NOTE: Weights are from random effects analysis Overall (I-squared = 16.5%, p = 0.278) Simon (2009) 

( ) 

0.05 (-0.02, 0.12)-0.01 (-0.20, 0.19)

( )

0.05 (-0.02, 0.12) -0.01 (-0.20, 0.19) 

( ) 

-1 -.5 0 .5 1 

Comparison Costs More CCM Costs More 

Meta-Analysis of Economic Outcomes 

Study 
Major Conclusions

• Meta‐analysis: significant clinical effe

all clinical clinical domains domains except n 2’ =s

• Across diagnostic groups• Across care settings• Economics: Improved outcome comes a

no net cost

• Systematic Systematic review: Similar Similar on broade

broade

of studies
Limitations

• Majority of studies on depression

• Most but not all of these from primary ca• However, Ho ver we , CCMs are no longer just “depres

depres

primary care” interventions

• Conservative Conservative bias toward negative negative resu

resu

• Primary, secondary, tertiary analyses inclu

inclusion inclusion of underpowered underpowered analyses analyses

• Meta‐analysis excludes adjusted outcome
State of the Art 

Next Steps

• Enhancing clinic‐based CCMs

• Moving beyond clinic walls

•Implementation strategies
Enhancing Clinic‐Based CCM

• E hn ance CCM for greater eff t ec s (e.g.,

depression in bipolar disorder)

• Identify & augment “active ingredients”

• Cross‐diagnosis considerations

• Physical as well as mental health focus
Moving Beyond Clinic Walls

• TeleHealth delivery

• VA National Bipolar Telehealth Progra

(begun 2011)

• “Supra”‐clinic level

• State‐level initiatives (e.g., DIAMOND i• Commercial insurers
Implementation Strategies

• Implementation science agenda

• Identifying optimal implementation strate

• Interaction with context (e.g., Medical Ho

• Example: The Bipolar Disorders CCM
The Bipolar CCM:

National National Practice Practice Guideline Guideline

Endorsements

• Canadian N t e work for M d oo & A i nx e

Treatments, 2006:

• Yatham et al, Bipolar Disord 2006; 8(6):7

• VA / DoD Clinical Practice Guidelines, 

• http://www.healthquality.va.gov/Mana

of

_Bi.asp

• SAMHSA National Registry of Evidenc

Programs Programs and Practices Practices (NREPP) (NREPP), in p

p

• http://nrepp.samhsa.gov/Index.aspx
Implementing CCMs

Recovery‐Oriented Collaborative Ca

(ROCC; R01 79994)

• Close the research‐to‐practice gap• CCMs not fully disseminated disseminated in real world

• Organizational barriers

• Lack of specificity, training for providers on impl

specific components

• Aligning Aligning organizational/financial organizational/financial incentives incentives

• Need specified implementation strategy to e

uptake and promote promote return‐on‐investment investment
ROCC G l oa s

• Apply enhanced version of CDC’s Replicating

Effective Programs (REP) framework to impl

bipolar CCM

• Implement CCM comparing Enhanced REP (a

customized facilitation) to standard REP incommunity‐based practices

• Hybrid Type II design

• Patient outcomes• Provider acceptance

• Return‐on‐investment (cost‐effectiveness of E‐R
Pre-implementation Implementation D

Identification of problem Identifying appropriate 

Orientation 

Cross-functional team 

Disseminate package 

FacS

intervention Package intervention 

Core elements 

Menu options 

Disseminate package Training Technical assistance Evaluation 

SB

Menu options 

Enhanced REP Conceptual Framew

Replicating Effective Programs (REP) was developed by CenDisease Control to rapy idl translate HIV prevention p rograms community-based settings, and based on Social Learning ThRogers’ Diffusion Model. Enhanced REP includes additional components based on the PARiHS framework: developing re

and promoting provider self-efficacy 

Kilbourne et al. Imp Science 2007; Kilbourne et al. Psychiatric Serv
-

Pre-implementation Implementation DIdentification of problem Identifying appropriate 

Orientation 

Cross-functional team 

Disseminate package 

FacS

intervention Package intervention 

Core elements 

Menu options 

Disseminate package Training Technical assistance Evaluation 

SB

Menu options 

Enhanced REP Conceptual Framew

Pre-implementationWhat do providers consumers need? (Pain points) What do providers, consumers need? (Pain points)What are the appropriate EBPs?Packaging: translation of manuals into user Packaging: translation of manuals into user -friendly core elem

friendly core elem

Flexibility (menu options) 
Pre-implementation Implementation D

Identification of problem Identifying appropriate 

Orientation 

Cross-functional team 

Disseminate package 

FacS

intervention Package intervention 

Core elements 

Menu options 

Training Technical assistance Facilitation 

SB

Menu options 

Enhanced REP Conceptual Framew

Implementation Cross-functional teams: buy functional teams: buy-in input in, input Training and orientation Technical assistance (6 months): regular meetings/calls Technical assistance (6 months): regular meetings/calls Facilitation: 6-12 months 
Pre-implementation Implementation D

Identification of problem Identifying appropriate 

Orientation 

Cross-functional team 

Package intervention 

FacS

intervention Package intervention 

Core elements 

Menu options 

Package intervention Training Technical assistance Evaluation 

SB

Menu options 

Enhanced REP Conceptual Framew

T h ec nology Transfer for Implementa

Dissemination: on beyond implementation Spread and sustainability Reimbursement models 
Enhanced Enhanced and Standard Standard REP

Enhanced REP Standa

Package Customized CCM package based on Community Working 

Group input Group input 

Standard package 

Training Customized training program via provider input for staff 

Site in Site in-services & rollout services & rollout 

Standard program & 

TA &Facilitation

Active calls to site by implementation expert with implementation expert with 

providers, leaders- facilitate organizational/financial incentives 

Technical 

as needed as needed 

Kilbourne, Neumann, Pincus et al: Imp. Science 2007
‐

Enhanced REP Facilitation

• Initiation and needs assessment• Relationship Relationship‐building building across providers providers• Benchmarking and feedback

• Adaptation and alignment with site nee• Ongoing Ongoing marketing marketing and sustainability sustainability

S l l 2006 Ki h F ili i M l Stetler et al 2006; Kirchner Facilitation Manual 
REP: Balancing Fidelity, Flexib

• Distilling intervention into Core Elements

• Theory, internal logic of program that achieves 

effect

• Example: self‐management = active discussion

benefits benefits of behavior behavior change

• Customize “menu options” – stakeholder i

• M do e by whi hc core element is d li e vered• Example: self‐management group, 1:1, phone

• Designed to enhance fidelity over time, bu
Enhanced REP

Facilitation

Component Mean Hours 

Enhanced RE

Initiate contacts/needs

assessment

22

Relationship building 112Benchmarking and review 15Adaptations/alignment 11Marketing and publicity 13Total 172 hrs (~$5,50
‐

Enhanced Enhanced REP: Future Directio

Directio

• B i us ness case for E hn anced REP/CCM REP/CCM

• ROCC Outcomes/CEA underway

• VA manualization of Enhanced REP/Fac

• VA SMI ReEngage ReEngage Initiative Initiative• Homeless HSR Initiative testing adaptive

implementation implementation strategy strategy using E‐REP

• MH QUERI Facilitation training (PC‐MHI, M
Implementation Roadmap

Earl “ y Lessons “Boots‐on‐the‐Ground

• Implementation skills akin to MI, sel f-efficacy • Build your research with the end-users in mi• Look for positive deviance (practice Look for positive deviance (practice -to -resea

resea

• Can’t spend your way to adoption • Measure and publicize success Measure and publicize success • Relationships are key: leverage win-wins