A Systems+IT Approach to Accelerate Adoption of 

Depression Care in Safety-Net Diabetes Clinics: A 

Comparative Effectiveness Trial 

Shinyi Wu, Ph.D., Assistant Professor

Epstein Department of Industrial and Systems Engineering

September 19, 2013
SECTION TITLE | 2

ACKNOWLEDGEMENTS

• U.S. Department of Health and Human Services

• Los Angeles County Department of Health Services

• Study participating sites and patients

• DCAT team at USC, R&I of LACDHS, and 4PatientCare
• Objectives:

– Demonstrate a Systems+IT Approach to Accelerate 

Adoption of Depression Care Management

– Present findings of a comparative effectiveness trial 

“DCAT” to test the intervention

– Discuss prospects of Systems+IT Approach to deliver 

better and safer healthcare at lower cost

Overview
Poll Question #1

• What is your primary role in VA? 

– student, trainee, or fellow

– clinician

– researcher

– manager or policy-maker

– Other
Poll Question #2

• Which best describes your experience in 

health services research?

– I have not done health services research

– I am involved in health services research in VA 

settings 

– I am involved in health services research outside 

VA settings

– I am involved in health services research both 

outside and in VA settings 
Poll Question #3

• Which best describes your experience in 

depression care? [Select all that apply]

– My work is not related to depression care

– I am involved in primary care for people with 

depression

– I am part of the collaborative depression care team

– I am involved in mental health services 

– I am involved in research related to depression care
Adopted from http://www.healthshire.com/93-depression-quotes/

Suffering in Silence
Significance of Treating Depression

• Depression is a common comorbidity in primary care, 

affecting 16% of adults in the United States1

– impairs patient functional status

– worsens clinical outcomes

– increases healthcare cost

• The economic burden of depressive disorders is estimated to 

be $83.1 billion

• A significant public health challenge is sub-optimal 

identification and treatment of clinically significant depression 

among low-income minority patients.

2

Backgrounds
Depression and Diabetes

• The odds of comorbid depression in patients 

with Diabetes Mellitus (type 2 diabetes) 

doubles that in the non-diabetic population3.

• Among low-income minority diabetes 

population, 1 in 3 patients are suffering from 

major depressive disorder4.

Backgrounds
Evidence-based Depression Care 

Practices

• Screening adults for depression when staff-assisted 

depression care supports are in place (U.S. 

Preventive Services Task Force, 2009)

• An adaptive treatment approach with periodic 

monitoring is needed to find successful choices of 

antidepressant medication (American College of 

Physicians, 2008)5

• Team-based collaborative depression care model is 

effective6

Backgrounds
Barriers of Adopting and Sustaining 

Evidence-based Depression Care

• Patient barriers 

– Patient preference, cultural perceptions, and 

socio-economic barriers7

• Provider/organizational barriers8

– use of psychotherapy

– increased workload for clinic staff 

– delay in receiving outcomes data

– lack of resources to sustain the program

Backgrounds
Poll Question #4

• Which of the following is required for sustainable adoption of 

collaborative depression care by organizational decision-makers 

and providers [Select all that apply]

– Provider access to easily applied treatment guidelines

– treatment consistent with patient preferences

– technologies to facilitate information exchange between 

primary care physicians and depression care providers

– technologies to facilitate routine monitoring of patient 

depression symptoms and treatment adherence and 

satisfaction
A Healthcare Systems Engineering Approach

• Diabetes-Depression Care-management Adoption 

Technology (DCAT) is an innovative fully automated 

telephonic assessment and remote monitoring (ATA) 

call system 

– periodic screening and monitoring

– treatment adherence tracking

– relapse prevention

– identification of patient unmet needs

• The assessed data were integrated with an enhanced 

disease management registry (DMR) for automated 

provider notification, tasking, and emergency alerts 
The DCAT Schema
Automated Telephonic Assessment Call Modules

 PHQ (PHQ-2 or PHQ-9)

 Pain

 Activity 

 Psychotherapy

 Antidepressant Adherence

 Patient call request

DCAT Antidepressant Adherence Module
Yes

MRUN

Clinic

Patient name, DOB, Address

Patient telephone number, best call time and day, do not call time and day

Language (English, Spanish)

PIN selection (y/n), PIN number

DCAT enrollment date

Baseline PHQ-9 score

DCAT status (active, graduated, declined, dismissed, invalid phone#, deceased)

Data from Disease Management Registry (DMR) to intelligent 

interactive messaging solution system (updated daily)

Yes

DCAT Baseline PHQ-9 ≥8 (y/n)

DMR Clinician assessed PHQ-9 ≥8 in the past 6 months (y/n)

ASR Call PHQ-9 ≥8 in the past 6 months (y/n)

DCAT Baseline currently taking antidepressant (y/n)

DMR Prescribed antidepressant in the past 6 months (y/n)

Received psychotherapy in the past 6 months (y/n)

No to

ALL Qs

Yes to

at least

one item

DMR Currently prescribed antidepressant (y/n)

DCAT Enrollment

ASR Call Decision Table

ATA integrates patient clinical information to trigger calls
Yes

MRUN

Clinic

Patient name, DOB, Address

Patient telephone number, best call time and day, do not call time and day

Language (English, Spanish)

PIN selection (y/n), PIN number

DCAT enrollment date

Baseline PHQ-9 score

DCAT status (active, graduated, declined, dismissed, invalid phone#, deceased)

Data from Disease Management Registry (DMR) to intelligent 

interactive messaging solution system (updated daily)

Yes

DCAT Baseline PHQ-9 ≥8 (y/n)

DMR Clinician assessed PHQ-9 ≥8 in the past 6 months (y/n)

ASR Call PHQ-9 ≥8 in the past 6 months (y/n)

DCAT Baseline currently taking antidepressant (y/n)

DMR Prescribed antidepressant in the past 6 months (y/n)

Received psychotherapy in the past 6 months (y/n)

No to

ALL Qs

Yes to

at least

one item

DMR Currently prescribed antidepressant (y/n)

DCAT Enrollment

ASR Call Decision Table

ATA input includes patient preferences:

 

Patient telephone number, best call time and day, do not call time and day

Language (English, Spanish)

 

 

 

 

 

 

 Enrollm
ATA calls are patient-specific and dynamic
ATA calls notify providers to take actions 
Automated Telephonic Assessment System Design
DCAT Provider Task Examples
DCAT Alert System
Systems Engineering Principles
DCAT Implementation

• Quality assurance testing was conducted 

before and throughout the implementation of 

the system.

• Issues included

– Wrong assignment of tasks

– Missing log-in credentials for some providers

– Suboptimal documentation of task completion

• We added or modified features of the system 

based on feedback during implementation.
Implementation Results

• In the period 10/19/2011 – 1/9/2013

– 2899 outgoing calls were made

– 1100 tasks were generated

• The majority of tasks generated had been 

processed

– Average task processing time were longer than 

anticipated, most of the lengthy processing times 

occurred in the beginning of the study

• 32 alerts from 22 patients over 15 months
Feasibility Evaluation of DCAT

• During the initial 10 months of implementation 

• 51.6% calls are completed

• 79.7% patients had completed at least one assessment. 

• Assessment completion rate is not correlated with PHQ 

scores

• The system captured, on average, 98.5 percent of the voice 

or touchtone responses.

• Pairwise comparison showed PHQ scores from ATA calls are 

indifferent from those obtained by clinicians or recruiters.
Evaluate Effectiveness of DCAT

• A quasi-experimental comparative effectiveness trial was 

conducted

• Three programs are compared

– Usual Care (UC)

– Supported Care (SC): clinics with disease management program (DMP)

– Technology Care (TC): DMP+DCAT

• About 1400 patients with Diabetes were enrolled from 8 

primary care safety net clinics in the Los Angeles county

• The recruitment had a 92% acceptance rate. The majority 

(about 65%) were women and Spanish-speaking (about 83%). 

The baseline depression rate is 28.8%.
DCAT Comparative Effectiveness Trial

• Patients are assigned according to the locations they 

receive care
Methods: Data Collection 

• Data were collected for each ATA call and registry 

tasks

• All patients were interviewed at baseline, 6-, and 12-

months follow-up to assess depression symptoms, 

diabetes symptoms, comorbid conditions, health 

status, functional status, self-care behaviors, stress, 

and satisfaction with care

• Healthcare utilization data (clinic visits, ER visit, 

hospitalizations, prescriptions, and lab tests) were 

obtained from electronic medical records
Balance Check of Selected Baseline Variables

Arms (Exact or Mean Value) Group Comparison p-value

Characteristics UC SC TC

UC vs. 

SC

UC vs. 

TC

SC vs. 

TC

Female 63% 69% 59% 0.002 0.02 0.46

Age 55.04 52.09 52.57 <0.0001 <0.0001 0.42

Preferred Language –

Spanish 89% 78% 82% <0.001 0.003 0.16

Onset Age of Diabetes 45.01 41.84 42.23 <0.0001 <0.0001 0.57

On Insulin 26% 63% 63% <0.0001 <0.0001 0.84

SF Physical Score 43.04 45.81 43.95 0.0001 0.21 0.01

SF Mental Score 50.05 49.03 50.49 0.23 0.58 0.10

PHQ-9 6.67 6.93 6.37 0.50 0.43 0.17
An Extended PSM for Multiple Comparisons

STEP 1: Define Generalized 

Propensity Score

STEP 2: Estimate Generalized 

Propensity Scores

STEP 3: Predict Program 

Effects
Define Generalized Propensity Score

• Propensity score

– the probability that a patient is likely to receive treatment 

given the patient’s covariates

• Generalized Propensity Score9-13

– the conditional probability of receiving a particular 

treatment given the pre-treatment variables

• In DCAT study, the generalized propensity scores 

were interpreted as the probabilities that a patient is 

likely to be in UC, SC, or TC arms respectively given 

this patient’s baseline characteristics
Estimate Generalized Propensity Score

• All empirically important baseline variables 

that may influence outcomes were used in the 

estimation of generalized propensity score.

• In DCAT study, 25 clinically important baseline 

characteristics were used.

– 10 demographic variables such as age, gender, etc.

– 15 health status variables such as diabetes onset 

age, PHQ-9, chronic pain, etc.
Predict Program Effects

• 6 continuous and 4 binary outcomes were 

selected to evaluate effectiveness of DCAT

Outcome Description

Continuous

PHQ-9 Severity of depression

Diabetes Self Care Number of days per week of self care

SF Mental Score Overall mental health status

SF Physical Score Overall physical health status

Satisfaction in Healthcare Available to You for 

Your Diabetes

Overall care satisfaction

Satisfaction in Help Received with Emotional 

Problem

Overall mental care satisfaction

Binary

PHQ-9 ≥ 10 Threshold of clinical depression

≥50% PHQ Score Reduction Depression treatment effect

Satisfied to Very Satisfied in Health Care for 

Diabetes

Dichotomous indicator of care satisfaction for 

diabetes

Satisfied to Very Satisfied in Help Received with 

Emotional Problem

Dichotomous indicator of overall mental care 

satisfaction
Linear Regression Analysis Results

Outcome TC SC UC

PHQ-9 Score -1.34 

(0.01)

-0.88

(0.07)

Diabetes Self Care

SF Mental Score

SF Physical Score

Satisfaction in Healthcare

Available to You for Your 

Diabetes

0.24

(0.01)

Satisfaction in Help 

Received with Emotional 

Problem

0.25

(0.02)

0.38

(<0.01)

M1: Outcome_6_Mon ~ Arm + 

Outcome_Baseline + Team + Insulin_0

Outcome TC SC UC

PHQ-9 Score -1.38

(0.01)

-1.38

(0.01)

Diabetes Self Care

SF Mental Score

SF Physical Score

Satisfaction in Healthcare

Available to You for Your 

Diabetes

0.21

(0.03)

Satisfaction in Help 

Received with Emotional 

Problem

0.26

(0.02)

0.42

(<0.01)

M1_PS: model M1 plus propensity scores

Significant Improvement

at alpha=0.05

Insignificant 

Effect

Significant 

Negative Effect at 

alpha=0.05

Coefficient

(p-value)

Coefficient

(p-value)

Coefficient

(p-value)
Logistic Regression Analysis Results

M1: Outcome_6_Mon ~ Arm + 

Outcome_Baseline + Team + Insulin_0 M1_PS: model M1 plus propensity scores

Outcome TC SC UC

PHQ≥10 -1.04

(<0.01)

-0.48

(0.1)

≥50% PHQ Score 

Reduction

0.34

(0.11)

Satisfied to Very Satisfied 

in Health Care for 

Diabetes

1.04

(<0.01)

0.73

(0.01)

Satisfied to Very Satisfied 

in Help Received with 

Emotional Problem

0.51

(0.04)

0.94

(<0.01)

Outcome TC SC UC

PHQ ≥10 -1.11

(<0.01)

-0.76

(0.02)

≥50% PHQ Score 

Reduction

0.51

(0.03)

Satisfied to Very Satisfied 

in Health Care for 

Diabetes

1.01

(<0.01)

0.85

(0.01)

Satisfied to Very Satisfied 

in Help Received with 

Emotional Problem

0.56

(0.03)

0.97

(<0.01)

Significant Improvement

at alpha=0.05

Insignificant 

Effect

Significant 

Negative Effect at 

alpha=0.05

Coefficient

(p-value)

Coefficient

(p-value)

Coefficient

(p-value)
Summary of CER Results

• Both SC and TC significantly improve 

depression outcomes

• Only TC significantly improve overall patient 

satisfaction

• Neither SC nor TC were significant in 

improving diabetes self care

• Results are different by including propensity 

scores as variables in prediction model
Discussion

• DCAT demonstrates an example of applying Systems+IT

approach to facilitate adoption of depression care management 

in safety-net primary care settings

– Through patient ATA calls and enhanced disease registry for provider 

notification, tasking, and alerting

– Augment diabetes-depression care management
Discussion

• Is Systems+IT approach a promising solution for better care at 

lower cost?

– Many industries use technology to automate standardized protocols, 

producing highly reliable and easily scalable results that better achieve 

the consumer’s desired results. 

– However, healthcare has not yet benefited from this application of 

technology. 

– Automation of processes of care can improve quality and provide 

patient-centered solutions for chronic diseases in an efficient, 

economical manner, even as enrolled populations are set to expand 

dramatically

– More research is needed to understand, design, test, and evaluate 

Systems+IT approach 
Systems Approach to Health: 

A working definition14

A systems approach to health is one that applies 

scientific insights to understand the elements that 

influence health outcomes; models the relationships 

between those elements; and alters design, processes, 

or policies based on the resultant knowledge in order 

to produce better health at lower cost
Reference

1. Egede, L. E., Zheng, D., & Simpson, K. (2002). Comorbid depression is associated with increased health 

care use and expenditures in individuals with diabetes. Diabetes Care, 25(3), 464-470. 

2. Ani, C., Bazargan, M., Hindman, D., Bell, D., Rodriguez, M., & Baker, R. S. (2009). Comorbid chronic illness 

and the diagnosis and treatment of depression in safety net primary care settings. J Am Board Fam Med, 

22(2), 123-135. doi: 10.3122/jabfm.2009.02.080035

3. Anderson, R.J., Freedland, K.E., Clouse, R.E., Lustman, P.J. (2001). The prevalence of comorbid 

depression in adults with diabetes: A meta-analysis. Diabetes Care, 24(6), 1069-1078. doi: 

10.2337/diacare.24.6.1069

4. Ell, K., Katon, W., Cabassa, L.J., Xie, B., Lee, P. J., Kapetanovic, S., Guterman, J. (2009). Depression and 

diabetes among low-income Hispanics: Design elements of a socio-culturally adapted collaborative care 

model randomized controlled trial. The International Journal of Psychiatry in Medicine, 39(2), 113-132. 

doi: 10.2190/PM.39.2.a

5. Qaseem, A., Snow, V., Denberg, T.D., Forciea, M.A., Owens, D.K. (2008). Using second-generation 

antidepressants to treat depressive disorders: A clinical practice guideline from the American College of 

Physicians. Annals of Internal Medicine, 149(10), 725-733. doi:10.7326/0003-4819-149-10-200811180-

00007

6. Katon, W. Unutzer, J., Wells, K., Jones, L. (2010). Collaborative derpession care: History, evolution and 

ways to enhance dissemination and sustainability. General Hospital Psychiatry, 32(5), 456-464. 

Reference
7. Bell, R. A., Franks, P., Duberstein, P. R., Epstein, R. M., Feldman, M. D., Fernandez y Garcia, E., & Kravitz, R. L. 

(2011). Suffering in silence: reasons for not disclosing depression in primary care. Ann Fam Med, 9(5), 439-

446. doi: 10.1370/afm.1277 

8. Palinkas, L.A., Ell, K., Hansen, M., Cabassa, L., Wells, A. (2010). Sustainability of collaborative care 

interventions in primary care settings. Journal of Social Work, 11(1), 99-117. doi: 

10.1177/1468017310381310

9. Imbens G (2000) The role of the propensity score in estimating dose-response functions. Bimetrika 87(3): 

706–710.

10. Caliendo M and Kopeinig S (2008) Some practical guidance for the implementation of propensity score 

matching. Journal of Economic Surveys 22(1): 31–72.

11. Pasta D (2000) Using propensity scores to adjust for group differences: Examples comparing alternative 

surgical methods. 25th Annual SAS Users Group International Conference, Indianapolis, IN.

12. Rosenbaum P and Rubin D (1983) The central role of the propensity studies for causal effects. Biometrika 

70(1): 41–55.

13. Spreeuwenberg M, Bartak A, Croon M, Hagenaars J, Busschbach J, Andrea H, Twisk J, and Stijnen T (2010) 

The multiple propensity score as control for bias in the comparison of more than two treatment arms: an 

introduction from a case study in mental health. Medical care 48(2): 166–174

14. Kaplan, G., Bo-Linn, G., Carayon, P., Pronovost, P., Rouse, W., Reid, P., Saunders, R. (2013). Bringing a 

systems approach to health. Discussion Paper, Institute of Medicine and National Academy of Engineering, 

Washington, DC. http://www.iom.edu/systemsapproaches

Reference
Thank you!

• Questions?

Contact information:

Shinyi Wu, PhD, shinyiwu@usc.edu