The Power of Observational Data to Compare 

Treatments for Type 2 Diabetes on Long-Term 

Outcomes 

www.hcfe.research.va.gov

Julia Prentice and Steve Pizer

HERC Cyber Seminar 

April 16, 2014

Funding for this work was provided by the Health Services Research and Development Service of the U.S. 

Department of Veterans Affairs (Grant No. IIR 10-136). We are indebted to Donglin Li and Aaron Legler for 

programming support. The views expressed in this presentation are those of the authors and do not necessarily 

represent the position or policy of the Department of Veterans Affairs or Boston University. 
Acknowledgements

Thanks to our clinical collaborators!

Paul Conlin, MD- VA Boston Healthcare System

Walid Gellad, MD, MPH- VA Pittsburgh Medical Center

David Edelman, MD- VA Durham Medical Center

Todd Lee, PharmD, PhD- University of Illinois at Chicago
Type 2 Diabetes

• Seventh leading cause of death in United States

 

• Significant cause of morbidity 

 - Microvascular and macrovascular complications

• Progressive nature requires sequence of 

medications
What Treatment?

• Metformin 1st line treatment

• Over 12 classes of glucose lowering medication

 - Sulfonylureas (SU), thiazolidinediones (TZD), DPP-4 inhibitors, insulin 

• Evidence is based on randomized clinical trials and 

observational studies
Randomized Clinical Trial (RCT) 

Limitations

• Relatively short time frames 

 - <=12 months

• Short-term outcomes

 - Glycemic control

• More expensive

• Smaller sample sizes

• Clinical trial settings

• Non-established treatment

 
Randomized Clinical Trial (RCT) 

Compared to Observational Studies

RCT

• Relatively short time frames

• Short-term outcomes

 - Glycemic control

• More expensive

• Smaller sample sizes

• Clinical trial settings

• Non-established treatment

 

Observational Study

• Longer follow-up periods

• Long-term outcomes

 - AMI/Stroke

• Cheaper

• Larger sample sizes

• “Real world” settings 

• Established treatment
RCT: Causal Relationship Between 

Treatment and Outcomes

• In RCTs, randomization ensures that

– Observed (and unobserved) covariates are balanced 

between treatment and control groups

– Only difference is treatment assignment

– Thus, only cause of outcome difference is treatment

• No bias b/c coin flip is only driver of sorting and 

coin flip has no impact on outcomes

Flip of a Coin Sorting

Treatment 

group

Comparison 

group

Outcome
Potential Selection Bias in 

Observational Studies 

• In non-randomized studies, things get messy b/c there are 

many drivers of sorting that also affect outcomes.

Patient 

characteristics

Observed: health, 

income, ed, dist.

Unobserved: health,

skills, attitudes 

Provider 

characteristics

Observed: staff,

costs, congestion,

Unobserved: 

culture, attitudes,

leadership

Sorting

Institutional 

factors

laws, programs

Treatment 

group

Comparison 

group

Outcome
Limitations of Causation in Observational 

Studies

• Unobserved characteristics influence treatment

• Outcomes would be better or worse due to these 

unmeasured differences
Causation in Observational Studies

• Can we find a variable that acts like randomization 

in RCT?

 -Instrumental variable (IV)

• Yes! Local practice pattern not affected by 

individual patient’s health status 

Local 

practice 

pattern 

variation 

(IV)

Sorting

Treatment 

group

Comparison 

group

Outcome
Comparing Type 2 Diabetes Treatments

on Long-Term Outcomes

 • SU compared to TZD as second line agents

 

 

 

• Neural protamine Hagedorn (NPH) compared to 

analogue insulin

• Use prescribing practice variation as IV
Comparing SU to TZD 
Second Line Agents

• Metformin is established as 1st line treatment 

• SUs are no longer consistently recommended as 2nd

line agent

• Generic and used for decades

• Concerns about long-term effects

 - Have potential to cause hypoglycemia

 - Recent studies have found cardiovascular risk
Second Line Agents

• TZDs and DPP-4 inhibitors also available

• DPP-4 inhibitors recently entered the market

 - Not widely used in the VA

• Adverse events associated with TZDs

 - Cardiovascular, bladder cancer, osteoporosis 
Research Objective

• Are there differences in long-term 

outcomes when comparing SU to TZD? 
Study Population

• All patients with VA Rx for Metformin, SU or 

TZD in 2000-2007; follow through 2010

• Exclude those w/o Medicare 

• Include patients with history of metformin in 

baseline and SU or TZD as second agents 

– 80,936 patients

– 73,726 start SU; 7,210 start TZD
Study Timing

Time

12 month baseline

Index date

Start SU or TZD

Outcome period

• Latest index date is end of 2009

• Follow patients until first outcome or end of 

 2010
Outcome Variables

• Mortality

• Acute myocardial infarction (AMI) or stroke 

• Hospitalization for an ambulatory care-sensitive 

condition (ACSC)

– 13 adult conditions defined by AHRQ:

– E.g., CHF, COPD, PN, dehydration, long-term

complications of diabetes, UTI, asthma, angina, 

uncontrolled diabetes, short-term complications of 

diabetes, lower extremity amputation
Descriptive Statistics

Covariates Mean or Percent

Age 69.2 

HbA1c>=9 8 

Obesity 41 

Retinopathy 14 

Nephropathy 10 

Neuropathy 20 

Cerebrovascular 13 

Cardiovascular (severe) 25 

Peripheral vascular 14 

Outcomes

Mortality 10 

AMI or stroke 5 

ACSC hospitalization 17 
Treatment Variable 

• Start on SU compared to TZD
Percent on same 2nd line agent Two 

Years Later

0

10

20

30

40

50

60

70

80

90

100

SU TZD

Source: Tabulations of HCFE study cohort; 2001-2010
Other Control Variables

• Age, race, sex, baseline HbA1c, microalbumin, 

serum creatinine, BMI

 

• Components of Young diabetes severity index

• Elixhauser Dx-based comorbidity groups 

• Year effects, hospital effects
Instrumental Variable

Provider proportion 

of SU prescriptions 

in baseline

SU/(TZD+SU) 

Likelihood of 

starting SU

Likelihood of 

starting TZD

Outcome

• Provider-level prescribing patterns

– Proportion of second line agent prescriptions that are for SU

– Calculated at clinic level if provider wrote prescriptions for fewer 

than 10 unique patients (70% of the time) 

– Provider assigned at index date
Significant Variation Between Providers 

in SU Prescribing

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Jun-01

Dec-01

Jun-02

Dec-02

Jun-03

Dec-03

Jun-04

Dec-04

Jun-05

Dec-05

Jun-06

Dec-06

Jun-07

Dec-07

Jun-08

Dec-08

Jun-09

Dec-09

25%

50%

75%
Treatment Group Characteristics

Individual treatment

Covariates Start on SU

(n=73,726) 

Start on TZD

(n=7,210) 

Age 69.1 70.1 

HbA1c>=9 9 5 

Obesity 41 39 

Retinopathy 14 16 

Nephropathy 10 12 

Neuropathy 19 22 

Cerebrovascular 13 14 

Cardiovascular (some) 24 28 

Cardiovascular (severe) 26 23 

Peripheral vascular 14 16 
Balancing Effect of IV

Individual treatment Provider SU Prescribing Rate

Covariates Start on SU

(n=73,726) 

Start on TZD

(n=7,210) 

Bottom 50%

(n=40,453) 

Top 50%

(n=40,483) 

Age 69.1 70.1 69.2 69.2 

HbA1c>=9 9 5 8 8 

Obesity 41 39 41 41 

Retinopathy 14 16 14 14 

Nephropathy 10 12 10 10 

Neuropathy 19 22 19 20 

Cerebrovascular 13 14 13 13 

Cardiovascular 

(some)

24 28 25 25 

Cardiovascular

(severe)

26 23 25 25 

Peripheral vascular 14 16 14 14 
Process Quality Controls

• Provider-level process quality

 -Proportion of provider’s labs w/ A1c > 9

 -Proportion of provider’s labs w/ LDL > 100

 -Proportion of provider’s BPs > 140/90

 -Calculated in same way as instrument

Provider proportion 

of SU prescriptions 

in baseline

SU/(TZD+SU) 

Start SU

Start TZD

Process Quality Outcome
IV Implementation

• First equation Start SU/TZD = Xprovider Rx patterns + Xpatient + Xprocess quality+ u1 

• Provider SU prescribing history predicts individual treatment

 - Coefficient= 2.22 (95% CI: 2.10, 2.33)

• Powerful instrument!

 - F statistic of 1,374
IV Implementation

• Second equation 

 Outcome = Start SU/TZD + ȗ1+ Xpatient + Xprocess quality + u2 

• Cox proportional hazard models

 -Includes all covariates and controls 

 -Includes residual from 1st equation 

 -Residual controls for selection bias
Starting SU compared to 

TZD at Index Date^

^Adjusted Hazard Ratios from Cox Proportional Hazard Models

* Significant at P<0.05

1.21

1.18

1.23

1

1.1

1.2

1.3

Mortality ACSC Hospitalization AMI/Stroke

* 

* 

* 
Falsification Test

• Further test to confirm validity of SU prescribing rates as 

instrument

• Selected sample that just started MET and never started on SU 

(n=76,860)

• Follow for one year

• SU provider prescribing rates should have no influence on 

outcomes
Effect of Provider SU Share^

Outcome Hazard Ratio 95% Confidence 

Interval

Mortality 1.30 0.94, 1.79

ACSC Hospitalization 1.23 0.93, 1.62

AMI/Stroke 1.11 0.70, 1.77

^Adjusted Hazard Ratios from Cox Proportional Hazard Models
Conclusions

• Evidence of increased risks for patients who 

start SU compared to TZD as 2nd medication

• Consistent with other recent research

• Supports recent guideline changes to no 

longer recommend SU as preferred 2nd agent

• Future research should examine newer 

medications
Comparing Long-Acting Insulins
Insulin Choices

• Many patients with Type 2 diabetes requires insulin

• Choice of synthetic human (NPH) or analogue 

insulin

 - E.g. Glargine or Detemir 

• Analogue insulin designed to have a longer half life

 - Mimics natural insulin profile

• Analogue insulin significantly more expensive 
Short-Term Outcomes

• No difference in glycemic control 

• No difference in severe hypoglycemic events

• Fewer nocturnal hypoglycemic events on 

 analogue insulin
Long-Term Outcomes?

• Lower nocturnal hypoglycemia 

hypothesized to increase adherence

• Increased adherence decrease long-term 

complications and lower costs

• Short timeframe of studies prevents 

 conclusions on long-term outcomes
Long-Term Outcomes?

• Cost effectiveness studies provide mixed results

• Rely on clinical trial data to model long-term 

complications

 - May not reflect real world clinical settings

• Retrospective claim studies 

 - May not account for selection bias

 
Research Objective

• Are there differences in long-term outcomes 

when comparing NPH and analogue insulin?

 
Study Population

• All patients with VA Rx for DM meds in 2000-

2007; follow through 2010

• Exclude those w/o Medicare

• Include patients with history of metformin, SU 

or TZD in baseline that start on insulin

– 142,940 patients

– 118,878 start NPH; 24,062 start analogue
Study Timing

Time

12 month baseline

Index date

Start NPH or 

Analogue Insulin

Outcome period

• Latest index date is end of 2009

• Follow patients until first outcome or end of 

 2010
Similar Design to SU/TZD study

• Outcomes 

 -Mortality; ACSC hospitalization

• Control variables

 -Demographics; Labs (HbA1c, mircoalbumin, 

 serum creatinine); BMI; Elixhauser 

 comorbidities; Young Severity index; Year and 

 Hospital effects 

• Process quality variables 

 
Descriptive Statistics

Covariates Mean or Percent

Age 69.3 

HbA1c>=9 26 

Obesity 42 

Retinopathy 25 

Nephropathy 28 

Neuropathy 31 

Cerebrovascular 17 

Cardiovascular (severe) 37 

Peripheral vascular 23 

Outcomes

Mortality 33 

ACSC hospitalization 18 
Treatment Variable 

• Start on analogue insulin compared to NPH
First and Last Prescription Same 

Type of Insulin

0

10

20

30

40

50

60

70

80

90

100

NPH Analogue Insulin

Source: Tabulations of HCFE study cohort; 2001-2010
Instrumental Variable

Provider proportion of 

analogue prescriptions in 

baseline

Analogue/(Analogue+NPH) 

Likelihood of

starting NPH

Likelihood of 

starting analogue

Outcome

• Provider-level prescribing patterns

– Proportion of long acting insulin prescriptions that are for analogue

– Calculated at clinic level if provider wrote prescriptions for fewer than 

10 unique patients (53% of the time)

– Provider assigned at index date
Significant Variation Between Providers 

in Analogue Prescribing

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Jun-01

Dec-01

Jun-02

Dec-02

Jun-03

Dec-03

Jun-04

Dec-04

Jun-05

Dec-05

Jun-06

Dec-06

Jun-07

Dec-07

Jun-08

Dec-08

Jun-09

25%

50%

75%
Treatment Group Characteristics

Individual treatment

Covariates Start NPH

(n=118,878) 

Start on analogue

(n=24,062) 

Age 69.0 70.6 

HbA1c>=9 27 21 

Obesity 42 39 

Retinopathy 25 25 

Nephropathy 27 31 

Neuropathy 30 34 

Cerebrovascular 17 19 

Cardiovascular (some) 24 25 

Cardiovascular (severe) 37 38 

Peripheral vascular 22 24 
Balancing Effect of IV

Individual treatment Provider Analogue

Prescribing Rate

Covariates Start NPH

(n=118,878) 

Start on analogue

(n=24,062) 

Bottom 50%

(n=71,468) 

Top 50%

(n=71,472) 

Age 69.0 70.6 69.1 69.5 

HbA1c>=9 27 21 28 25 

Obesity 42 39 41 42 

Retinopathy 25 25 26 25 

Nephropathy 27 31 26 29 

Neuropathy 30 34 30 31 

Cerebrovascular 17 19 18 17 

Cardiovascular 

(some)

24 25 24 24 

Cardiovascular

(severe)

37 38 38 37 

Peripheral vascular 22 24 23 22 
IV Implementation

• First equation Start NPH/analogue = Xprovider Rx patterns + Xpatient + Xprocess quality+ u1 

• Provider analogue prescribing history predicts individual 

treatment - Coefficient= 2.76 (95% CI: 2.68, 2.83)

• Powerful instrument!

 - F statistic of 5,135
IV Implementation

• Second equation 

 Outcome = Start NPH/analogue + ȗ1+ Xpatient + Xprocess quality + u2 

• Cox proportional hazard models

 -Includes all covariates and controls 

 -Includes residual from 1st equation 

 -Residual controls for selection bias
Effect of Analogue Insulin^

Variable Mortality ACSC

Hospitalization

Analogue insulin compared to 

NPH

1.01 0.99

Age 1.05* 1.02*

Prescribed Metformin 0.78* 0.90*

Congestive heart failure 1.42* 1.61* 

Cerberovascular disease 1.05* 1.04*

Drug abuse 1.10* 1.23*

Depression 1.07* 1.09*

^Adjusted Hazard Ratios from Cox Proportional Hazard Models

Model also includes demographics, Elixhauser comorbidites, Young diabetes severity index components, prescribed 

sulfonylurea or thiazolidinedione, VA medical center and year effects 

* Significant at P<0.05
Analogue Insulin is Not Cost-Effective 

• No difference in mortality or ACSC 

hospitalization risk when comparing analogue 

and NPH

• Analogue insulin is not cost-effective 

• Significant cost implications

 - Gellad et al. (2013) estimates savings of $189 million
Policy and Research Implications

• Soon to expire patents will alter cost estimates

• Generic options may not easily enter the 

market 

 -Estimated to be only 20 to 40% cheaper 

• Future research should focus on quality of life 

outcomes
IV is a Powerful Tool

• Prescribing pattern variation is a strong IV

• Can be used to determine causality in 

observational data

• Expansion of electronic medical records

• Overcomes limitations of clinical trials
Questions or Comments?

Julia Prentice

Julia.Prentice@va.gov

(857)-364-6057

www.hcfe.research.va.gov
How Instrumental Variables Works

• Use Eq 1 to estimate ȗ1

• Add ȗ1 to Eq 2, so estimate of Treatment 

effect no longer biased by corr(u2,u1) 

Eq 2: Outcome = Start SU/TZD + ȗ1+ Xpatient + Xhospital + u2 

Eq 1: Start SU/TZD = Xprovider Rx patterns + Xpatient + Xhospital+ u1