The Practice of Cost‐Effective

Analysis: Dii es gn ng, C d ti on ucting 

Interpreting CEA under Non‐Ide

Circumstances

A Presentation for the VA Health Economics Resource Center

21 April 2010

Risha Gidwani, DrPH

Lead Health Economist

Abt Bio‐Pharma Solutions, Inc.
Overview

• Current Practice of CEA• Desi i gn ng and Cd i on ucting Your Own CE• Interpreting a Published CEA
Current Practice of CEA

• Academia

– Reference Case

– Societal perspective

– Lifetime Horizon

– Compare new strategy to gold stand d ar or best al i ternative

– TreeAge

• Ph i l armaceutical‐sponsored

– International Audiences and U.S. MCOs

– NICE, CADTH, PBAC– Thi dr ‐party payer perspective

– Non‐lifetime horizon

– Compare new strategy to placebo or to competitor

– Excel‐based
Designing and Conducting Conducting a 
Designing and Conducting Conducting a 

– Model structure– Estimating Costs– Estimating Probabilities– Obt i i Obtaining Utility V l a ues

– Sensitivity Analyses
Model Structure

• Time Horizon

– Lifetime– Long enough to capture the majority of costs & effects

• Perspective

– Societal– Third‐party payer

• Markov model

– Use if patients transition from one health state to another

– Health states should be collectively exhaustive and mutual

– Cycle l h engt sh ld ou be appropriate for disease in question
Markov models

• Frequently used in health services or clinical decision making• Benefit:

– More reflective of real life; patients do not just stay in one health 

• Drawback:

– Memory‐less quality

– Does not incorporate the effect of duration of disease

• ‐ problem with both Markov process and Markov cycle models

‐ A patient who had has uncontrolled diabetes for 8 years is more likely to d

diabetic retinopathy than a patient who has had uncontrolled diabetes fo

‐ M k ar ov mod l e does not take this into account; would assign si l ngu ar prob 

diabetic retinopathy regardless of the amount of time you have spent in t

– Can account for some of this using tunnel states and tracker varia

TreeAge
Inputs: Estimating Probabilit

• One of the most difficult aspects of a CEA• Also one of the most important• Potential Difficulties:

– Extrapolating Extrapolating beyond time horizon horizon of available available da– Lack of head‐to‐head trials– Lack of RCT data
Extrapolating beyond the horizo

existing data

• A clinical trial generally does not have a long enough follow‐

directly populate a CEA

– E.g., clinical trial lasts for 2 years; your CEA has a a 30‐year hor

• Last Observation Carried Forward ( ) LOCF

• Can lead to substantial inaccuracies

• A better approach applies a distribution to derive probability 

Years 3‐30:

1. Understand how your data are distributed

‐ Normal probability plot, Weibull probability plot, etc

2. Once you know how your data are organized, you can apply the app

distribution distribution to extrapolate extrapolate beyond the available available data

• Run one analysis at the time horizon for which you have obs

inputs, and compare those results to the time horizon that i

predicted predicted data inputs to see how results results vary
Lack of head‐to‐head data

• You want to compare Drug A to Drug B• Evidence exists comparing Drug A to placebo 

Drug B to placebo

• It is ill‐advised to take Drug A data from one 

and Drug B data from another study and plug 

directly into your model

• Doing so breaks randomization and introduce

( g, e. . different patient pp ) o ulations)

• Use a Bayesian meta‐analysis approach
Bayesian meta‐analysis approa

approa

• Use when you don’t have direct comparisons of Drug 

Drug B, but you have an evidence network that links 

Æ common comparator Æ Drug B

• Use these data and specify a regression model to th

compare

Drug A to Drug B

• This met od h is su c e t y fficiently co p e m l x to wa at rr n its ow

seminar(s)

• Needs the involvement of a Bayesian statistician
Inputs: Using data from RC

• Benefit: Randomization

– Accounts for observed and unobserved difference

patients across treatment arms

• Drawback:

– Clinical trial sample often does not reflect the pat

popul i at on

• Clinical trials have very strict inclusion/exclusion criter

– The clinical clinical trial assesses assesses drug efficacy; efficacy; the CEA m

evaluating drug effectiveness
Inputs: Inputs: Using data from chart re

• Patients not randomized to treatment• Selection bias• Propensity scoring can mitigate the effect of selectio

– Propensity Propensity score: composite composite variable variable that minimizes minimizes the dif

dif

patient characteristics across groups

• Use the propensity score to match patients from each group

– Has its limitations:

• Need large samples

• May still have biases – because you are only matching based 

variables

• N d ee a l t o of group overlap ‐‐ you d ’t on want to compare the 

from Group A to the best cases from Group B

• Propensity scoring is not a substitute for randomizat
Inputs: Cost Estimates

DATA INPUTS

Category Non-VA

Outpatient Medicare CPT MEP

Medicare CPT, MEP

Inpatient DRG, HCUPPharmacy Dr g Red Book Drug Red Book

- Medical Care supplement, July 2009, is an excellent source o

information on costing and cost sources
Inputs: Utility Values

• Utility: patient preference for health• Disease‐specific vs. generic

‐ Disease‐specific: directly provide utility

‐ SG, TTO, VAS

‐ Generic: provides HRQoL associated with a general heal

you apply preference weights to derive utility

‐ EQ‐5D, HUI, QWB, SF‐6D

• Patient sample vs. community sample• Be consit t s en ab t ou wh t a ever meth do is used 

elicit utilities for your CEA

SG: Standard Gamble; TTO: Time Trade‐Off; VAS: Visual Analog Scale; HUI: Health Utilities Index; QWB: Quality 
CEA Results

• Results come as ∆ Cost

II 

incremental cost‐effectiveness ratios (ICERs)and will fall into one of four

II 

quadrants:

Quadrant I: More cost and more

WTP

Quadrant I: More cost and more

effective. May be cost‐effective

Quadrant II: More costly and less

effective. Do not adopt strategy.

Q d III L l d lQuadrant III: Less costly and less

effective. May be cost‐effective

Quadrant IV: Less costly and more

effective. Definitely adopt strategy.

III 
Sensitivity Analyses

Sensitivity analyses:• One‐way• Multiple‐way• Tornado diagramsOverall Robustness of Model Results

• Probabilistic Sensitivity Analyses

– Uncertainty about model parameters, uncertainty 

probability estimates
Interpreting Published Cos

Effectiveness Analyses
Important Factors to Critique Critique

• Boundaries of the CEA• Extrapolation Beyond the Evidence Bas• Esti i mat on of Costs

• Estimation of Utilities

• Presentation of Results

• Assessing whether whether Model Results Results are

Robust
Boundaries of the CEA:

Time Horizon

Short‐term:

– Pros:

• Smaller impact of assumptions regarding clinical estimates

• Memory‐less cap y acit of Markov models not as much of a pro

– Cons

• Results may change with a longer time horizon

• Example: Drug A saves more lives than Drug B, and is also less 

– Drug A is cost‐saving compared to Drug B in a 2‐year horiz

– Drug A is cost‐effective compared to Drug B in a 3‐year ho

• At 2 years, patients in both arms were living, and Drug A costs 

was cost‐saving

• At 3 years, some patients on Drug B began to die. They now in

treatment costs. Drug A now costs more than Drug B. Howeve

cost effective because it is saving more QALYs than Drug B
Boundaries of the CEA:

Patient Characteristics

• What are the characteristics of the average p

entering the model?

– Do these characteristics have good face validity?– How common are these characteristics in the ove

patient popul ? ation

– Would results be different in another population?
Boundaries of the CEA:

Model Perspective

• Is the perspective of the mod l e really wh t a th

authors state it is?

– E.g., a model that does not consider consider direct non‐m

m

costs does not have a societal perspective

• Mod l e s with a non‐societal perspective

– Consider how results would change if direct non‐

or indirect indirect costs were included included

• E.g., Alzheimer’s disease ‐‐ excluding direct non‐medical costs (careg

would underestimate costs.

• E.g., Multiple Sclerosis ‐‐ excluding indirect costs (productivity) would 

underestimate costs
Excluding certain costs

Direct non‐medical (informal caregiver costs)

– If both treatments treatments have roughly roughly the same effect on patients patients

be independent and patients in each arm live for a similar a

time, then ICER will not be much affected

– If one treatment results in sig. improvement in patients’ abi

independent, then ICER will be affected, and the exclusion o

non‐medical costs is problematic

• E.g., Treatment for DME ‐‐ Tx A reverses vision loss, Tx B 

vision loss

Indirect (economic productivity)

– CEAs should include these when

• The average patient in the model is 18+ or < 65

– The age of the patient in the model is far younger than retireme

• Disease or tx impacts patient productivity at work

• Disease or tx requires that patient skip work (e.g., chem
Estimating Costs: Present Present Va

Inflation Adjusting

‐ Authors should note the base‐year of the cost estim‐ If there is a substantial time between the base‐year 

inflation‐adjusted year (e.g., 2000 to 2009), ask your

whether whether the rate of medical medical inflation inflation in this time per

reasonably apply to this good or service

‐ e.g., actual price of good may have decreased 

time
Extrapolating beyond the empirica

cl l inica information

• Cli i l Clinical Ti l r a s as source of if i n ormation

– Does the CEA model have a longer time ho

than the clinical clinical trial?

• Did authors use Last Observation Carrie

Forward?

• Did authors specify a distribution?

– If so, how did they decide on the distributio

• Chart review as source of information

– Did they authors use propensity scoring?

– Was this warranted?
Estimating beyond the empirical 

Treatments of Interest

If authors used clinical trials as the source of

information, did a head‐to‐head trial exist co

Treatment A to Treatment B?

– If not, how different are the patient populations i

source data?
Estimation of Utilities

• Was the measure appropriate?

– If generic generic method was used, is it sensitive sensitive enough to captu

captu

difference in different health states used for the CEA?

– If a disease‐specific method was used, has it been validated

• Was the measure consistent?

– All generic or disease‐specific– Was a consistent methodology used?

• All generic: was the same instrument used?

– All EQ‐5D, all HUI, all QWB, all SF‐6D

• All disease‐specific:

– All SG or all TTO or all VAS?

– Patient or community sample
Interpreting Interpreting the results results of a C

Description of Conclusion

• Results should be couched in terms of comparator

– Incorrect: Incorrect: Drug A is cost‐effective effective– Correct: Drug A is cost‐effective compared to Drug B

• Results should be interpreted in light of “average pat

model

– Incorrect: Drug A is cost‐effective compared to Drug B

– Correct: Drug A is cost‐effective compared to Drug B in the 

of hypertension in an elderly male U.S. population
Assessing whether model res

are robust

• Was a probabilistic sensitivity analysis

preformed?

– What parameters were varied?

• How much were these parameters varied?

• Are results results robust to the probabilistic probabilistic

sensitivity analysis?
Conclusion

• CEA can be a very useful tool, but the r

r

of the model depend very much on its

assumptions and the way in which mod

inputs were calculated

• Methods used to create model structu

derive model inputs should be evaluate

a critical eye before the assessing the v

of results
Q ti ? uestions

?