Comparative Effectivenes

Research: Opportunities f

the Implementation Resea

Community

Aanand D. Naik, M.D.,

Laura Petersen, M.D., M.P.H.
Outline

(1) What is CER?

(1) Definitions(2) Conceptual questions

(2) Why the attention on CER?

(1) Link to 2009 healthcare reform debate

(3) Current examples of CER

(1) AHRQ’s Effective Health Program

(4) Role of Implementation Science

(1) Obvious roles(2) Novel opportunities
Disclaimer

The views expressed in this presentare those of the authors and do not necessarily represent the views of thDepartment of Veterans Affairs (VA)QUERI; the Agency for Healthcare Research and Quality; or the John MEisenberg Center for Clinical Decisioand Communications Sciences at BaCollege of Medicine.
Comparative Effectivenes

“...a rigorous evaluationof the impact of differentoptions that are 

available for treating a given medical condition for a

particular set of 

patients.”

CBO, 2007
CER Definition Extende

• Compare similar treatments--competing drugs, or 

approaches--surgery versus drug therapy

• Analysis may focus on: 

– Relative medical benefits and risks of each opt– Weigh costs and benefits of those options

• Key issue is determining benefits for different type

patients for a given treatment

• In settings providing same treatment, can address 

differences in

– Diagnoses, systems of care, tests and follow-u
CER Bottom-line

• The core question of comparative effectiv

research—which treatment works best, fowhom, and under what circumstances— fundamental concern for patients and clinconfronting a health problem

• The direct comparison of existing health c

interventions to determine which works b

which patients and which poses the great

benefits and harms
Additional Considerations 

Defining CER

• Strength of Evidence

- What is the minimal level (threshold) of needed when comparing treatments?

• Applicability of Evidence

- How specific to the clinical question doeevidence of effectiveness need to be?
An Important Conceptual 

Distinction When Defining C

• Two distinct forms of comparative 

effectiveness research 

– A comparative effectiveness review

• Evidence synthesis

– A comparative effectiveness study

• Evidence generation
Outline

(1) What is CER?

(1) Definitions(2) Conceptual questions

(2) Why the attention on CER?

(1) Link to 2009 healthcare reform debate

(3) Current examples of CER

(1) AHRQ’s Effective Health Program

(4) Role of Implementation Science

(1) Obvious roles(2) Novel opportunities
Why Comparative 

Effectiveness Research?

• The Federal government has a stake in C

– Private sector has limited incentives fo– CER is not generally required for FDA 

as safe and effective

– The Federal government plays a subst

in financing health care in the U.S.

– Obama administration believes CER w

role in healthcare reform by better alignbenefits, costs & quality
Scope of the Opportunity in H

Care Reform

• Major challenges in 21st Century hea

care include evaluating all innovatiodetermining which:

– Represent added value– Offer minimal enhancements over exis

choices

– Fail to reach their potential– Work for some patients and not for oth

(AHR
Current US Activities in CE

• AHRQ Effective Practice Centers 1998 to • AHRQ Effective Health Care Program 200

present

– $30 million in 08-09; $50 million in 2009

• Department of Veterans Affairs (QUERI p• Drug Effectiveness Review Project (at OH• Other agencies: CMS, NIH (limited activiti• Health plan and other private efforts (inclu

Cochrane Collaboration)
American Recovery and

Reinvestment Act 2009 and C

• Congress allocated $1.1 Billion for CER

– AHRQ: $300M: Build on existing Effective Hea

program

– NIH: $400M (appropriated to AHRQ, transferre

*RC1- Challenge grants*RC2- Grand Opportunities grants

– Office of the Secretary: $400M (allocated at dis– The Federal Coordinating Council for CER crea

offer guidance and coordination on the use of f

– Funds are available through 9/30/2010
Outline

(1) What is CER?

(1) Definitions(2) Conceptual questions

(2) Why the attention on CER?

(1) Link to 2009 healthcare reform debate

(3) Current examples of CER

(1) AHRQ’s Effective Health Program

(4) Role of Implementation Science

(1) Obvious roles(2) Novel opportunities
CER at AHRQ

• Effective Health Care Program (EHP)

– Authorized by Section 1013 of the Medicare Pr

Drug, Improvement, and Modernization Act of 2

– Formal AHRQ program created in 2005

• Legislation mandated AHRQ is to conduct 

support research on:

– “the outcomes, comparative clinical effectivene

appropriateness of health care items and servi(including prescription drugs)”

– EHP focus has been to provide patients, clinici

policy-makers with reliable, evidence-based heinformation
Improving Quality and Safety 

• EHP uses CER to impact physician-patien

decisions based on these principles:

– Relevancy: is focused on actual clinica

decisions

– Timeliness: is fast and up-to-date

– Transparency: process involves public 

nomination and ongoing public comme

– Objectivity: employs methods and scie

rigor in systematic reviews to ensure acand unbiased reports

– Impact on priority populations and con
Priority Conditions for the

Effective Health Care Progr
Who Conducts the Resear

• Coordinated by AHRQ personnel• Contracts with multiple independent partn

• Stakeholder Group• Scientific Resource Center (at OHSC)• Evidence-based Practice Centers (EPCs) – 15 nat

centers

• Developing Evidence to Inform Decisions about 

Effectiveness Centers (DEcIDEs) – 13 national cen

• Centers for Education & Research on Therapeutics 

– 14 national centers

• John M. Eisenberg Clinical Decisions and Commu

Science Center - (transitioning from OHSC to Bayl
AHRQ

Effective Health Care Progra

Effective Health Care Progra
The John M. Eisenberg Cente

Clinical Decisions and 

Communications Science

• Data synthesis results in the production o

comparative effectiveness reviews

• The Eisenberg Center uses reviews to co

key messages that are disseminated into products tailored for three populations:

• a. Providers• b. Patients• c. Policy-makers
The Research Process The Research Process
Engage Stakeholders in Topic Select

the EHC Program

Convened EHC Program Stakeholder Panel and Program Priorities Work Group
EHC Process – Initial
EHC Process – Evolving
How Products Are Used How Products Are Used

9Inform clinical guideline development9Identify future research priorities 9Inform policy, including coverage decisions9Inform clinician and patient decisions
Outline

(1) What is CER?

(1) Definitions(2) Conceptual questions

(2) Why the attention on CER?

(1) Link to 2009 healthcare reform debate

(3) Current examples of CER

(1) AHRQ’s Effective Health Program

(4) Role of Implementation Science

(1) Obvious roles(2) Novel opportunities
Roles for the Implementatio

Research Community

• Obligation to build on dissemination• Implementation community can redefine 

dissemination function

• Innovative opportunities to narrow transla

gap
Implementation of key messa

• Passive recipient of CER key disseminati

messages

• Traditional implementation role

– VA QUERI Steps 4, 5, 6

• Obligation for implementation community

– Dissemination is not adequate– Implementation science is necessary to ensu

uptake of effectiveness data 
Redefining Dissemination

• Actively make transition from disseminati

implementation seamless

• Work actively with dissemination leaders, 

Eisenberg Center

– Potential roles

• Design and evaluation of dissemination products• Shape development of key messages
EHC Process – Initial
“From a Science of Dissemina

to Science of Implementatio

• Bring implementation variables into key 

message framing

– Contextual and local variables– Measurement models

• Earlier consideration of implementation

– Be a part of the discussion 

• Key concept development• Key clinical question development

– Mirrors evolution of Eisenberg Center

Margarita Alegr
EHC Process – Evolving
Translation Barriers

Sung et al. JAMA. 2003, 289(1
Translation Highway

Westfall et al. JAMA. 2007, 297(44)
Translation Time Lag

• 17 year lag from first 

publication to highly cited clinical trial 

Contopoulos-Ioannidis et al. Science 321:1298-99

• B-Blockers in MI

– 16 year lag 1980 -

1996 between BHAT trial and designation as quality measure

– In 1996, only 62.5%– In 2005, over 90% Le
Linear Approach to Transla

• T1 

Æ T2

– 17 years for undisputed clinical trials

• T2 

Æ T3

– 10 years for widespread guideline implement

• Does the linear approach to translation 

contribute to the excessive lag in implementation?
Novel Opportunities

• Implementation Research Community ca

novel (non-linear) approaches

– Integrate Implementation variables into CE re

(synthesis)

– Include Implementation outcomes as analytic 

components of CER (generation)
AHRQ

Effective Health Care Progra

Effective Health Care Progra
Implementation and Compara

Effectiveness Reviews

• Include implementation variables within C

reviews

– Contextual factors, geographic and subgroup 

variance, measures of reach/penetrance

• Incorporate these variables as part of the 

cost-benefit calculations

– This could actually change key dissemination 

messages
Implementation and CE Rese

• Integrate Implementation into CER studie

– Understand how implementation variables ca

change the analysis of CER results 

(Berwick JAMA 2008: Rapid Re

• Barriers to intervention uptake, adoption• Unexpected outcome event rates

– Include implementation process and outcome 

measures as comparative effectiveness outcduring CER generation
Implementation Should Rede

Comparative Effectivenes

• Efficacy is the measurement of validity in 

controlled settings

• Effectiveness is the measurement of valid

the real-world

– Implementation should be the bridge– Implementation science should establish the 

for comparative effectiveness

– Constant feedback between CER and observ

implementation effects
Recasting Translation Path

• Not really an example of linear translation

– Identification “door-to-balloon time” was an 

implementation outcome

– Implementation scientists identified process 

measures to enhance this outcome

– Continuous cycles of efficacy & implementati

Naik an
For Further Discussion

1) Integrating implementation within t

CER dissemination products

2) Incorporating implementation outco

in reviews of comparative effectiv

3) Making implementation variables a 

of comparative effectiveness stud
ACKNOWLEDGEMENT

• Chantele Singleton—clinical researc

manager at HSRD CoE and the JohEisenberg Center for Clinical Decisioand Communications Sciences at BaCollege of Medicine.
Comments & Questions