# Presentation: 663353

## Comparative Effectiveness Research

- Scott R. Smith, PhD
- Director, Pharmaceutical Outcomes Research
- Agency for Healthcare Research & Quality

## Today’s Outline

- Brief background on AHRQ’s Effective Health Care program.
- Peter Juhn, Johnson & Johnson, Inc.
- Jennifer Bright, Society for Healthcare Epidemiology of America.
- Gail Shearer, Consumers Union.
- Summary, more information & questions.

## Medicare Modernization Act 2003

**Notes:**
The Medicare Modernization Act of 2003 (MMA) launched a new era in national health policy in the United States.  In addition to increasing access to outpatient prescription medications through a new federal program, this landmark legislation creates new opportunities and a renewed national interest in improving the quality, safety, and effectiveness of health care across all settings.  The Part D drug benefit is in the forefront of this interest since the appropriate use of medications holds the potential to improve the health of many Americans as well as the effectiveness of nearly all other health care services in which pharmaceuticals are an integral component.  Nevertheless, expanded access and use of drug therapies also holds the potential for poor outcomes resulting from the adverse drug effects of medications, suboptimal treatment response, inappropriate choice of therapy, and gaps in knowledge on the true effects of a therapy across all patient populations and conditions for which it is used.  These results are especially true for treatments that over time are used in a broader spectrum of patients, clinical settings, and conditions than studied in pre-marketing clinical trials.  Furthermore, pre-marketing studies are generally not designed to evaluate multiple endpoints, detect rare adverse effects, evaluate health outcomes over long periods, or compare all treatment options available to patients and their health care providers.  As the federal government becomes a major payer of prescription drugs while also providing insurance coverage for hospital and medical care through Medicare, a new national focus has emerged on understanding the outcomes, effectiveness, and comparative effectiveness of treatments in clinical practice.  This includes understanding the balance of risks and benefits for all forms of therapies as they are used beyond clinical research settings in routine clinical care.

## Section 1013 of the MMA

- To improve the quality, effectiveness and efficiency of health care delivered through Medicare, Medicaid and the S-CHIP programs
- $15 million was appropriated in fiscal years 2005, 2006, 2007 (pending) for AHRQ to conduct and support research with a focus on
- outcomes
- comparative clinical effectiveness and
- appropriateness of health care items and services (including pharmaceutical drugs),
- including strategies for how these items and services are organized, managed and delivered.

**Notes:**
The need by patients, health care providers, and policy makers for objective information about the outcomes and effectiveness of therapeutic interventions was specifically addressed in Section 1013 of the MMA.  Section 1013 authorized the Agency for Healthcare Research and Quality (AHRQ) to conduct systematic reviews and research on topics of highest priority to Medicare, Medicaid, and the State Children's Health Insurance Program (SCHIP).  To implement Section 1013, AHRQ established the Effective Health Care program with a goal to increase knowledge about the balance of risks and benefits of therapies so that health care decisions are improved.  The program is comprised of three components that work collaboratively to evaluate, synthesize, develop, and disseminate scientific evidence about the effectiveness of various treatment options.  The first component of the program is the Evidence-based Practice Centers (EPCs) which conducts systematic reviews of available scientific evidence about the effectiveness of different treatments and identify gaps in the existing knowledge base.  To help fill these gaps, the program supports new research and the development of new methods through the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) network and the Centers for Education and Research on Therapeutics (CERTs) program.  Most authors of articles in this supplement are also investigators affiliated with the DEcIDE and CERTs centers.  The third component of the program is the John M. Eisenberg Clinical Decisions and Communications Science Center which translates the program's scientific findings into non-technical language that is communicated to different audiences.

## Types of comparisons

- Drug vs drug.
- Surgery vs watchful waiting.
- Inpatient versus outpatient treatment.
- Implantable device versus drug therapy.
- Specialty approach 1 versus specialty approach 2.
- Models of care (12-step, case management, skills training).

## Working Framework for Comparative Effectiveness

## Scientific Resource Center 

- Methodology resource
- Stakeholder input
- Dossier process
- Formative help
- Research coordination

## Effective Health Care Program 

- Evidence synthesis (EPC program)
- Systematically reviewing, synthesizing, comparing existing evidence on treatment effectiveness.
- Identifying relevant knowledge gaps.
- Evidence generation (DEcIDE, CERTs)
- Development of new scientific knowledge to address knowledge gaps.
- Accelerate practical studies.
- Evidence communication/translation  (Eisenberg Center)
- Translate evidence into improvements
- Communication of scientific information in plain language to policymakers, patients, and providers.

## Slide 9

## 1. Evidence Synthesis

- Evidence-based Practice Centers
- Created in 1997
- 13 centers
- Produce evidence reports/technology assessments
- Work with public and private sector partners
- User driven

## Methods of Systematic Review

- Systematic and comprehensive literature search
- Uniform extraction and tabulation of data from studies
- Qualitative and quantitative synthesis of data
- Critical appraisal of studies to identify factors that may lead to biased results

## First Published Comparative Effectiveness Reviews

- Management strategies for gastroesophageal reflux disease.
- Benefits and safety of analgesics for osteoarthritis.
- New diagnostic technologies for breast cancer screening.
- Epoetin and darbepoetin for managing anemia in cancer.
- Off-label use of atypical anti-psychotic medications
- Renal artery stenting for mild renal artery stenosis.
- Therapies for localized prostate cancer.
- Oral medications for diabetes management.
- Medications for depression management.
- Drug therapies and other interventions for osteoporosis.

## 2. Evidence Generation

- DEcIDE (Developing Evidence to Inform Decisions about Effectiveness) Network.
- Capitalizing on data
- New methods
- Answers for questions that don’t require multi-year, multi-million $ trials

## Evidence Generation

_**DEcIDE**_** Research Network**
- The main purpose of the DEcIDE network is to expeditiously develop _valid scientific evidence_ about the outcomes, clinical effectiveness, comparative effectiveness, safety, and appropriateness of health care items and services.
- The network is comprised of academic and clinic-based centers with access to electronic health information databases and the capacity to conduct accelerated research.

## What Does DEcIDE Primarily Do?

- Analyze _existing_ health care databases to compare the effectiveness & outcomes of treatment.
- Analyze _existing_ disease, device, and other registries.
- Conduct _methodological studies_ to improve research on clinical effectiveness of treatments.
- Ability to conduct _prospective studies_ including registries and practical clinical trials.

## Summary &“Unofficial” DEcIDE Statistics

- 13 DEcIDE centers.
- Over 60 core clinical scientists.
- Over 500 affiliated personnel.
- Access to >120 different health databases.
- Health data on over 50 million Americans.
*Nation’s largest network of researchers in therapeutic effectiveness.*

## Other DEcIDE Resources

**International data**
- all 570,000 residents of British Columbia
- GPRD (5% sample of UK)
**11 Medicaid programs**, incl some SCHIP
**Commercial pharmacy & lab vendor data**
- Ingenix
- PharmMetrics
- Medstat (private health plans)
**Electronic Medical Records e.g., Intermountain Health**
**Registries** heart failure, CAD, breast cancer, HIV, and Hep C

## Other Capabilities of DEcIDE

| 	Design & implement prospective studies. | 	Conduct clinical economics studies. |
| --- | --- |
|  	Develop algorithms to identify inappropriate drug prescribing patterns. |  	Evaluate the effects of benefit and formulary structure on health outcomes. |
| 	Conduct simulations and modeling. | 	Carry out evaluations of innovative healthcare services. |
| 	Evaluate patient and prescriber decision-making tools. | 	Examine clinical benefits of genetics testing. |

## Advantages of Observational Studies

- RCT’s rarely powered for ADE’s.
- RCT’s have poor external validity.
- May not generalize to the patient population for whom the intervention may be applied.
- Women, children, minorities, very elderly are under-represented.
- Multiple drug choices and alternatives.
- Co-morbid conditions.
- RCT setting, health care providers is “artificial”.
- Longer term follow-up is usually achievable.
- Facilitates risk management and risk minimization.
- Enables better translation.
- Interventions that are efficacious under a highly specific set of circumstances often fail to replicate across a wide variety of settings, conditions, patients.

## Key Challenges

- Methods
- Data
- Sorting out interaction between setting and intervention.
- Comparing therapies on the relevant clinical endpoints.

## Other challenges

- Reducing the timeline from question to answer
- Building data systems that answer policy questions
- Data: wealth of data available from large healthcare databases.

## Example DEcIDE Studies

- _Completed_
- Use of Clopidogrel in Stents (*JAMA*)
- Comparative Effectiveness of Beta-blockers (*American Journal of Cardiology*)
- Antipsychotic Treatment in Older Adults (*CMAJ*)
- Antidepressants and Aspiration Pneumonia (*J Clin Epi*)

- _In Progress_
- Bariatric Surgery in Patients with Type II Diabetes Mellitus
- Effectiveness & Safety of DMARDs
- Drug Therapies for COPD
- ACE-I Exposure in Pregnancy
- Coronary & Cardiac Stents
- Federated data networks
- Medication Therapy Mgmt

## Methods Tools

- AHRQ’s Registries for Evaluating Patient Outcomes: A User’s Guide
- _Purpose_: serve as a guide the design and implementation of patient registries, the analysis and interpretation of data from patient registries, and the evaluation of the quality of a registry or one of its components.

## User’s Guide

**Broad, multi-stakeholder involvement**
- Authors:  39 selected contributors with relative equal distribution from industry, academia, government and services providers
- Reviewers:  35 reviewers including NIH, FDA, CMS, OHRP, OCR, IOM.
- Case Studies:  20 case studies from 28 contributors
- Sections: Creating, Operating, Evaluating

## Obtaining User’s Guide 

- AHRQ's Clearinghouse (1-800-358-9295).
- Download from http://effectivehealthcare.ahrq.gov/reports/new_research.cfm
- Registries for Evaluating Patient Outcomes: A User’s Guide (Long Form) **AHRQ Pub. **_**No. 07-EHC001-1**_
- User’s Guide to Registries for Evaluating Patient Outcomes: Summary (Short Form) **AHRQ Pub. **_**No. 07-EHC001-2**_
- Announcing...Registries for Evaluating Patient Outcomes: A User’s Guide (Flyer) **AHRQ Pub. **_**No. 07-M029**_

**Notes:**
SAS Macro on Doubly Robust Estimation 
Instrumental variables research series

## Published Monday: Emerging Methods in Comparative Effectiveness & Safety

- A series of 23 articles by AHRQ researchers on new approaches in comparative effectiveness methods are compiled in a special October edition of *Medical Care.*
- A valuable new resource for scientists committed to advancing the comparative effectiveness and safety research.
- All articles available for free download on AHRQ’s website or from AHRQ’s Clearinghouse (Pub. No. OM07-0085).

- DEcIDE program _[http://effectivehealthcare.ahrq.gov/reports/med-care-report.cfm](http://effectivehealthcare.ahrq.gov/reports/med-care-report.cfm)_

## 3. Evidence Translation

- Translate scientific evidence and make it accessible to patients, providers, and policy- makers to promote informed health choices
- Conduct research to communicate complexities of decision-making and inform development of health care information products and decision aids
- Construct prototypes for decision aids and house MMA 1013 related decision aids

## Slide 28

## Slide 29

## Comparative Side Effect Information for Antidepressants

**Comparative Side Effect Information for Antidepressants**

- Based on systematic review of literature from EPC program.

## Opportunities

- Growing international interest in effectiveness research and comparative effectiveness studies.
- Converging interest of federal agencies in drug safety (e.g., Sentinel Network).
- Part D data.
- NIH’s CTSA program.
- IOM Reports.
- EHR/PHR.
- Collaborations with pharmaceutical & devices industries.

## Slide 32

## Program Outputs

**Research**
**Reports**

**Systematic**
**Reviews**

**Consumer ****Guides**

## More Information?

- Clancy CM, Slutsky JR.  Commentary: A Progress Report on AHRQ's Effective Health Care Program. Health Serv Res. 2007 Oct;42(5):xi-xix.
- Clancy C.  The effective health care program: evidence-based medicine joins the consumer age.  MedGenMed. 2006 Feb 27;8(1):56.
- --------------------------------------------------------------------------------------------------------------
- Garrison LP Jr, Neumann PJ, Erickson P, Marshall D, Mullins CD.  Using Real-World Data for Coverage and Payment Decisions: The ISPOR Real-World Data Task Force Report.  Value Health. 2007 Sep-Oct;10(5):326-35.
- From the Institute of Medicine...
- 	Institute of Medicine. 2007. Learning What Works Best: The Nations Need for Evidence on Comparative Effectiveness in Health Care.
- http://www.iom.edu/ebm-effectiveness.
- http://www.iom.edu/Object.File/Master/43/390/Comparative%20Effectiveness%20White%20Paper%20(F).pdf

## Scott R. Smith, PhDScott.Smith@ahrq.hhs.gov

*Even More Information *_*[http://](http://effectivehealthcare.ahrq.gov/)*__*[effectivehealthcare.ahrq.gov](http://effectivehealthcare.ahrq.gov/)*__*[/ ](http://effectivehealthcare.ahrq.gov/)*_

## Thank you!