# Presentation: 094232

## AHRQ Quality Indicators 101: Background and Introduction to the AHRQ QIs

- Mamatha Pancholi, MS, Program Officer,
- Center for Delivery, Organization, and Markets
- Jeffrey J. Geppert, JD, EdM, Research Leader, Battelle Memorial Institute
- September 17, 2008

## Overview

- Origins
- Current Modules
- Advantages & Challenges
- Recent Improvements
- Future Vision
- Uses of the AHRQ QIs
- Reporting Template
- National Quality Forum Endorsement
- Additional Activities
- QI Learning Institute

## Learning Objectives

- Purpose of this Webinar is to provide an introduction to the AHRQ Quality Indicators for persons interested in using them for public reporting.
- By the end of this Webinar, participants should be able to:
- Identify and describe the five major categories of the QIs.
- Identify advantages and limitations of the QIs.
- Describe multiple uses of the QIs.
- Discuss significance of the National Quality Forum’s endorsement of several QIs.
- Identify the purpose and application deadline for the Quality Indicators Learning Institute.

## Overview

- Origins
- Current Modules
- Advantages & Challenges
- Recent Improvements
- Future Vision
- Uses of the AHRQ QIs
- Reporting Template
- National Quality Forum Endorsement
- Additional Activities
- QI Learning Institute

## Quality Indicators & HCUP

- HCUP:  Partnership among States, industry, and AHRQ
- Uniform database for cross-State studies; includes clinical, demographic, and resource use information
- Represents all inpatient discharge data from participating States—represents approximately 90 percent of all discharges

## AHRQ Quality Indicators (QIs)

- Developed through contract with UCSF-Stanford Evidence-based Practice Center
- Use existing hospital discharge data, based on readily available data elements
- Incorporate a range of severity adjustment methods, including APR-DRGs and comorbidity groupings
- Current modules: Prevention, Inpatient, Patient Safety, Pediatric and Neonatal

## Example Indicator Evaluation

**Notes:**
Indicators go through many iterations to obtain final definition.  The team begins with a review of the literature to determine which areas are ripe for measure development.  An empirical analyses is completed and an initial measure is defined.  This definition will be evaluated by a panel of experts consisting of approximately 14 multidisciplinary professionals, who assist in refining the measure.  Additional empirical analyses are completed at this time as well.  The measure could then proceed to a final definition or could be sent back to the panel for additional review.  Recently, we have added validation and field testing to the measures development cycle.

## Structure of AHRQ QI

- Definitions based on
- ICD-9-CM diagnosis and procedure codes
- Often along with DRG, MDC, sex, age, procedure dates, admission type, admission source, discharge disposition, discharge quarter
- Numerator is the number of cases “flagged” with the outcome of interest (e.g., postoperative sepsis, avoidable hospitalization for asthma, death)
- Denominator is the population at risk (e.g., pneumonia patients, elective surgical patients, county population from census data)
- The observed rate is numerator/denominator
- Volume counts for selected procedures

## Overview

- Origins
- Current Modules
- Advantages & Challenges
- Recent Improvements
- Future Vision
- Uses of the AHRQ QIs
- Reporting Template
- National Quality Forum Endorsement
- Additional Activities
- QI Learning Institute

## AHRQ Quality Indicators

**Inpatient QIs**
**Mortality**
**Utilization**
**Volume**

**AHRQ Quality Indicators**

**Prevention QIs**
**(Area Level)**
**Avoidable **
**Hospitalizations/**
**Other Avoidable **
**Conditions**

**Patient Safety **
**QIs**
**Complications**
**Unexpected Death**

- Neonatal QIs

**Notes:**
These were the 3 original modules of AHRQ Quality Indicators, the PQIs, the IQIs, and the PSIs.  The PDIs were consolidated into their own module and we just added a 5th  module, the Neonatal QIs. 
Prevention Quality Indicators (November 2001)
Area level (14 indicators and 3 composites)
Inpatient Quality Indicators (June 2002)
Area level (4 indicators)
Provider level (28 indicators and 2 composites)
Patient Safety Indicators (March 2003)
Area level (7 indicators )
Provider level (19 indicators and 1 composite)
Pediatric Quality Indicators (February 2006)
Area level (5 indicators and 3 composites)
Provider level (12 indicators and 1 composite)
Neonatal Quality Indicators (2008)
Provider level (3 indicators)

## Prevention Quality Indicators

- Bacterial pneumonia
- Dehydration
- Urinary tract infection
- Perforated appendix
- Low birth weight
- Angina without procedure
- Congestive heart failure

- Hypertension
- Adult asthma
- COPD
- Diabetes cx - short term
- Diabetes cx - long term
- Uncontrolled diabetes
- Lower extremity amputation

## Inpatient Quality Indicators

- _Volume Indicators_
- Esophageal resection
- Pancreatic resection
- AAA repair
- CABG
- Percutaneous transluminal coronary angioplasty
- Carotid endarterectomy

- _Utilization Indicators_
- Cesarean delivery rate
- Primary cesarean delivery rate
- VBAC rate
- VBAC rate, uncomplicated
- Laparoscopic cholecystectomy rate
- Incidental appendectomy in the elderly rate
- Bilateral cardiac catheterization rate

## Inpatient Quality Indicators (cont.)

- _Mortality Indicators for Inpatient Conditions_
- Acute myocardial infarction
- AMI, without transfer cases
- Congestive heart failure
- Gastrointestinal hemorrhage
- Hip fracture
- Pneumonia
- Acute stroke

- _Mortality Indicators for Inpatient Procedures_
- AAA repair
- CABG
- Craniotomy
- Esophageal resection
- Hip replacement
- Pancreatic resection

***Two additional mortality indicators (carotid endarterectomy and PTCA) are included in the IQI software but are not recommended as stand-alone measures. ***

## Patient Safety Indicators

- Complications of anesthesia
- Death in low mortality DRGs
- Decubitus ulcer
- Failure to rescue
- Foreign body left during procedure *
- Iatrogenic pneumothorax *
- Selected infections due to medical care *
- Postoperative hemorrhage or hematoma
- Postoperative hip fracture
- Postoperative physiological and metabolic derangement
- Postoperative PE or DVT

*The indicators marked with * are also provided as area-level indicators.**  *

## Patient Safety Indicators (cont.)

- Postoperative respiratory failure
- Postoperative sepsis
- Postoperative wound dehiscence *
- Technical difficulty with procedure *
- Transfusion reaction *
- Birth trauma – injury to neonate
- Obstetric (OB) trauma – cesarean delivery (w/ and w/o 3rd degree lacerations)
- OB trauma – vaginal delivery with instrument (w/ and w/o 3rd degree lacerations)
- OB trauma – vaginal delivery without instrument (w/ and w/o 3rd degree lacerations)

*The indicators marked with * are also provided as area-level indicators.*

## Pediatric Quality Indicators

- _Inpatient Indicators_
- Accidental puncture and laceration
- Decubitus ulcer
- Foreign body left in after procedure
- Iatrogenic pneumothorax in neonates at risk
- Iatrogenic pneumothorax in non-neonates
- Pediatric heart surgery mortality
- Pediatric heart surgery volume
- Postoperative hemorrhage or hematoma
- Postoperative respiratory failure
- Postoperative sepsis
- Postoperative wound dehiscence due to medical care
- Transfusion reaction

## Pediatric Quality Indicators (Cont’d)

- _Area-Level Indicators_
- Asthma admission rate
- Diabetes short-term complication admission rate
- Gastroenteritis admission rate
- Perforated appendix admission rate
- Urinary tract infection admission rate

## Overview

- Origins
- Current Modules
- Advantages & Challenges
- Recent Improvements
- Future Vision
- Uses of the AHRQ QIs
- Reporting Template
- National Quality Forum Endorsement
- Additional Activities
- QI Learning Institute

## Advantages 

- Public Access
- All development documentation and details on each indicator available on Web site  _[www.qualityindicators.ahrq.gov](http://www.qualityindicators.ahrq.gov/)_
- Software available to download at no cost
- Standardized indicator definitions
- Can be used with any administrative data: HCUP,* MEDPAR,** State data sets, payer data, hospital internal data
- Hospitals can replicate data
- *Healthcare Cost & Utilization Project
- **Medicare Provider Analysis and Review

**Notes:**
Advantages of the AHRQ QIs:
Software is available in a SAS version as well as a Windows version.
Indicator definitions are updated yearly and released every February.

## Advantages (cont’d)

- Scope
- Over 100 individual measures
- Each measure can be stratified by other variables including patient race, age, sex, provider, geographic region
- Include priority populations and areas:  Child health, women’s health (pregnancy and child-birth), diabetes, hypertension, ischemic heart disease, stroke, asthma, patient safety, preventive care
- Focus on acute care but do cross over to community and outpatient care delivery settings

**Notes:**
The IQIs consist of 28 provider level indicators and 4 area level indicators for a total of 32 measures for inpatient hospital care.
The PQIs consist of 14 area level indicators for ambulatory sensitive conditions.
The PSIs consist of 20 provider level indicators and 7 area level indicators for a total of 27 measures for patient safety.
The PDIs consist of 13 provider level indicators and 5 area level indicators for a total of 18 measures for pediatric inpatient care.
The NQIs consist of 3 provider level indicators and X area level indicators for a total of X measures for neonates.

## Advantages (cont’d)

- Harmonization of measures
- Indicator maintenance, updates
- Tools and technical assistance
- National benchmarks
- National Healthcare Quality Report
- National Healthcare Disparities Report
- HCUPnet

**Notes:**
Yearly updates of measures including ICD-9 coding updates, DRG code updates, reference population update (use a 3 year rolling population), evidence based on literature review (leads to updating the measure specs).  All updates are incorporated into the QI software which is released every February.

## Current Limitations & Challenges

- Outcomes data less actionable than processes
- Lack clinical detail
- Risk adjustment challenges
- Accuracy hinges on accuracy of documentation and coding
- Data potentially subject to gaming
- Time lag

**Notes:**
The limitations have to do with the data source—administrative data.

## Overview

- Origins
- Current Modules
- Advantages & Challenges
- Recent Improvements
- Future Vision
- Uses of the AHRQ QIs
- Reporting Template
- National Quality Forum Endorsement
- Additional Activities
- QI Learning Institute

## Recent Improvements

- Composite Measures: Developed composite measures for the IQIs, PSIs, PQIs, and PDIs
- Risk Adjustment/Admin Data: Additional risk adjustment methods for AHRQ QIs
- Updated Literature Reviews:  Completed IQIs, Pediatric QIs, and PSIs
- Reporting Template:  Tested and refined
- NQF Endorsed QIs

**Notes:**
Admissions for uncontrolled diabetes or short-term complications per 100,000 population is based on PQI #1 - Diabetes Short-term Complication Admission Rate and PQI #14 - Uncontrolled Diabetes Admission Rate.
·         Admissions for diabetes long-term complications per 100,000 population is based on PQI #3 - Diabetes Long-term Complication Admission Rate.
·         Admissions for lower-extremity amputation among patients with diabetes per 100,000 population is based on PQI #16 - Rate of Lower-extremity Amputation among Patients with Diabetes.
Composite measures, the reporting template, and select QIs (33) were submitted to NQF for potential endorsement.  Evaluation by the Technical Advisory Panels of NQF are underway.

## Overview

- Origins
- Current Modules
- Advantages & Challenges
- Recent Improvements
- Future Vision
- Uses of the AHRQ QIs
- Reporting Template
- National Quality Forum Endorsement
- Additional Activities
- QI Learning Institute

**Notes:**
Future vision of the next iteration of the AHRQ QIs.

## Future Vision: Join Forces to Improve Data

- Use HIT to improve timeliness
- Add clinical detail for accuracy, credibility
- Condition present on admission
- Lab values
- Expand outpatient reach (e.g., emergency department, physician data)
- Pilot cross-site data, new data links
- New tools for expanded data
- Continue privacy & data security

**Notes:**
Some ideas that AHRQ will be exploring for this “joining forces” effort are:

Use HIT to improve the timeliness of the reporting of data to state Partner oraganizations

Add clinical detail to improve quality measurement, including an indicator of whether each diagnosis was present before admission to distinguish complications of hospital care from patient comorbidities.  And a recent AHRQ-funded study shows that there are 20 lab test values that would greatly enhance the accuracy of the claims data for hospital quality reporting.

Expand statewide outpatient data, develop new data analysis tools for outpatient and clinically enriched data, and do this with continued vigalence of patient privacy and data security,

## Followup:  State Pilots to Add Clinical Detail and Robustness

- AHRQ has funded HCUP Partner projects to add more clinical data to statewide administrative databases
- Present on Admission (POA)
- Laboratory values
- Other

## Overview

- Origins
- Current Modules
- Advantages & Challenges
- Recent Improvements
- Future Vision
- Uses of the AHRQ QIs
- Reporting Template
- National Quality Forum Endorsement
- Additional Activities
- QI Learning Institute

## General Uses of the AHRQ QIs

- Hospital Quality Improvement – Internal and External
- Individual hospitals and health care systems
- Hospital association member-only reports
- National, State, and Regional Reporting
- National Healthcare Quality/Disparities Reports
- Commonwealth Fund’s Health Performance Initiative
- Pay-for-Performance by Hospital
- CMS/Premier Demo
- Anthem of Virginia
- Hospital Profiling
- Blue Cross/Blue Shield of Illinois
- Comparative Public Reporting

**Notes:**
A recent NAHDO study indicated that states are moving to the model of administrative data fro data collection for reasons of cost and availability.

## 12 States Use QIs for Public Hospital Reporting

**Notes:**
Currently 12 States are using the QIs to publicly report hospital performance.  

## Overview

- Origins
- Current Modules
- Advantages & Challenges
- Recent Improvements
- Future Vision
- Uses of the AHRQ QIs
- Reporting Template
- National Quality Forum Endorsement
- Additional Activities
- QI Learning Institute

## QI Reporting Template(s)

- Challenge:  Presentation Matters!!
- Approach:  Two Model Templates
- Composite scorecard
- Health topic/disease
- Report Sponsors Choose:
- Overall approach
- Topics, composites, & measures to report
- How scores will be calculated
- The medium to be used

**Notes:**
The idea for the reporting template originated as a request from the QI user community.  These reports are designed specifically to report comparative information on hospital performance based on the AHRQ QIs.  There are two model reports, one developed in mid-2006, that groups individual indicators together and organized them into Health Topics.  This first model is known as the Health Topics Model Report.  The second model, known as the Composite Model Report was developed later in 2006, and is based on a set of four composite measures using a substantial portion of the IQIs, the PSIs, and the PDIs.  These composites were created on the basis of substantial statistical analyiss as well as an expert review.

## The Evidence Base

- Model reports are based on:
- Literature review and analysis
- Interviews with experts
- Focus groups with different populations
- Cognitive interviews

**Notes:**
Extensive search and analysis of the literature on hospital quality measurement and reporting, as well as public reporting on health care quality more broadly;
Interviews with quality measurement and reporting experts, purchasers, staff of purchasing coalitions, and executives of integrated health care delivery systems who are responsible for quality in their facilities;
Two focus groups with chief medical officers of hospitals and/or systems and two focus groups with quality managers from a broad mix of hospitals;
Four focus groups with members of the public who had recently experienced a hospital admission; and 
Four rounds of cognitive interviews (a total of 62 interviews) to test draft versions of the two model reports with members of the public with recent hospital experience, basic computer literacy but widely varying levels of education.

## Overview

- Origins
- Current Modules
- Advantages & Challenges
- Recent Improvements
- Future Vision
- Uses of the AHRQ QIs
- Reporting Template
- National Quality Forum Endorsement
- Additional Activities
- QI Learning Institute

## QIs Submitted to the National Quality Forum (NQF)

- Submitted September 2006
- Select Individual QIs (34)
- Composite Measures (4)
- Reporting Template/Framework (2)
- Additional QIs Submitted
- Perinatal Project (3)
- Perioperative Project (3)

**Notes:**
This project was the National Voluntary consensus Standards for Hospital Care:  Additional Priorities.

## NQF Endorsed QIs

- Diabetes Project
- PQI #1 - Diabetes Short-Term Com-
- 	plication Admission Rate and PQI #14 - Uncontrolled Diabetes Admission Rate
- PQI #3 - Diabetes Long-Term Complication Admission Rate
- PQI #16 - Rate of Lower-Extremity Amputation Among Patients With Diabetes
- Hospital Additional  Priorities 2005-06, Pneumonia Mortality Supplement
- IQI #20 – Inpatient Pneumonia Mortality

## NQF Endorsed QIs

- Ambulatory Care Project: Phase 3
- PQIs & PQI Composite
- Standards of Hospital Care: Additional Priorities 2007
- Patient Safety, Adult (8)
- Patient Safety, Pediatrics (4)
- Pediatrics (2)
- Surgery & Anesthesia (11)
- Perioperative Care
- Postop PE/DVT
- Perinatal Care
- Birth Trauma Injury to Neonates
- Nosocomial Bloodstream Infections in Neonates

## Overview

- Origins
- Current Modules
- Advantages & Challenges
- Recent Improvements
- Future Vision
- Uses of the AHRQ QIs
- Reporting Template
- National Quality Forum Endorsement
- Additional Activities
- QI Learning Institute

## ICD-9-CM Coding Proposals

- AHRQ has proposed several coding changes to ICD-9-CM that were needed to enhance the accuracy and usefulness of some indicators
- Retinopathy of prematurity
- Necrotizing enterocolitis
- Disruption of postoperative wound
- Deep vein thrombosis
- Transfusion reaction

## Measure Harmonization

- Working with other measure developers to align specifications of similar measures
- JCAHO
- CMS
- Researchers

## Validation Studies

- AHRQ sponsored
- Phase I
- Simple Review
- Indepth Review
- Supplemental Review
- Phase II
- Currently Recruiting

**Notes:**
Focus on selected PSIs and includes neonate, pediatric and adult populations.
Simple review would address the question of whether cases flagged by eh AHRQ PSI did or did not have the clinical event.
In-depth Review would further address the potential preventability of the clinical event.  Data collection would focus on the process of care that either led to the event or might have prevented the event.
Supplemental Review would augment either of the previous levels, and focus on the AHRQ PSI inclusion and exclusion criteria and risk factors that might be important for improving indicator specifications and for interpreting and using the rates.
Thus far we have 36 organizations that have expressed an interest in participating in these studies.

## Validation Pilot, Phase I

- Pilot Objectives:
- Gather evidence on the scientific acceptability of the PSIs
- Medical record reviews, data analysis, clinical panels, evidence reviews
- Consolidate the evidence base
- Improve guidance on the interpretation and use of the data
- Evaluate potential refinements to the specifications

## Validation Pilot, Phase I

- Conclusions
- The five evaluated PSIs have variable PPVs, which should be considered in selecting indicators for public reporting and pay-for-performance
- Pilot-tested a mechanism for supporting ongoing validation work, which can be applied to estimate sensitivity in Phase II

## Validation Pilot, Phase II

- Pending OMB review
- Estimate sensitivity (false negatives) in addition to PPV (false positives)
- 16 organizations have indicated an interest in participating in Phase II
- Encourage hospitals in HCUP partner States to participate
- Contact Jennifer Cohen (_[cohenj@battelle.org](mailto:cohenj@battelle.org)__) _

## AHRQ QI Development

- Emergency Preparedness
- PQIs for the Medicaid Population

## Overview

- Origins
- Current Modules
- Advantages & Challenges
- Recent Improvements
- Future Vision
- Uses of the AHRQ QIs
- Reporting Template
- National Quality Forum Endorsement
- Additional Activities
- QI Learning Institute

## AHRQ QI Learning Institute

_**Purpose:**_** **To provide a forum for discussing and facilitating the use of the AHRQ Quality Indicators (QIs) in statewide and regional programs that report hospital quality measures to the public.
- Provide opportunities for discussion and peer-to-peer learning through Webinars, conference calls, and in-person meetings
- Increase familiarity with the AHRQ QI reporting template and related tools.
_**Target Audience:**_** **Leaders (or their representatives, including project staff and consultants) who are directly involved with developing public reporting programs from:
- State agencies/task forces,
- State hospital associations, and/or
- Coalitions** **

## Possible Topics of Monthly Webinars 

-    Selecting Measures and Sample Size
- 	Preparing Data for the Report
- 	Classifying Hospitals
- 	Displaying the Data
- 	Explaining the QIs on Your Web Site
- 	Creating a Consumer-Friendly Web Site
- 	Marketing and Promoting Your Report
- 	Evaluation of Public Reporting Program

**Notes:**
Roadmap for Developing a Public Reporting Program

## Applying for Membership 

- If you are interested in becoming a member of
- the AHRQ QI Learning Institute:
- Send an e-mail to: _[QualityIndicatorsLearning@ahrq.hhs.gov](mailto:QualityIndicatorsLearning@ahrq.hhs.gov)__ with the subject heading “Application Information for the QI Learning Institute.”  _
- AHRQ will respond with application details.
- Application due by September 30, 2008.

**Notes:**
 THEY MUST APPLY!!!!!!!
Due to the great interest in the Learning Institute and a maximum of 250 participants, interested parties are being asked to apply for membership.  We may need to limit the number of persons in order to involve as many eligible programs as possible.  

## For More Information...

- Quality Indicators:
- Web site: _[http://](http://qualityindicators.ahrq.gov/)__[qualityindicators.ahrq.gov](http://qualityindicators.ahrq.gov/)__[/](http://qualityindicators.ahrq.gov/)_
- QI documentation and software are available
- E-mail: _[support@qualityindicators.ahrq.gov](mailto:support@qualityindicators.ahrq.gov)_
- Support Phone: (888) 512-6090 (voicemail)
- Staff:  _[Mamatha.Pancholi@ahrq.hhs.gov](mailto:Mamatha.Pancholi@ahrq.hhs.gov)_
- QI Learning Institute
- QI Learning Institute Web site:
- 	_[http://](http://www.ahrq.gov/qilearninginstitute/)__[www.ahrq.gov/qilearninginstitute](http://www.ahrq.gov/qilearninginstitute/)__[/](http://www.ahrq.gov/qilearninginstitute/)_
- E-mail: _[QualityIndicatorsLearning@ahrq.hhs.gov](mailto:QualityIndicatorsLearning@ahrq.hhs.gov)_