# Presentation: 103828

## Model Reports for theAHRQ Quality Indicators

- Shoshanna Sofaer, Dr.P.H.
- School of Public Affairs
- Baruch College

## Overview

- Background and Purpose
- Evidence and experience base
- Key design elements
- The sponsor’s role

- AHRQ Annual Meeting

- September 27, 2007

## Background and Purpose

- The AHRQ Quality Indicators (QI) are quality measures based on hospital administrative data that are available in almost all states
- They grew out of the AHRQ Health Care Utilization Project’s data base of hospital utilization information
- The resulting indicators are based on evidence review and consultation with clinical experts
- Software is available to transform state level data into scores on Quality Indicators

- AHRQ Annual Meeting

- September 27, 2007

## Background and Purpose

- There are four sets of QIs:
- Prevention Quality Indicators (PQIs)
- Inpatient Quality Indicators (IQIs)
- Patient Safety Indicators (PSIs)
- Pediatric Quality Indicators (PedQIs)
- The Model Reports incorporate all but the PQIs

- AHRQ Annual Meeting

- September 27, 2007

## Background and Purpose

- Our purpose was to develop evidence based ways for reporting on the AHRQ QIs that leave sponsors flexibility to make choices about
- Which indicators to report
- How scores will be calculated
- The medium to be used

- AHRQ Annual Meeting

- September 27, 2007

## Background and Purpose

- There are two Model Reports and a memo for sponsors to guide them in applying the Model Reports to their own circumstances
- These documents are currently in the final stages of HHS review; when cleared, they will be posted on the AHRQ website for all to use.

- AHRQ Annual Meeting

- September 27, 2007

## Evidence and experience base for these tools

- Review of literature and existing evidence
- Direct experience in designing and evaluating reports of comparative quality information to the public
- Direct experience in testing comparative quality reports with the public

- AHRQ Annual Meeting

- September 27, 2007

## Evidence and experience base

- Interviews with experts and stakeholders
- Focus groups with hospital quality managers
- Focus groups specifically about the AHRQ QIs with consumers
- Multiple rounds of cognitive interviews with consumers about the draft design

- AHRQ Annual Meeting

- September 27, 2007

## Two model reports

- One model report takes all the IQIs, PSIs and PedQIs and puts them into health topics
- The second model report builds on four “composites” created by the AHRQ QI team, using multivariate statistical analysis
- Readers can “drill down” to individual indicators in either the topics or the composites

- AHRQ Annual Meeting

- September 27, 2007

## Key design elements

- Reports are designed primarily for the web, but can be adapted to print
- User can select
- health topics and composites of interest
- specific indicators of interest
- Hospital(s) for which they want to see data
- Sponsor has flexibility – not all topics or indicators need to be included

- AHRQ Annual Meeting

- September 27, 2007

## Key design elements

- The Model Reports include:
- Text for report home page
- Hospital search page
- Health topic or composite selection page
- For each health topic, composite and indicator, user friendly labels and definitions of often complex and arcane clinical terms

- AHRQ Annual Meeting

- September 27, 2007

## Key design elements

- For each topic and composite, a “word icon” comparison chart that show which hospitals were “better than average”, “average” or “worse than average”
- This particular chart has been tested in lab studies and substantially increases understanding

- AHRQ Annual Meeting

- September 27, 2007

## Compare Hospital Scores on surgery for cancer of the esophagus & pancreas

- Click on any of the indicators to see details on how each hospital performed on that particular indicator.

**Compare Hospital Scores on surgery for cancer of the esophagus & pancreas**

- AHRQ Annual Meeting

- September 27, 2007

## Key design elements

- For each composite and each indicator:
- A horizontal bar graph with accompanying text that shows
- Results for each hospital
- State average (can be sub-state if sponsor chooses)
- To maximize “evaluability” bar graphs are laid out so hospital at the top is the best and hospital at the bottom is the worst
- Again, this is based on strong evidence that this approach maximizes understand and also hospitals’ QI responses

- AHRQ Annual Meeting

- September 27, 2007

## Death rate from surgery to remove part of the esophagus 

- This graph shows you the percent of patients who died after an operation to remove part of their esophagus (the tube leading from the throat to the stomach).  This information is for patients admitted during 2004.

- When choosing a hospital, you should look for the hospital that has a _lower_ number of deaths. A _lower_ number is shown by a _shorter_ bar on the graph below.

**State Average:** The average rate of patients who died in the hospital after surgery to remove part of the esophagus, across your state.  This number is included so you have:

-    a better idea of what is normal for your state.

-    a standard to compare the other hospitals to.

- AHRQ Annual Meeting

- September 27, 2007

## Key design elements

- “Back end text” regarding
- How to use this report
- Things to keep in mind while reading/using the report
- What is quality?
- Other resources on quality
- Technical details about the report

- AHRQ Annual Meeting

- September 27, 2007

## The Sponsor’s role

- Select topics and indicators to report
- Decide on scoring methodology, including statistics for determining who is and is not “average”
- Decide what hospitals will be included
- Decide whether other hospital data will be included

- AHRQ Annual Meeting

- September 27, 2007

## The Sponsor’s role

- Gather and process data
- Create actual website, including search and linking functionality
- Decide on additional resources about quality to be added
- Add language regarding the methods used for scoring (and perhaps selecting indicators)

- AHRQ Annual Meeting

- September 27, 2007

## The Sponsor’s role

- Other critical sponsor roles:
- Managing the stakeholders
- Developing a plan for promoting the report so it will actually be seen and used
- We have developed a sponsor guide to specify and support this work.

- AHRQ Annual Meeting

- September 27, 2007

## Value of a Model Report

- Gives you a picture of how an entire report would look, when evidence is applied and careful testing is done
- Gives you a basis for creating your own report
- The AHRQ QIs Model Reports have been submitted to NQF
- We hope they will use them to articulate and endorse a framework of principles and practices for comparative public quality reports

- AHRQ Annual Meeting

- September 27, 2007