# Presentation: 319639

## CAHPS Surveys and Patient Safety

- Beth Kosiak, Ph.D.
- Senior Social Scientist
- Center for Quality Improvement and Patient Safety
- AHRQ

## Overview of Presentation

- Explanation and background of the CAHPS project—what  CAHPS is, who is involved, products currently available and under development
- Specific relevance to patient safety
- Data and resources available

## Quality From the Consumer Perspective

- CAHPS (originally Consumer Assessments of Health Plans Survey) is a family of surveys and reports that measure patients/consumers perspectives on quality, based on the best science in the field.
- These products are designed to promote public reporting for consumer choice and accountability, quality improvement, and quality monitoring
- In 1995, 3 grantees, Harvard, RAND and RTI, were awarded a competitive 5 year grant to produce surveys and reports for consumers in health plans. In 2001, Harvard, RAND and AIR were awarded the second 5 year competitive grant, to produce a surveys and reports for different populations and providers. Westat was awarded the support contract for both cycles
- CAHPS has established a rigorous evidence-based process to create surveys, methods of administration, reports of results to consumers and providers
- Results from CAHPS surveys are used in the National Healthcare Quality and National Healthcare Disparities Reports as indicators of patient centeredness
- CMS, CDC, NIDRR, DoD, VA, NCQA, AMGA, ABMS, AARP

## Current Status of CAHPS for Health Plans

- NCQA adopted health plan CAHPS as part of its HEDIS accreditation set in 1998
- CMS has administered and publicly reported CAHPS results on its website for Medicare Managed Care and Medicare Fee-for-Service annually since 1998
- Medicaid, state purchasing cooperatives, private employers have used the Medicaid and commercial versions of health plan CAHPS since the late 1990s
- Currently, over 130 million Americans are enrolled in health plans that use CAHPS results
- CAHPS for health plans is seen as the industry standard

## Satisfaction Versus Experience of Care

- Measure reports of experience “How often did the nurse come within 15 minutes when you rang the call button?
- Measure ratings “Using any number from 0-10, where 0 is the worst hospital possible and 10 is the best hospital possible, what number would you sue to rate this hospital?”
- Ceiling effects in satisfaction questions
- Don’t tap into detail of important aspects of experience

## CAHPS Guiding Principles

- Focus on asking those questions for which the patient is the best or only source of information
- Ask questions only of those patients who have had the experience (skips)
- Anchor responses within a specific timeframe for improved comparability
- Similar response scales
- Use of a set of core items specific for all providers of a certain type (i.e., health plans) with the option of adding items for more specificity
- Input from all affected parties
- Use of the best science—focus groups, cognitive testing, psychometric testing

## CAHPS Products Rely on the Best Science and a Rigorous Development Process

- Literature review
- Public call for measures, federal register notice
- Team assessment of existing surveys according to evidence on reliability, validity, scope of survey use
- Formation of technical expert panels for advice and guidance—include stakeholders from industry, consumer advocacy, researchers and purchasers
- Conduct consumer and provider focus groups to determine what they think is important to include
- Generate domains (topics) of survey based on above steps

## Development Process (continued)

- Team creates draft survey
- Cognitively tests survey with real patients
- Revises and conducts second set of cognitive tests with patients
- Revises and releases for public comment through federal register notices
- Hold stakeholder meetings to get further input
- Speak at professional conferences to get feedback
- Incorporate feedback from all of above to develop field test version

## Development Process (continued)

- Release field test version and draft protocol for administration for public comment
- Devise sample and administration protocol that will allow for statistically valid comparisons to be made among facilities (health plans, hospitals, nursing homes, dialysis facilities, individual clinicians and group practices)
- Test in field with nationally representative sample of facilities and patients
- Conduct field test using mixed mode and telephone, starting to test IVR and web
- Analyze results to see which items worked, which domains are reflected by factor analysis
- Revise survey

## Development (continued)

- Conduct focus groups with patients/consumers to test items and domains included in field test
- Provide results to survey analysis team to determine which items are most important to patients
- Revise survey
- Develop reports by testing explanatory language for areas to be publicly reported
- Cognitively test drafts of reports with patients/consumers
- Revise reports

## CAHPS is Growing

- Hospital CAHPS (HCAHPS)
- In Center Hemodialysis (ICH) CAHPS
- Nursing Home Resident (NH CAHPS Resident)
- Nursing Home Family (NH CAHPS Family)
- Ambulatory CAHPS: Individual Clinician, Group Practice, Health Plan
- Behavioral Health (ECHO)
- Persons with Mobility Impairments (PWMI)
- American Indian CAHPS
- Assisted Living CAHPS
- Home Health CAHPS

## Development of CAHPS Patient Safety Items

- Qualitative research sponsored by both CMS and AHRQ shows that patients think of facility cleanliness as a patient safety issue
- The term patient safety encompasses a wide array of events, many that are non-medical
- Must be sure that the question refers to something most patients in that facility (hospital, dialysis center, nursing home) would experience or observe directly

## Examples of Patient Safety Questions 

- .In the last 3 months, how often did dialysis center staff check you as closely as you wanted while you were on the dialysis machine?
- In the last 3 months, how often did dialysis center staff change their gloves between patients?
- Has any dialysis center staff ever told you how to get off the machine if there is an emergency at the center?
- In the last 3 months, how often was the dialysis center as clean as it could be?
- During this hospital stay, how often were your room and bathroom kept clean?
- Before giving you any new medicine, how often did hospital staff describe possible side effects in a way you could understand?

## Hospital CAHPS

- Domains: Care from Nurses, Care from Doctors, Hospital Environment, Experiences in the Hospital,Discharge, Overall Rating, About You
- Extensively tested
- Recently endorsed by NQF
- Plans for nationwide voluntary implementation by CMS late this fall; initial “dry run” before data is publicly reported by hospital
- De-identified data will be publicly available, discussions currently underway;
- Future potential for comparing HCAHPS and Hospital Culture Survey data

## National CAHPS Benchmarking Database (NCBD) http://ncbd.cahps.org

- National repository for CAHPS survey data, supports benchmarking and research; funded by AHRQ and administered by Westat through SUN
- Survey sponsors can voluntarily submit data at individual respondent level (de-identified) and receive free, customized report with national, regional and plan type (HMO, PPO, PCCM) averages
- Currently includes Commercial Adult, Commercial Child, Medicaid Adult, Medicaid Child,SCHIP, Medicare Managed Care
- 2004 Health Plan Data: Over 220 commercial plans (N=111,680 adults, N=7,024 children), 149 Medicaid plans (N=59,515; N=86,159), and 288 N=134,420Medicare plans
- Plans to include comparable data from hospitals, dialysis facilities, and individual clinician level

## How to Get Access to NCBD 

- Two major data sets: CAHPS Survey Response Data, and Survey Administration and Health Plan/Provider Characteristics Data
- All survey information exclusive of Rs identifying information. Mode and language of administration and descriptive information (state, zipcode).
- Survey administration, response rates, dates of completion, type of organization, size of enrollment, tax status and ownership and location
- Written request to Executive Research Committee of the NCBD Advisory Board including research project, data files, analytic methods, data storage, products from research

## Resources Available for Free: SUN

- CAHPS Survey Users Network: _[www.cahps-sun.org](http://www.cahps-sun.org/)_, email _[cahps1@westat.com](mailto:cahps1@westat.com)_, phone1-800-492-9261
- Free, limited, one on one technical assistance
- Easy access to all CAHPS products, resources
- Updates on product development, new tools
- Guidance on using survey instruments and results
- Opportunities to network with Consortium and peers

## Comments, Questions?

- Beth Kosiak
- Bkosiak @ahrq.gov
- 301-427-1322

## What Tools Does AHRQ Have? Patient Information

- Your Medicine: Play It Safe
- A 12 page brochure outlines steps to help patients use prescription medicines safely
- Includes detachable pocket-sized medicine record form

## What Tools Does AHRQ Have?Patient Information

- Five Steps to Safer Health Care—Cut down from 20. Extremely popular.  Disseminated by Dept. of Labor, Office of Personnel Management, Medicare program, others
- Ask questions if you have doubts or concerns
- Keep and bring a list of ALL the medicines you take
- Get the results of any test or procedure
- Make sure you understand what will happen if you need surgery

## What Tools Does AHRQ Have?Patient Safety Improvement Corps

- AHRQ and VA partnership, states apply
- Teams of state health staff and hospitals (up to total of 4 people)
- Three 1 week sessions spread over several months and homework
- Provide knowledge and skills: conduct investigations of reports of medical error, prepare reports on findings, develop and implement interventions, measure and evaluate impact, ensure sustainability of effective interventions

##  What Tools Does AHRQ Have?Hospital Survey on Patient Safety Culture

- Use to establish baseline of patient safety culture at your hospital
- Identify areas for improvement
- Free and in the public domain
- Provides advice on how to draw sample, field survey

## Other AHRQ Tools for Quality and Patient SafetyCAHPS Surveys

- Patient perspective on quality of care—reports and ratings of patients experience of care—_not_ the same as satisfaction
- Competitive award to 3 grantees:Harvard, RAND, and AIR (RTI in 1st grant cycle). Started in 1995
- Surveys with guidance on how to draw patient sample, administer survey and report results, free and in the public domain; can administer by phone
- CAHPS written at 6th grade level
- Website cahps-sun.org: All survey tools on line, and limited free individual technical assistance
- 130 million Americans in health plans that use CAHPS surveys. NCQA adopted as HEDIS standard, Medicare fields nationally every year

## CAHPS Surveys 

- Managed care and fee for service health plans (Medicare,Medicaid and commercial)
- Hospitals (HCAHPS),
- Dialysis facilities (ICH CAHPS),
- Individual clinician/group practice (ACAHPS)
- Behavioral health (ECHO)
- American Indian (AI CAHPS)
- Nursing Home Resident, Nursing Home Family (NHCAHPS)
- Persons with Mobility Impairments (PWMI CAHPS)
- Assisted Living
- Home Health

## CAHPS Surveys and Patient Safety

- HCAHPS asks about medications
- ICH CAHPS asks about changing of gloves, monitoring of dialysis machine

## CAHPS and Health Literacy

- Almost all CAHPS surveys ask if doctors/nurses/staff “explained things in a way you could understand”
- Selected surveys include questions about explanation of blood test results, availability of translators, etc.

## Other AHRQ Tools for Quality and Patient SafetyCAHPS Quality Improvement Guide

- Created by Harvard team under contract to Medicare, originally to help health plans use their survey results for QI
- Web-based tool, will undergo revision in coming year to expand beyond health plans
- Gives practical guidance on how to identify an area for QI using patient survey results, design an intervention, implement, and evaluate

## Other AHRQ Tools for Quality and Patient SafetyEvidence Based Practice Centers (EPCs)

- AHRQ contracts with 13 centers that conduct comprehensive reviews of the literature on a topic. Used for technology assessment and coverage decisions, identification of safe practices, breast cancer quality measures, etc.
- Topics nominated by professional organizations, health plans, employers, patient groups. AHRQ panel reviews and chooses topics each year to support with its own funds
- Currently an EPC is working on a “Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies.” Evaluating any tool or process to reduce the quality gap.

## Other AHRQ Tools for Quality and Patient SafetyNational Quality Measures Clearinghouse www.qualitymeasures.ahrq.gov

- Over 592 quality measures, updated weekly, on a wide range of clinical and patient experience areas
- Summaries of the measures themselves and the evidence supporting them, provided
- User obtains detailed specifications from submitter

## Other AHRQ Tools for Quality and Patient SafetyQuality Tools www.qualitytools.ahrq.gov

- Provides tools useful for quality improvement projects
- Provides links to other sites with quality tools
- Provides guidance on how to go about choosing a quality measure

## Other AHRQ Tools for Quality and Patient Safety AHRQ Electronic Newsletter 

- Regular updates on breaking news from AHRQ supported research
- Information on conferences, meetings, etc.

## Other AHRQ Tools for Quality and Patient SafetyTalkingQuality www.talkingquality.gov

- “Soup to nuts,”step by step instructions to those who are interested in providing comparative quality performance to consumers
- Over 200 examples of actual reports provided through weblinks

## And last, but not least...AHRQ awards grants to researchers and practitioners

- Consider AHRQ’s Patient Safety and Quality grants that are focused on implementation of practice
- Feel free to contact anyone at AHRQ for help
- All our email addresses are: 1st initial, last name, at ahrq.gov
- bkosiak@ahrq.gov