# Presentation: 404454

## What Can Patients Tell Us To Improve  In-Center Hemodialysis Care?&What Can Patients Tell Us About the Cultural Competency Of The Care They Receive?

**Margarita Hurtado, PhD, MHS**
**American Institutes for Research (AIR)**

***AHRQ 2007 Annual Conference***
***Rockville, September 27, 2006***

## Using CAHPS Surveys to Improve the Dialysis Experience

## Acknowledgements: ICH-CAHPS QI Team

- AHRQ:
- David Stevens
- Beth Kosiak
- CAHPS Grantees:
- AIR: Margarita Hurtado
- RAND: Denise Quigley and Donna Farley
- Harvard: Paul Cleary & Susan Edgeman-Levitan
- Westat: Lise Rybowski
- CMS: National and ESRD Network Staff

## Patient Interviews: A “Typical” Day at the Dialysis Center

- “I’m in very early in the morning and I usually don’t stay in the waiting room very long. [I am in] as soon as they open the door. The first thing we do is weigh us. And then they put us in a chair, give us a blood pressure test, and then we get the needles put in. We get a blanket.... I’m on for 3 hrs and 15 min., and then another 1⁄2 hour before the bleeding stops.” 							- A patient

**Notes:**
I also want to of course acknowledge, the dialysis patients. Without them this project would not have been possible and it is for them and with them that we are carrying out this effort.

## The ICH-CAHPS QI Demonstration

- Examined Using CAHPS Survey Data for QI
- 4 ESRD Networks: New England(#1), Mid-Atlantic (#5), Texas (#14), Intermountain(#15)
- 7 Dialysis Centers: 2 partner w/ each Network/Grantee team; 1 with Network 15
- 3 Grantees: American Institutes for Research, RAND, Harvard
- Products:
- Provider Report
- QI Projects
- Lessons Learned Report

## Pre and Post CAHPS Survey Measures
Interviews with Network staff
2 site visits to dialysis centers
Interviews with dialysis center staff & patient representatives
Lessons learned report across all sites

- Interviews with Network staff
- 2 site visits to dialysis centers
- Interviews with dialysis center staff & patient representatives
- Lessons learned report across all sites

**Evaluation of the QI Demonstration**

**Notes:**
Dialysis center staff interviewed: 
Social workers, clinical managers, charge nurse, nephrologists, dietician, and administrators

Interview topics:
Defining QI intervention; how it was implemented; experiences of implementation; how QI decisions are made; facilitators and challenges to the project

## Participating Dialysis Centers

**Experience with quality improvement**. 4 with limited experience, 2 with moderate experience, and 1 with extensive experience
**Size** (13 - 216 chairs). 1 small, 2 mid-size, and 4 large centers
**Geography**. 3 in rural towns, 2 mid-sized rural towns, 1 in mid-sized urban town, 2 in mid-sized cities, and 1 large  city (inner city)
**Race/ethnicity of patients**. 3 mostly white with some African Americans, 2 mixed, 1 mostly Latino, 1 mostly African-American and Hispanic
**Ownership**. 5 were owned by chains, 1 was independent, 1 was owned by a hospital

## QI Project Objectives

- Obtain a more complete picture of the quality of dialysis care by including patients’ perspectives
- Develop a report on CAHPS survey results useful to providers for QI
- Examine the usefulness of the CAHPS survey results for QI purposes
- Document the quality improvement strategies and interventions the dialysis centers carried out based on CAHPS survey results
- Compile lessons learned

**Notes:**
It is important to remember that the CAHPS survey is one of several sources of information for the project.
One source that can provide information about potential areas for improvement and one that can provide the “big picture” or “bird’s eye” view of  patient experiences of care over time. CAHPS survey data needs to be complemented with other sources of information at the ground level, that are specifically tailored to each of your projects and can provide more detailed and contextual information about the activities and interventions you decided to implement.

## What Did We Ask Patients About Their Dialysis Care?

**Summary Measures**
- Nephrologists’ Communication and Caring (7 items)
- Quality of Dialysis Center Care and Operations (22 items)
- Staff Communication and Caring
- Staff Professionalism and Competence
- Dialysis Center Operations
- Providing Information to Patients (11 items)
**Ratings **
- Nephrologists
- Staff
- Center

**Notes:**
We included a lot of questions about aspects that are important to patients and obtained input from those most familiar with dialysis care. As you can see the survey covers many important areas. 

However, I do want to point out that the CAHPS survey MAY not cover all the aspects that you expected to change or observed change in. For example, in some centers the QI activities resulted in better relationships and communication among staff and this is not something we ask about in the CAHPS survey since we are focusing on what patients know best. So, the CAHPS survey results most likely provide information on some key aspects related to your QI project but they do not provide all the information you need. 

## Slide 10

## Using CAHPS for Quality Improvement: The QI Cycle (PDCA)

##  1. Plan: Identify Opportunities for Improvement and Plan Strategy

- Examine CAHPS survey results (and any  differences by population subgroups)
- Define potential opportunities for improvement from CAHPS survey
- Define areas where need additional information to define specific areas for improvement
- Establish QI team
- Establish goals for improvement
- Investigate potential strategies

## PLAN: Define potential areas for improvement 

**PLAN: **Define potential areas for improvement

**How Often Dialysis Center Staff Cared About Patients as People**
- Your center **can improve** compared to the national average. Patients were asked, “how often did you feel that the dialysis center staff really cared about you as a person?” The proportion of patients at your center who said “always” was 26% lower compared to the national average and 64% lower compared to the average of the top 10% of dialysis centers.

**Notes:**
Example: One center focused on  questions about whether staff cared for them as a person.  This center then administered a short questionnaire to patients that asked them what would make them feel that they were treated like a person.

On ITEM:
For Hospital Survey, the composite most highly correlated with hospital rating and willingness to recommend to friends or relatives was “Communication with nurses”- respect, listening, explaining

Other important factors to consider when defining opportunities for improvement
Ability to identify tangible actions that can be taken to improve performance
Probability of quality improvement success with those actions

## 2. Do- Develop and Test Strategy

- _Define specific intervention(s)_
- _Define measures to monitor progress, criteria and benchmarks_
- Implement interventions
- Adapt/adjust interventions to organizational context

**Notes:**
Example:  One center focused on communication and perception of caring based on the CAHPS survey results. To address the issues, the center trained staff on communication and provided them with scripts with answers to common questions. 
They also added a formal complaint process so that patients could be assured that their complaints would be addressed.

## 3: Check Progress and Monitor Strategy

- Gather data on measures selected to monitor progress
- Analyze data on measures to monitor progress
- Evaluate progress made against predefined criteria
- Implement further changes as needed

**Notes:**
Example: One center focused on caring about people and developed several small activities, such as calling patients by name. This center conducted a short survey at different points in time to assess whether staff addressed patients by name, treated them with respect, and whether they were put on a machine in a timely manner.

## Step 4: Act-Reassess Strategy & Respond

- _Implement CAHPS survey again and examine changes in scores for areas chosen_
- Assess what worked and what didn’t work
- Modify strategy, if needed
- Introduce improvements into the organization on a broader scale, as appropriate
- Start the cycle again

**Notes:**
Example: One center focused on connecting patients to the machine within 15 minutes. The second CAHPS survey showed an improvement, although not statistically significant. This center added to the timely dialysis start, the concept that theentire center had to run in a timely manner.

## Change in Patient Ratings of Dialysis Staff for One Center

**Notes:**
2005 n= 340; 2006 n=271; Top Center n=9 [Covington]

Here, similar to what we saw for the summary measures, the rating seems to have increased rather than decrease which is what we saw with the nephrologists. The proportion of patients who rated staff poorly appears to have decreased and the ones how rated them highly appears to have increased.

However, we cannot be confident that this is not a blip in the data.

## “The change I’ve noticed in the last 3 months is that I know the staff better than I did.”

“We all fuss because we want individual TVs... I have no problem with anything we’ve had from the staff, they’re all really nice.”

“They do what they can to relieve problems here, and if they can’t handle it, they send us to the hospital.”

- “We all fuss because we want individual TVs... I have no problem with anything we’ve had from the staff, they’re all really nice.”
- “They do what they can to relieve problems here, and if they can’t handle it, they send us to the hospital.”

**What Did Patients Say After the QI Project?**

## Using CAHPS Surveys for QI: Lessons Learned I

- CAHPS surveys can provide useful information for QI
- Need to plan for QI **before** the survey (over-sampling; supplemental QI items)
- Consumer reports are not appropriate for QI; need item-level information
- Benchmarks should include top performers, not just averages

## You may encounter resistance with respect to use of patients reported data for QI- be prepared with evidence on CAHPS surveys validity and reliability
LDOs usually define areas for QI and strategies and focus is on clinical measures
Most providers have not focused QI on patient experiences. They will need training
Involve patients and staff in the QI process- they know best
It is very difficult to change organizations. It takes time and perseverance.

- LDOs usually define areas for QI and strategies and focus is on clinical measures
- Most providers have not focused QI on patient experiences. They will need training
- Involve patients and staff in the QI process- they know best
- It is very difficult to change organizations. It takes time and perseverance.

**Using CAHPS Surveys for QI: Lessons Learned II**

## Measuring Cultural Competency From The Patient’s Perspective

## Acknowledgements: CAHPS CC Team

- AHRQ:
- Charles Darby
- CAHPS Grantees:
- AIR: Margarita Hurtado
- Harvard: Karen Bogen
- RAND: Beverly Weidmer*, Quyen Ngo-Metzer, Robert Weech-Maldonado
- *Lead for the intergrantee team

## Developing the Cultural Competence  Modules

- Literature Review
- Development of a Conceptual Model*
- Instrument Review (incl. CAHPS)
- Call for Measures
- Item Development and Translation
- Cognitive Testing
- Field Test
- * Ngo-Metzer et al. 2006. Cultural competency and quality of care. Available at www.cmwf.org

## Conceptual Framework for Culturally Competent Care

**A**

**Patient Factors**

**Provider Factors**

**Health Care System Factors**

**B**

- A. 	Patient provider communication
- 	Respect for patients/Shared decision-making
- B. 	Experiences leading to trust/distrust
- 	Experiences of Discrimination
- 	Linguistic Competency (incl. health literacy)

## Cognitive Testing	

- Conducted in-person interviews with 18 participants in May 2007
- Participants included a mix of men and women,  level of education, age, race/ethnicity, country of origin (for Spanish speakers)
- Interviews conducted in English (9) and Spanish (9)
- Tested items as a self-administered survey and interviewer-administered survey

## Overview of Items Tested	

- Language Access (10 items): Use and rating of interpreters, discrimination by language
- Patient-Doctor Communication (10 items): Health and mental counseling, use of complementary/alternative medicine
- Shared Decision Making (7 items): Alternatives, preferences, values
- Discrimination (12 items)*: Insurance, race/ethnicity, language, source, how knew
- Trust (7 items): truth-telling, caring, level of trust, rating, cared for you as a person
- Health Literacy (41 items):** Tools; problems/concerns; self-management; medicines; forms; test results; navigation
- * One of these items is added to the language access
- **Separate project focused on this domain

## Revised Cultural Competence Modules and Items 	

- Language Access (14 items)
- Patient-Doctor Communication (10 items)
- Shared Decision Making (5 items)
- Discrimination (3 items)*
- Trust (6 items)
- Health Literacy (30 items)
- * 1 of these items is added to the language access module

## To Learn More About CAHPS QI and CC visit:

- www.cahps.ahrq.gov

## Slide 29

## Questions?

- Margarita Hurtado, Ph.D., M.H.S.
- American Institutes for Research
- mhurtado@air.org
- Tel. (301) 592-2215