# Presentation: 104334

## Development of Home Care Quality Indicators Based on the MDS-HC

- Brant E. Fries, Ph.D.
*University of Michigan*
- May 7, 2002

**Notes:**
Development of Home Care Quality Indicators Based on the MDS-HC, a presentation by Brant Fries of the University of Michigan. Presented at the AHRQ-NGA meeting, State Long Term Care Systems: Reducing Cost, Quality and Access, held in Indianapolis, IN May 6-8, 2002.

## Agenda

- RAI-HC as the basis for Quality Indicators
- Home Care Quality Indicators (HCQIs)
- Development
- Summarizing HCQIs
- Use of HCQIs in evaluating the MI Choice Programs

**Notes:**
I will discuss the use of quality indicators for the Michigan MIChoice Program. 

## Agenda

- RAI-HC as the basis for Quality Indicators
- Home Care Quality Indicators (HCQIs)
- Development
- Summarizing HCQIs
- Use of HCQIs in evaluating the MI Choice Programs

**Notes:**
The RAI home care assessment tool was developed by clinicians.

## RAI-Home Care Assessment System

- Developed by inter*RAI*, a multi-nation group of clinicians, researchers and policymakers
- Community analogue to the RAI, mandated in U.S. nursing homes

**Notes:**
The RAI-HC was developed by a multi-national groups of clinicians, researchers and policymakers to measure program improvements.  

## Improvements in the RAI

- Primary purpose:
- Improve care plans through improved 		assessment

**Notes:**
Everything should be designed around improving care plans through assessment. 

## Improvements in the RAI

- Three parts of the RAI-HC
- Minimum Data Set (MDS-HC)
-  Triggers
- Client Assessment Protocols (CAPs)
- 	(Care planning guidelines)

**Notes:**
Differentiate between RAI and MDS. The three parts include the MDS for home care. There are care planning guidelines around clinical problems (environment, ADLs, and others). Both components are linked by algorithms called triggers that indicate whether a person has a problem. We like think we “put assessors at risk of doing the right thing.”

## Improving Assessment Process

- Items clearly defined, including:
- full definitions
- examples and exclusions
- time delimiters
- Cover all relevant domains
- individuals’ strengths and weaknesses
- tradeoff of breadth/depth and length

**Notes:**
Items have to clearly identified and have clear definitions for terms.  We give examples of terms and situations and their place in the system such as when a person is dependent or independent.  You have got to have clear questions so two interviewers get the same answers to the same questions.  You need specific time delimiters - is it in the last day, last 7 days, last month?  You also need to cover all domains, strengths as well as weaknesses. They is always a trade off breadth and depth. If everything is done right, everyone is mad at us because one area can’t be covered in enough detail to satisfy professionals in that area. This is never the end of an assessment, only the beginning, core items to do an assessment.

## Improving Assessment Process

- Use all possible sources of information
- individual, formal/informal caregivers, MD, medical record, etc.
- self-reporting may be inaccurate
- assessor decides when sources are 		inconsistent

**Notes:**
This process uses all sources of information.  We need to triangulate multiple sources of information to determine the right answer given disparate information. 

## Improving Assessment Process

- Careful testing of psychometric properties
- Training manual
- Ongoing refinement - RAI-HC Version 2

**Notes:**
We need to carefully test psychometric properties such as sensitivity, inter-rater reliability. A training manual is there to answer questions and improve reliability. Any instrument lives, breathes, and changes over time.

## Applications of MDS-HC Data

**ASSESSMENT**

**Care Plan (CAP)**

**Eligibility Systems**
**(MI Choice)**

**Quality Measures**
**                   ****(HCQI)**

**Case-Mix Algorithm**
**             ****(RUG-III/HC)**

**Notes:**
Remember we are doing this to better care planning and to provide care for specific clients, not to collect blind data. We can use the data collected blindly later.  Other applications may be feasible after the primary use of the assessment information - providing care plans.

## RAI Family of Instruments

- Chronic care/nursing homes	RAI 2.0
- Home Care				RAI-HC 2.0
- Mental Health				RAI-MH
- Acute Care				RAI-AC
- Post-Acute Care-Rehabilitation	RAI-PAC
- Assisted Living				RAI-AL
- Palliative Care				RAI-PC

**Notes:**
This is the family of survey instruments. 

## Slide 12

**Notes:**
These instruments were designed to deal with the Tower of Babel effect -- everyone with different ideas.  We must find a common basis for applications.

## Common Basis

- All inter*RAI *instruments have common basis of care planning
- Major items in common
- Possible to link across time and setting
- Start of a “language” to describe long-term care users

**Notes:**
We start with a common basis for care planning, ensuring that each tool has major items in common with other tools so that items are measured the same way on different instruments and follows care across instruments, without developing a measurement effect. 

## Implementation of RAI-HC

- Inter*RAI* grants royalty-free license to governments
- Adopted by 10 states, Department of Veterans Affairs
- International adoptions
- Used in fee-for-service and managed care programs

**Notes:**
InterRAI grants a royalty-free license to governments and technical support groups.  It has been adopted by 10 States, and the U.S. Department of Veterans Affairs. RAI has international applications (developed and used in Canada, Japan, and Switzerland). It has also been used for fee for service and managed care organizations -  the tools are independent of a financing modality.

## Agenda

- RAI-HC as the basis for Quality Indicators
- Home Care Quality Indicators (HCQIs)
- Development
- Summarizing HCQIs
- Use of HCQIs in evaluating the MI Choice Programs

**Notes:**
RAI has helped with development of home care quality indicators.

## Uses of MDS-HC Data for Quality Measurement

- User Profiles
- Whom are we serving?
- Performance Benchmarks
- Are we serving the “right” people?
- Outcome Measures
- What happens to the people we serve?
- Quality Indicators
- How do care strategies affect the people we serve?

**Notes:**
MDS-HC data is used for a variety of quality which is a secondary use of data and helps answer: who are we serving, are we serving the right people? The tool utilizes user profiles and benchmarks from the data. We can look at outcome measures, what happens to the people served over time.

## Why HCQIs Are Important

- HCQI= Home Care Quality Indicators
- Citizens, legislators, administrators want “proof” that programs work

**Notes:**
Quality indicators are important tools for helping the public, legislators and managers understand how the programs are working and who they serve. 

## Uses of HCQIs

- Regulation
- Who is doing a substandard job?
- Management
- How well am I doing?  Compared with last year?
- Consumers
- Where should I get care?
- Best practices
- Who is doing an outstanding job?
- Benchmarking
- How do I compare with others?

**Notes:**
Home Care Quality Indicators (HCQIs). There are a variety of reasons why people want to know whether what they do is working.  Which providers are good or substandard, which are the best providers to use, who has outstanding best practices? Who is doing a good job and how do I compare to others? It is only by comparison to others that you see how you are doing. 

## HCQI Authors

**John P. Hirdes Ph.D.**
**Brant E. Fries Ph.D.**
**John N. Morris Ph.D.**
**David Zimmerman Ph.D.**
**Naoki Ikegami M.D., Ph.D.**
**Dawn Dalby M.Sc.**
**Suzanne Hammer M.Sc.**
**Pablo Aliaga M.Sc.**
**Rich Jones, Ph.D.**

**Notes:**
The authors of the instrument you can see here on the slide. Not on list is Mary James.

## Considerations in Developing HCQIs

- Reliability and validity of data items
- Points of comparison
- Prevalence, incidence
- Validity of indicators
- Application – when agency is responsible
- Prevalence: follow-up data only
- Incidence: intake to follow-up

**Notes:**
During development, we picked only reliable and valid indicators. If we depend on other items that are not reliable or valid, the instrument is not useful.  We found 2 points of comparison – prevalence and incidence.  Prevalence shows whether a person has a problem. Incidence is more sensitive (did someone gain a problem under your care) and therefore is a more important measure.  The validity of indicators is also essential.  Prevalence is only follow up data not admission data, which represents what happened before.  Incidence covers intake to follow up time under your control  (three month period).

## HCQI Research in a Nutshell

- Two year effort in Canada, USA, Japan
- Involved many stakeholders
- Started with QIs from other sectors
- Workgroups in Canada and Michigan
- Identification of exclusions
- Analysis with data from Canada, US, Italy
- HCQIs with reasonable prevalence
- Adjustments

**Notes:**
Development was a 2 year effort that included Canada, Japan and the US, as well as many stakeholders and multiple governments.  We started thinking about what QIs we already have to see what is useful in health care. Work groups then went through the list and discovered other QI measures that might also be useful.  We also consider who might be excluded from the population being evaluated.  Weight loss evaluation for terminally ill is not a useful indicator.  We insisted on analysis of data that worked in multiple populations. If data didn’t work in at least two populations, it was not considered a stable indicator.  Technical adjustments such as these are critical in developing useful QIs for practice.

## Prevalence HC Quality Indicators

-  Nutrition
- Inadequate Meals
- Weight Loss
- Dehydration
- Pain
- Disruptive/Intense Pain
- Unmanaged Pain
- Physical function
- No Assistive Device for Clients with Difficulty in Locomotion
- ADL/Rehabilitation Potential and No  Therapies

- Psychosocial function
- Social Isolation with Distress
- Delirium
- Negative mood
- Medication
- No medication review
- Safety/Environment
- Falls
- Any injuries
- Neglect/Abuse
- Other
- No Influenza Vaccination
- Hospitalization

**Notes:**
Here are lists of the quality indicators we developed. 

## Incidence HC Quality Indicators

- Psychosocial function
- Failure to improve/ incidence of cognitive decline
- Failure to improve/ incidence of difficulty in communication
- Other
- Increased health instability

- Incontinence
- Failure to improve/ incidence of bladder continence
- Ulcers
- Failure to improve/ incidence of skin ulcers
- Physical function
- Failure to improve/ incidence of decline in ADL
- Failure to improve/ incidence of impaired locomotion in the home

**Notes:**
Incidence quality indicators measure changes that occur after a person enters a program and include incontinence, ulcers, physical functioning and psycho-social function.

## Adjusting HCQIs

- Risk adjustment
- Should we adjust?
- Team identified candidate risk adjusters
- Analyze Ontario, Michigan and Italian data:
- Adjustment in same direction/ magnitude in 2 out of 3 countries

**Notes:**
Risk adjustment is controversial.  If your population is higher risk, should you be adjusted for additional risk or should you put programs in place to adjust for your heavier risk population?  But it may help you explain away things you could be addressing. We do risk adjustment for things that are not under control of the agency. 

## Example: Two Nutrition HCQIs

**Notes:**
Here is an example of two HCQI’s with risk adjusters.

## Adjusting HCQIs

- Selection/Ascertainment adjustment
- Should we adjust?
- Use intake rates to derive agency-level measure of bias
- Analysis of Ontario and Michigan data

**Notes:**
Selection or ascertainment bias is also controversial. If your staff are well trained, the agency is likely to pick up issues that an agency with less trained staff miss. The agency doing a good job looks worse and the agency doing the poorer job will not be identified as such. We can use intake data to give a baseline for further comparisons. Selection means you take or avoid people with conditions you identify. Ascertainment bias covers conditions you should have noted during an assessment. 

## Risk/Ascertainment Adjustments for Mood, 8 Michigan Agencies

**Notes:**
This slide shows mood unadjusted illustrated against mood adjusted.  There is not a lot of difference with risk adjustment but ascertainment adjustment shows great difference in risk. We believe these two adjustments are important to make.The highest agency did a very good job in identifying people with mood problems. Without an ascertainment adjustment, they look far worse than other agencies. 

## Two HCQIs, by Agency

**Disruptive/intense daily pain**

**Delirium**

**Notes:**
This shows adjustment for 2 HCQI’s by agency. Most of Canada seems to be doing better on some measures.  Every country does some things well, some things bad, and the rest in the middle. Every country has something to bring to the table.This process helps establish benchmarks that agencies can use to determine their progress and compare to others. 

## All HCQI – Agency “A”

**Notes:**
Summary of HCQI using a radar plot as one visual representation.  Risks are clumped and any point shows an agency peaking somewhere. The red area is the average incidence and the yellow the actual for each indicator. The more red, the better job the agency is doing. 

## All HCQI – Agency “B”

**Notes:**
This is another radar summary of HCQI as visual representation  It shows a different pattern in terms of what the agency does well. 

##  People want simple quality measures

- Good Housekeeping Seal
- Consumer Report Circles
- Olympic Medals
- Michelin Stars

**Notes:**
People want simple measures, and we are struggling with how to do that. 

## Average Relative QIHC, by Michigan Agency

**Notes:**
One way to do it is by averaging.  But one agency that shows close to one as an average has mostly average scores, while the next averages out some exceptionally good and exceptionally bad indicators. Averages lose  important information for evaluation.

## Single Measure of Home Care Quality 

- People want simple, but...
- We lose critical information
- May not be feasible
- When we present multiple measures...
- Difficult to interpret
- Still seeking good “views”

**Notes:**
One measure leaves a lot out and loses critical information but too many measures becomes difficult to interpret.

## Agenda

- RAI-HC as the basis for Quality Indicators
- Home Care Quality Indicators (HCQIs)
- Development
- Summarizing HCQIs
- Use of HCQIs in evaluating the MI Choice Programs

**Notes:**
In Michigan we have evaluated the use of RAI for home care, training, and CAPS for care planning planning. Early data has just collected information. The grant allowed us evaluate the data. 

## Are you just pissing and moaning, or can you verify what you’re saying with data? 

*Are you just pissing and moaning, or can you verify what you’re saying with data? *

**Notes:**
This cartoon captures a lot of what I’m talking about.

## Methods

- Used adjusted HCQIs
- 23 agencies
- Over 8 quarters, from Jan 99 to Dec 01
- Training and computerization in 2nd quarter

**Notes:**
We adjusted the indicators for risk and ascertainment selection bias for 23 agencies  and looked at data over eight quarters. 

## Change in Agency Average HCQI Score, by Period

- Worse

**Notes:**
This graph shows changes in an agency’s average performance.  These show the major results.  We looked at each QI and standardized all of them to one for the first quarter.  Then we measured indicators over time. Indicators that grew worse appear above the 100% line. Improvement is presented by a line dropping below 100% which shows a reduction in incidence. A couple are higher, some the same, and there are a bunch that are much lower (better) two years later. 

## Results

- Over 8 periods (2 years) – (p<.005)
- 16 HCQIs improved (e.g., mood, falls, hospitalizations, weight loss, social isolation, decubiti)
- 4 HCQIs remained the same (e.g., pain, disruptive pain, injuries, no assistive dev.)
- 2 HCQIs worsened (intense pain, rehab potential without therapies)

**Notes:**
This slide shows the results of our study. We saw significant changes over a two year period. Sixteen of the indicators improved, four stayed the same and two got worse. 

## Defining Good /Poor Quality

**GOOD**

**POOR**

**Notes:**
This is how one indicator would look along a bell curve – good and poor tail. You can charting this curve for each indicator and add up all the good versus poor tails to get a sense of agency performance. . 

## Average “Good”/ “Bad” HCQIs, by Quarter

**Bad**

**Good**

**RAI**
**Training**

**Notes:**
This shows average of good and bad indicators from all agencies.  Bad declined, but good indicators improved marginally. More people moved to the middle. This was not a surprising result. 

## Distribution of a HCQI

**GOOD**

**BAD**

**Notes:**
This slide shows indicators shifted from bad to the middle, but did not show more indicators falling in the good area. When this shift is seen in sectors, it indicates a good movement away from bad.

## Next Steps

- Further validation of HCQIs
- Develop archives for benchmarking
- Applicability to subpopulations
- Quality of Life?

**Notes:**
Our next steps include a further validation of the indicators, and we want to build benchmarking archives. We will also examine their applicability to sub-populations as some indicators make sense and some do not when a larger population is divided into other groupings.  Quality of life issues are not addressed by these quality of care indicators.

## Conclusions

- RAI-HC has potential to improve care directly, through improved care planning
- MDS-HC has multiple uses, including measuring quality of care
- HCQIs can be used to monitor care
- Directly computed from MDS-HC
- Useful for comparisons, benchmarking

**Notes:**
RAI-HC is a very useful tool for these purposes.