# Presentation: 579027

## Measurement for Improvement

- Readiness Curriculum Call
- Jerry Langley, IHI
- August 27, 2008

## Model for Improvement

- What are we trying to

- accomplish?

- How will we know that a

- change is an improvement?

- What change can we make that

- will result in improvement?

## Model for Improvement

- What are we trying to

- accomplish?

- How will we know that a

- change is an improvement?

- What change can we make that

- will result in improvement?

- Data to answer this question

## Measurement Guidelines

***Measurement Guidelines***

**The key measures plotted and reported each month should clarify your team’s aim and make it tangible.**
**Make use of your patient population data base (RPMS) for measurement.**
**Integrate measurement into the daily routine.**
**Plot data on the key measures each month during the collaborative. (the extranet will help you with this)**
**The question - *****How will we know that a change is an***** *****improvement? *****usually requires more than one measure. A balanced set of measures will ensure that the system is improved.**

## 		API/IHI 2008

## 		API/IHI 2008

## Types of Measures

- Project level
- Process measures to support PDSA

## Model for Improvement

- What are we trying to

- accomplish?

- How will we know that a

- change is an improvement?

- What change can we make that

- will result in improvement?

- Data to support the PDSA Cycle

- Data to answer this question

## The PDSA Cycle for Learning and Improvement

**The PDSA Cycle for Learning and Improvement**

**Act**

-  What changes
-    are to be made?
-  Next cycle?

**Plan**

-  Objective
-  Questions and
-    predictions (why)
-  Plan to carry out
-    the cycle (who,
-    what, where, when)

**Study**

-  Complete the
-    analysis of the data
- Compare data to
-         predictions
- Summarize what
-    was learned

**Do**

-  Carry out the plan
-  Document problems
-    and unexpected
-    observations
-  Begin analysis
-    of the data

## Aim: Improve primary care appointment availability through reducing and standardizing appointment types

- Aim: Improve primary care appointment availability through reducing and standardizing appointment types

**Reduction of appointment types will increase appointment availability**

**Improved access**

**DATA**

- Cycle 1:Define a small number of appointment types

- Cycle 2:Compare requests to the types for one week

- Cycle 3:Test the types with 1-3 physicians

- Cycle 4: Standardize appointment types

- Cycle 5: Staff education in
-                new standards

## PDSA Cycle Measures

**PDSA Cycle Measures**

- In addition to the family of measures reported each month in IPC, specific measures will be required to determine and document the success of PDSA Test and Implementation Cycles:
- 	Additional specific measures
- 	Stratification of Global measures
- These temporary cycle measures are not normally reported out

## Effective Use of Data

- Seek usefulness not perfection
- Develop a balanced set of measures
- Plot data over time
- Use pencil and paper until the information system is ready
- Use sampling
- Use qualitative and quantitative data

## Sometimes gathering  data can bring new and surprising  knowledge! 

**Sometimes gathering  data can bring new and surprising  knowledge! **

## Minimum Standard for Monthly Reporting in the Collaborative:Annotated Time Series

- Minimum Standard for Monthly Reporting in the Collaborative:**Annotated Time Series**

## 		API/IHI 2008

## 		API/IHI 2008

**Notes:**
Put several plots of key measures with annotations and perhaps a plot of one shorter term measure that was used to support a couple of cycles. Use real examples from your teams.

## Measures for Chronic Care

**Measures for Chronic Care**

## Family of Measures for Chronic Care

**Family of Measures for Chronic Care**

## Three Different Purposes for Measurement

| Aspect | Improvement | Accountability | Research |
| --- | --- | --- | --- |
| Aim: |  | 
 |  |
| Methods:
 |  |  |  |
| Bias: |  |  |  |
| Sample Size: |  |  |  |
| Flexibility of
Hypothesis: |  |  |  |
| Testing Strategy: |  |  |  |
| Determining if  a Change is an
Improvement: |  |  |  |
| Confidentiality of
the Data: |  |  |  |
| Frequency of Use: |  |  |  |

**Improvement of care**

- Comparison, choice, reassurance

- New knowledge

**Test observable**

- No test, evaluate current performance

- Test blinded or controlled

**Accept consistent bias**

- Measure and adjust to reduce bias

- Design to eliminate bias

**“****Just enough” data, small sequential samples**

- Obtain 100% of available, relevant data

- “Just in case” data

**Hypothesis flexible, changes as learning takes place**

- No hypothesis

- Fixed hypothesis

**Sequential tests**

- No tests

- One large test

**Run charts or Shewhart control charts**

- No change focus

- Hypothesis, statistical tests (t-test, F-test, chi square, p-values)

**Data used only by those involved with improvement**

- Data available for public consumption and review

- Research subjects’ identities protected

**Daily, weekly, monthly**

- Quarterly, annually

- At end of project

## Average percent of patients screened for colorectal cancer has improved from around 30 percent in March 2007 to 55 percent in June 2008. 

- Average percent of patients screened for colorectal cancer has improved from around 30 percent in March 2007 to 55 percent in June 2008.
- Average size of the population of patients has remained steady.
- What about the learning occurring at each of the pilot sites. To facilitate this kind of learning, a set of “small multiples” is created  to display the information for all the sites separately, yet still on one page.

## 		API/IHI 2008

## Diabetes Comprehensive Care

| Numerator: 
1) Patients with hemoglobin A1c documented during the Report Period, regardless of result.
2) Patients with blood pressure documented during the Report Period.
3) Patients with controlled blood pressure during the Report Period, defined as < 130/80.
4) Patients with LDL completed during the Report Period, regardless of result.
5) Patients with nephropathy assessment, defined as an estimated GFR and a quantitative
      urinary protein assessment (changed from positive urine protein or any microalbuminuria) during the Report Period OR with evidence of diagnosis and/or treatment of ESRD at any time before the end of the Report period.
6) Patients receiving any retinal screening during the Report Period, or a documented refusal of a diabetic eye exam.
7) Patients with diabetic foot exam during the Report Period, or a documented refusal of a diabetic foot exam.
8) Patients with A1c AND Blood Pressure AND LDL AND Nephropathy Assessment AND Retinal exam AND Diabetic Foot Exam.  

Denominator: 
Active Clinical patients diagnosed with diabetes (POV 250.00-250.93) at least one year prior to the Report Period, AND at least 2 visits in the past year, AND 2 DM-related visits ever.

SAME DATA GATHERING PLAN AS OTHER CLINICAL CRS MEASURES | 100%
Improvement
from FY06  |
| --- | --- |

## Average percent of diabetic patients with DM Comprehensive complete has improved from around 18 percent in March 2007 to 37 percent in June 2008. 

- Average percent of diabetic patients with DM Comprehensive complete has improved from around 18 percent in March 2007 to 37 percent in June 2008.
- Average size of the population of patients has remained steady.
- Again, what about the learning occurring at each of the pilot sites. Have the 14 pilot sites accomplished about the same results? To facilitate this kind of learning, we will look at a set of “small multiples” once more.

## 		API/IHI 2008

## Readiness Assignment

- Pick one or two measures that have surfaced as important to your organization. (could be positive or negative)
- Obtain monthly data from those measures and plot the data on run charts.
- Bring the run charts to a larger group and discuss what the data in the run charts say about your organization.
- Summarize the learning.