# Presentation: 103826

## Establishing Clinically Important Differences (CIDs) in Health-Related Quality of Life (HRQoL) Measures

- Fredric D. Wolinsky
- The University of Iowa
- Center for Health Policy and Research
- October 29, 2004

- Please click to change to the next slide

## Three Reasons Why Patients Receive Medical Treatment

- Increased longevity
- Prevention of future morbidity
- To feel better

- Please click to change to the next slide

## The Dilemma:Measuring the Success of Interventions

- For the first two dimensions—longevity and prevention—this is straightforward and relatively easy
- For HRQoL, however, it is not
- Indeed, physiological and laboratory test results are often used as indirect measures or substituted end points

- Please click to change to the next slide

## What Is HRQoL?

- The aspects of patients’ lives that they value
- At a minimum, measures should tap:
- Symptoms
- The functional consequences of those symptoms
- Emotional function
- Measures may be disease-specific or generic

- Please click to change to the next slide

## How Is HRQoL Measured?

- Questionnaires ask patients how they feel or what they are experiencing
- Response options are typically yes/no, _<_ 7-point scales, or visual analog scales
- Answers are summed within domains or dimensions to yield an overall score

- Please click to change to the next slide

## The Big Question

- When is the change between the baseline and follow-up HRQoL scores for the same patient clinically relevant?
- Note:  we are assessing ***intra***-individual change, not ***inter***-individual change

- Please click to change to the next slide

## The Answer

- The *blunt* version:  we don’t know
- The *politically correct* version: “...additional work to enhance the interpretability of [HRQoL] outcome measures, particularly in terms of clinical significance, is needed...”
- from Clancy and Eisenberg, *Science* 282:245-246, 1998

- Please click to change to the next slide

## Two Methods for Determining the Minimum Level of Real Change

- Distributional approaches
- Anchor-based approaches

- Please click to change to the next slide

## Common Distributional Approaches

***Effect size*** measures, where the average change is divided by the baseline standard deviation (Cohen, 1969)
- Small effect size _>_ .20
- Medium effect size _>_ .50
- Large effect size _>_ .80

- Please click to change to the next slide

## Are Effect Size Measures Arbitrary?

- Testa (1987) says yes, and then arbitrarily argues for .60 for medium effect size
- Feinstein (1999) suggested .56 based on mathematical properties of rxy
- Sloan et al. (1998) showed that if the entire range of a scale is covered in 6 SDs, then an effect size of .50 is the same as the anchor-based suggestion of 0.5 on a 7-point scale

- Please click to change to the next slide

## More on the 0.5 SD Effect Size

- Norman et al. (2003) recently reviewed 38 studies that calculated MIDs (minimally important differences)
- In all but six studies, the MID was ***very*** close to the .50 SD (M = .495, SD = .155)
- And, it did not matter whether it was generic or disease-specific, or how many response categories there were

- Please click to change to the next slide

## The Standard Error of Measurement (SEM)

- The SEM is defined as the SD times the square root of one minus its reliability coefficient, or
- 			SD * (SQRT 1 – rxx))
- 1-SEM has been argued to represent “reliable change”
- If rxx = .75, 1-SEM = .50 SD
- If rxx > .75, 1-SEM < .50 SD

- Please click to change to the next slide

## Anchor-Based Measures

- This approaches uses an independent criterion (i.e., anchor) instead of the statistical distribution
- Anchors are what the patient, the primary care physician (PCP), or the expert says would be the smallest difference that “is perceived to be beneficial and warrants a change in treatment”

- Please click to change to the next slide

## An Anchor-Based Example:Global Assessment Items

- Overall, how has your ***energy level ***[vitality]*** ***changed since your last interview?  Would you say your ***energy level*** is:  worse, about the same, or better?
- [If worse] How much worse do you feel? Would you say you feel: almost the same, hardly any worse at all; a little worse; somewhat worse; moderately worse; a good deal worse; a great deal worse; or a very great deal worse?

- Please click to change to the next slide

## An Anchor-Based Example:Global Assessment Items...

- This yields a 15-point response scale, (-7 to +7)
- Conceptually (expressed in absolute values):
- 0 and 1 reflect no change
- 2 and 3 reflect small change
- 4 and 5 reflect medium change
- 6 and 7 reflect large change
- The average gain scores (deltas) within these categories reflect CIDs

- Please click to change to the next slide

## Guyatt et al.’s Three HRQoL Measures With “Established” CIDs

- CHQ -- the Chronic Heart Failure Questionnaire -- for CAD/CHF patients
- CRQ -- the Chronic Respiratory Questionnaire -- for COPD patients
- AQLQ -- the Asthma Quality of Life Questionnaire -- for asthma patients

- Please click to change to the next slide

## These Measures Use 7-Point Response Sets with Two Types of Items

- _Standard_ items –the same questions asked of all patients in the study
- _Patient-specific_ items –different questions generated for each patient, based on a 3-step process:
- ask patients what is important to them
- review a standard list of items
- have patients select the five most important items

- Please click to change to the next slide

## Bad News and Good News About Patient-Specific Questions

**Bad news** – it takes more time to administer (about 8-10 minutes to derive the top five affected activities), and is cognitively taxing for older or disadvantaged patients
**Good news** – it increases relevance to each patient, and enhances follow-up participation rates

- Please click to change to the next slide

## The CHQ:Three Domains, 16 Items

- Dyspnea is tapped by 5 patient-specific activities affected by their disease
- Fatigue is tapped by 4 standard items
- Emotional function is tapped by 7 standard items

- Please click to change to the next slide

## The CRQ:Four Domains, 20 Items

- Dyspnea is tapped by 5 patient-specific activities affected by their disease
- Fatigue is tapped by 4 standard items
- Emotional function is tapped by 7 standard items
- Mastery is tapped by 4 standard items

- Please click to change to the next slide

## The AQLQ:Four Domains, 32 Items

- Activities are tapped by 5 patient-specific and 6 standard activities
- Symptoms are tapped by 12 standard items
- Emotional function is tapped by 5 standard items
- Environment is tapped by 4 standard items

- Please click to change to the next slide

## Guyatt et al. Used Two Streams of Data to Derive CIDs

- Panel of providers experienced in administering the CHQ and CRQ questionnaires
- same small panel for the CHQ & CRQ
- no panel for the AQLQ
- Patients with symptomatic disease
- very small samples
- CHF and COPD patients combined

- Please click to change to the next slide

## What the Panelists and the Patients Did

- Panelists ***defined*** CIDs by consensus
- Patients completed HRQoL measures at baseline and follow-up, and global change assessments at follow-up, from which anchor gain-scores were calculated
- By triangulation, a minimal CID standard was defined as a 0.5 _per-item_ change

- Please click to change to the next slide

## The Aims of Our AHRQ-Funded Study

- Refine and extend CIDs (small, medium, and large) for the CHQ, CRQ and AQLQ in CAD/CHF, COPD, or asthma patients
- Develop comparable CIDs for the SF-36 (V-2) in these three patient groups
- Identify SEM values corresponding to these CIDs

- Please click to change to the next slide

## Why Include the SF-36, V-2?

- It is _the_ most widely used HRQoL
- Cost-effectiveness studies frequently use it
- _No_ CIDs exist for the SF-36, despite the ***myth***, based on inter-individual data, that the CID equals 3-5 points for any subscale
- New response sets provide better reliability

- Please click to change to the next slide

## The SF-36, V-2:Eight Domains, 35 Standard Items

- Physical functioning; 10 items, 3 responses
- Role physical; 4 items, 5 responses
- Bodily pain; 2 items, 5 or 6 responses
- General health perceptions; 5 items, 5 responses
- Vitality; 4 items, 5 responses
- Social functioning; 2 items, 5 responses
- Role emotional; 3 items, 5 responses
- Mental health; 5 items, 5 responses

- Please click to change to the next slide

## We Rely On Three Streams of Data

- North American panels of expert physicians to define CIDs
- Patients attending general medicine clinics participating in a multi-wave longitudinal observational study
- The primary care physicians of the patients in that longitudinal study

- Please click to change to the next slide

## Data Stream One:The Expert Physician Panels

- Three separate panels of nine physicians each
- Potential physician panelists were identified using* Medline* searches for published studies using the CHQ, CRQ, AQLQ, or SF-36 among patients within the three disease groups
- Final panelist selection was based on obtaining a balance between generalists and specialists, geographic diversity, research groups, and availability

- Please click to change to the next slide

## The Three Tasks forThe Expert Physician Panels

- Approve search terms for electronic medical chart reviews that identify potentially eligible patients
- Achieve consensus on CID thresholds for small, medium, and large individual change for the better or worse
- Agree on the suggested wording of the global change questions and their response options that will be asked of patients at follow-up

- Please click to change to the next slide

## Data Stream Two:The Middle Aged or Older Patients

- 1,662 patients, or ~300 targeted in each disease group (CAD/CHF, COPD, and asthma) at each clinical site (St. Louis VAMC PRIME and related clinics; Indiana University PCC clinics)
- A three-part eligibility process—electronic medical chart review, PCP confirmation, and patient screening

- Please click to change to the next slide

## The Three Tasks forThe Middle Aged or Older Patients

- ~15-minute enrollment interview in the clinic to elicit the HRQoL patient-specific items
- ~30-minute baseline telephone interview that includes the disease-specific measure, the SF-36, socio-demographics, stress, sense of control, religiosity, and patient satisfaction measures
- Six bimonthly ~30-minute telephone interviews including most of the above measures and the retrospective change questions

- Please click to change to the next slide

## Data Stream Three:The Primary Care Physicians

- 46 GIM faculty who each have 50 or more potentially eligible patients
- 14 PCPs from the St. Louis VAMC clinics
- 32 PCPs from Indiana University clinics

- Please click to change to the next slide

## Three Tasks for The Primary Care Physicians

- Chart review and disease confirmation of their own potentially eligible patients
- Completion of a 6-item questionnaire about their patient’s disease severity, prognosis, and treatment history at the time of the enrollment visit
- Completion of a 6-item questionnaire at their patient’s subsequent visits about clinically significant changes since the last visit, and whether these resulted in a treatment modification

- Please click to change to the next slide

## How We Are Determining the CIDs

- Please click to change to the next slide

## Determining the Expert-Based CIDs

- A two-round Delphi approach including background materials
- A 4-6 hour in-person meeting to review the Delphi results and achieve consensus on the CIDs
- Iterative review of the panel reports to achieve consensus
- Each panel report has been published (*JGIM*, *AHJ*, *AAAI*), & the comparison paper is in press (*HSR)*

- Please click to change to the next slide

## Determining the PCP-Based CIDs

- At follow-up visits PCPs are asked if there has been a clinically significant change in the patient’s condition since the last visit, if so in what direction, and was that small, medium, or large (7 [not 15] responses)
- Within those response groups, the patient’s gain score in HRQoL (current-last) is calculated

- Please click to change to the next slide

## Determining the Patient-Based CIDs

- At follow-up visits patients are asked if there has been a significant change in their condition since the last visit, if so in what direction, and on a 7 point scale, how much better or worse (15 responses)
- Within those response groups, the patient’s gain score in HRQoL (current-last) is calculated

- Please click to change to the next slide

## Arriving at Common Ground

- Triangulate or consolidate the three (expert, PCP, and patient-based) sets of CIDs
- But, if they are too disparate, we have specified an *a priori* evidentiary hierarchy:
- 		--PCPs (it’s _***C***_IDs)
- 		--Patients (it’s *their* HRQoL)
- 		--Experts (they weren’t there)

- Please click to change to the next slide

## Results: The Expert Physician Data Stream

- Please click to change to the next slide

## Slide 40

- Please click to change to the next slide

## Slide 41

- Please click to change to the next slide

##  Results:The PCP Data Stream

- Please click to change to the next slide

## PCP Baseline Questionnaires

- Rating of patient severity:  M = 3.0 (1 = much worse, 3 = about average, 5 = much better)
- Approximate chance of patient being hospitalized or dying in two years:  MH = 28.1%, MD = 14.8%
- Patients on medication (97%), who ever had lab tests ordered (84%), or were referred to specialists (50%) for the target condition

- Please click to change to the next slide

## PCP Follow-up Questionnaires

- PCPs reported CIDs in the patient’s condition in 205 (19.4%) of the 1,057 follow-up visits
- Of these 205 linked visits:
- 65.4% were declines
- 56.5% were small changes
- 36.5% were medium changes
- 6.8% were large changes

- Please click to change to the next slide

## PCP Follow-up Questionnaires...

- Among the 205 patients with CIDs in their target condition, at that visit...
- 46.8% had orders for changes in medication
- 24.9% had orders for labs or procedures
- 12.7% had referrals to specialists

- Please click to change to the next slide

## Mean Transformed Gain Scores by the PCP’s Global Change Perception Category for the SF-36 Scales.

|  | N of Cases | Physical Function | Role Physical | Pain | General Health | Vitality | Social Health | Role Emotional | Mental Health |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Large 
Decline | 8 | -16.25 | -14.06 | -17.50 | 3.13 | -11.72 | -32.81 | -15.63 | -1.25 |
| Medium 
Decline | 51 | -1.47 | 2.21 | -1.82 | -0.16 | 0.61 | 0.25 | 2.61 | -0.69 |
| Small 
Decline | 75 | -1.73 | -2.08 | -5.94 | -3.67 | -4.00 | -6.17 | -0.56 | -0.09 |
| No 
Change | 852 | 0.63 | 3.89 | -0.63 | -0.75 | 1.19 | 1.92 | 1.14 | 0.88 |
| Small 
Improve-ment | 41 | 3.41 | 0.46 | 4.15 | 0.00 | 3.05 | 5.49 | 4.88 | 3.41 |
| Medium 
Improve-ment | 24 | 3.75 | 7.81 | -0.38 | 5.70 | 1.30 | 0.52 | 3.47 | -0.63 |
| Large 
Improve-ment | 6 | 14.17 | -18.75 | -8.67 | 0.00 | 5.21 | -2.08 | 1.39 | -0.83 |

- Please click to change to the next slide

## Mean Transformed PCP-Based CIDs for the AQLQ Scales.

|  | N of Cases | Activity | Emotional | Environment | Symptoms |
| --- | --- | --- | --- | --- | --- |
| Large Decline | 2 | 0.00 | 0.00 | 8.33 | -9.72 |
| Medium Decline | 7 | 3.68 | 4.29 | 2.98 | -6.94 |
| Small Decline | 25 | -5.85 | -10.00 | 2.67 | -7.22 |
| No Change | 157 | 3.79 | 4.54 | 4.41 | 3.43 |
| Small Improvement | 15 | 4.47 | 8.44 | 6.67 | 7.59 |
| Medium 
Improvement | 8 | 3.79 | 3.75 | -2.08 | 2.78 |
| Large Improvement | 2 | 7.58 | 11.67 | 14.58 | 20.83 |

- Please click to change to the next slide

## Mean Transformed PCP-Based CIDs for the CHQ Scales.

|  | N of Cases | Activity | Fatigue | Emotional |
| --- | --- | --- | --- | --- |
| Large Decline | 2 | -41.67 | -18.75 | -21.43 |
| Medium Decline | 18 | 6.71 | 0.23 | 2.65 |
| Small Decline | 19 | 0.39 | -4.82 | -3.38 |
| No Change | 356 | 1.18 | 1.04 | 2.17 |
| Small Improvement | 11 | -2.42 | 3.03 | -1.08 |
| Medium Improvement | 6 | 17.50 | 8.33 | 9.52 |
| Large Improvement | 4 | 23.33 | 17.71 | 1.19 |

- Please click to change to the next slide

## Mean Transformed PCP-Based CIDs for the CRQ Scales.

|  | N of Cases | Activity | Fatigue | Emotional | Mastery |
| --- | --- | --- | --- | --- | --- |
| Large Decline | 3 | -12.22 | -13.89 | -5.56 | -20.83 |
| Medium Decline | 23 | -1.59 | -1.99 | 1.04 | -5.43 |
| Small Decline | 28 | -3.48 | -1.49 | -0.60 | -3.87 |
| No Change | 317 | 1.73 | 0.70 | 2.28 | 1.47 |
| Small Improvement | 13 | 11.79 | -1.92 | -1.65 | -6.41 |
| Medium Improvement | 9 | 5.93 | 10.19 | 11.90 | 1.39 |
| Large Improvement | 0 | -- | -- | -- | -- |

- Please click to change to the next slide

## Results: The Patient Data Stream

- Please click to change to the next slide

## Patient Follow-up Questionnaires

- Patients reported CIDs in their condition in 2,981 (36.2%) of their 8,254 linked follow-up visits
- Of these 2,981 linked follow-up visits with CIDs:
- 62.5% were declines
- 45.6% were small changes
- 35.4% were medium changes
- 19.0% were large changes

- Please click to change to the next slide

## Mean Transformed Gain Scores by Global Change Perception Category for the SF-36 Scales

|  | Physical Function | Role Physical | Pain | General Health | Vitality | Social Health | Role Emotional | Mental Health |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Large 
Decline | -5.85 | -10.15 | -0.29 | -3.85 | -6.61 | -12.62 | -8.72 | -8.99 |
| Medium 
Decline | -4.72 | -7.62 | -5.73 | -3.41 | -6.10 | -9.33 | -7.64 | -9.99 |
| Small 
Decline | -2.63 | -4.87 | -1.78 | -2.46 | -3.35 | -7.98 | -5.64 | -6.35 |
| No 
Change | 0.08 | 1.16 | 0.43 | -0.21 | 0.62 | 1.38 | 0.40 | 0.37 |
| Small 
Improve-ment | 2.97 | 5.84 | 2.21 | 2.33 | 3.66 | 4.65 | 5.00 | 4.56 |
| Medium 
Improve-ment | 4.85 | 11.57 | 4.24 | 3.37 | 7.61 | 8.59 | 8.00 | 6.59 |
| Large 
Improve-ment | 7.02 | 11.85 | -0.68 | 6.63 | 9.87 | 6.58 | 6.04 | 6.72 |
| N of Cases | 7,969 | 7,972 | 7,958 | 7,953 | 7,971 | 7,942 | 7,957 | 7,960 |

- Please click to change to the next slide

## Mean Transformed Gain Scores by Global Change Perception Category for the AQLQ Scales

|  | Activity | Emotional | Environment | Symptoms |
| --- | --- | --- | --- | --- |
| Large Decline | -7.46 | -4.94 | -6.27 | -9.71 |
| Medium Decline | -6.51 | -8.95 | -3.07 | -10.76 |
| Small Decline | -3.43 | -5.23 | -2.51 | -4.31 |
| No Change | 1.04 | 0.69 | 1.79 | 1.78 |
| Small Improvement | 4.08 | 4.47 | 5.99 | 8.89 |
| Medium Improvement | 8.43 | 7.38 | 8.96 | 11.45 |
| Large Improvement | 9.37 | 10.88 | 12.78 | 10.28 |
| N of Observations | 1,840 | 1,839 | 1,841 | 1,839 |

- Please click to change to the next slide

## Mean Transformed Gain Scores by Global Change Perception Category for the CHQ Scales

|  | Activity | Fatigue | Emotional |
| --- | --- | --- | --- |
| Large Decline | -12.95 | -8.09 | -11.26 |
| Medium Decline | -12.25 | -6.80 | -5.75 |
| Small Decline | -2.96 | -3.62 | -5.42 |
| No Change | 1.02 | 0.59 | 0.23 |
| Small Improvement | 4.83 | 6.25 | 3.57 |
| Medium Improvement | 6.43 | 7.38 | 5.21 |
| Large Improvement | 9.94 | 10.34 | 7.62 |
| N of Observations | 3,245 | 3,246 | 3,245 |

- Please click to change to the next slide

## Mean Transformed Gain Scores by Global Change Perception Category for the CRQ Scales

|  | Activity | Fatigue | Emotional | Mastery |
| --- | --- | --- | --- | --- |
| Large Decline | -11.06 | -10.84 | -10.34 | -5.77 |
| Medium Decline | -6.30 | -6.99 | -6.03 | -7.42 |
| Small Decline | -3.76 | -5.64 | -5.87 | -6.25 |
| No Change | 0.52 | 1.00 | 0.05 | 0.51 |
| Small Improvement | 7.07 | 5.63 | 3.64 | 2.65 |
| Medium Improvement | 8.00 | 8.75 | 6.02 | 4.34 |
| Large Improvement | 15.38 | 14.51 | 8.65 | 4.04 |
| N of Observations | 2,881 | 2,883 | 2,882 | 2,884 |

** **

- Please click to change to the next slide

## Summary:The Expert Panels

- Used doubling and trebling rules to go from small to medium to large CIDs
- Assumed that CIDs for improvement and decline were symmetrical
- Used “state change” multiples for the SF-36
- Were a bit (heart, lung) to twice (asthma) as conservative as Guyatt’s original panels

- Please click to change to the next slide

## Summary:The PCPs

- Were less likely than patients to indicate that change had occurred (19% vs. 36%)
- Were more likely than patients to indicate that changes were small (57% vs. 46%) and less likely to indicate that they were large (7% vs. 19%)
- Did not alter the treatment regimen for 58% of patients with CIDs

- Please click to change to the next slide

## Summary:The Patients

- Because of the high (> .75) reliability estimates for the scales, the value of 1-SEM was often much less than that of 0.5 SD
- The anchor-based CIDs were not symmetrical, and the expert panel’s doubling and trebling rules did not hold
- The anchor-based CIDs are much smaller than the distributional or panel CIDs

- Please click to change to the next slide

## Conclusion

- Reconciling the CID estimates from the three data streams is not going to be straightforward
- So, stay tuned...

- Please click to change to the next slide