Understanding Patient Preferences in Individual Health Decisions:Conceptual and measurement considerations

Sara J. Knight, PhD

Interdisciplinary Program to Improve Care for Veterans with Complex Comorbid Conditions

Goals
        - Examine several conceptual models for understanding preferences
        - Review approaches to the measurement of patient treatment preferences
        - Discuss qualitative strategies for refining conceptual models of patient treatment preferences and health outcomes
        - Identify types of studies used to provide evidence for the construct validity of measures of patient values

Characteristics of a Strong Measure of Patient Preferences
        - Comprehensive in patient considerations
        - Strong in psychometric properties
        - Feasible in a busy clinic settings
        - Appropriate for all educational backgrounds
        - Related to health care recommendations and outcomes

What are possible contributions of improved patient preference assessment?
        - Clarification of individual differences in decision making
        - Understanding of how preferences may be constructed over time
        - Improved patient-centered care
        - Strengthened patient/physician relationship

What are preferences?
        - Valuation of health care goods, services, and interventions
        - Economic Theory
        - Maximize subjective expected utility
        - Utilities can be scaled in dollar-equivalence terms
        - Based on assumptions of consistency and rationality with some limits

Preferences versus Values
        - Preferences
        - Economic theory
        - Utilities
        - Values
        - Psychological theory
        - Attitudes, preferences, interests, goals, needs

Preferences versus Health Status
        - Health Status
        - Patient experience, well-being, function
        - Concurrent or retrospective
        - Preferences
        - Patient, consumer, or societal values and goals
        - Prospective

Economic Model

Preferences

Stated Choice

Revealed Choice

Demand

Utilization

Information

Resources and Constraints

U1 = u(Attrib11,   Attrib12 , ..., Attrib1n)

Utility of Choice Alternatives

U2 = u(Attrib21,   Attrib22 , ..., Attrib2n)

U3 = u(Attrib31,   Attrib32 , ..., Attrib3n)

Another Economic Model:Adapted from McFadden

Memory

Motivation/Affect

Attitudes

Preferences

Perceptions/Beliefs

Process

Experience

Information

Stated

Perceptions

Choice

(Revealed Preferences)

Stated Preferences

Attitude Scales

Time and Dollar

Constraints

Individual Decision Making
        - International Patient Decision Aid Standards (IPDAS) Collaboration Recommendations on Values Measurement
        - Describe procedures and outcomes to help patients imagine what it is like to experience their physical, emotional, and social effects
        - Ask patients to consider which positive and negative features matter most
        - Suggest ways for patients to share what matters most with others

IPDAS Effectiveness Recommendations:Does the patient decision aid ensure decision making is informed and values based?
        - Recognize a decision needs to be made
        - Know choice alternatives and their features
        - Understand that values affect decisions
        - Be clear about option features that matter most
        - Discuss values with their practitioner
        - Become involved in preferred ways

Individual Decision Making Model

Attribute

Value

Attribute

Value

Attribute

Value

Attribute

Value

Attribute

Value

Physician Recommendation

Treatment

Choice

Patient Preferences

Decision Quality Knowledge, Satisfaction,

Conflict, Regret

Treatment Outcomes 

Functional Status,

Quality of Life, Symptoms

Information

Preference/Choice

Concordance

Experience/Affect

Methods of Measurement
        - Utilities Elicitation
        - Standard Gamble
        - Time Trade-Off
        - Visual Analog Scale
        - Rating Scales
        - Attitude Questions
        - Conjoint Analysis
        - Combination Approaches

Utilities
        - Value between 0 and 1, where 0 represents the value of being dead and 1 the value of living with perfect health
        - Represents patient’s subjective value for choice attribute, such as a health state or a treatment characteristic

Example: Standard Gamble

Standard Gamble

Standard Gamble

Standard Gamble

Standard Gamble

Example: Time Trade-Off

	Imagine that you have two friends, Mr. Smith and Mr. Jones.

	Imagine that Mr. Smith’s health fits the following description; that it will stay the same for the rest of his life; and that he will live about 10 more years.

Time Trade-Off:The Health State
        - Mr. Smith has mild difficulties or problems with urinating or bowel function.
        - He is able to do most of his usual activities nearly all of the time.  He is not overly tired and his energy level is pretty good.
        - He usually has a good appetite.
        - He has very little or no pain and it is easily controlled by medication.
        - His ability to have sex and enjoy if has been mildly affected.
        - He hardly ever feels tense, worried, irritable, sad, fearful or depressed.

Time Trade-Off:The Choice

	Now imagine that your other friend, Mr. Jones, lives in perfect health, but will live somewhat less than ten years.  If you had to be one of these 2 people, who would you rather be?  Mark your answer below.

	10 years as Mr. Smith in health state A or 5 years as Mr. Jones in perfect health?

Example: Visual Analog Scale

Advantages of Utilities
        - Based on economic and psychological theory; meets assumptions of normative decision making
        - Experimental task involving a choice
        - Scores comparable across methods
        - Useful in studies of health economics and policy

Limitations of Utilities
        - Questionable reliability
        - Logical inconsistencies within methods
        - Limited evidence for validity
        - Inconsistencies in scores across methods
        - Not well accepted by patients
        - Difficult for patients with low literacy and low numeracy
        - Not related to treatment recommendations

Souchek et al., 2000

Attitude Measures
        - Attitudes are defined as a psychological tendency that is expressed by evaluating a particular entity with some degree of favor or disfavor
        - Usual measurement is through ratings or rankings

Eagly and Chaiken, 1996

Example: Attitude Measure

If you were forced to choose between the term “faulty gene” and “altered gene”, which would you prefer? Please circle your response on the scale below:

1= I definitely prefer “altered gene”

2= I prefer “altered gene”

3= I am happy with either term

4= I prefer “faulty gene”

5= I definitely prefer “faulty gene”

Wakefield et al., 2007

Evaluation of Attitude Measures
        - Requires few assumptions about underlying choice processes
        - Easy for participants
        - Holistic approach
        - Valuation of an attribute as a whole
        - Does not allow evaluation of levels or different categories of an attribute
        - Vulnerable to “halo” effects

Phillips, Johnson, Maddala, 2002

Conjoint Analysis
        - Allows attributes to be evaluated in relation to each other, or “conjointly”
        - Yields utilities
        - Assumptions
        - Each good or service is a bundle of potential attributes
        - Each individual has set of unique relative utilities weights for attribute levels
        - Combining utilities for difference attributes provides an individuals overall relative utility

Phillips, Johnson, Maddala, 2002

Conjoint Analysis

Would you choose Choice A or B?
| ATTRIBUTES | TEST A                                                                                    | TEST B                                                |
|------------|-------------------------------------------------------------------------------------------|-------------------------------------------------------|
| Accuracy   | The diagnosis is correct 80% of the time, 20% of the time it is incorrect                 | The diagnosis is correct almost 100% of the time      |
| Privacy    | You, your doctor, and your insurance company will receive information about your genetics | Only you will receive information about your genetics |
| Cost       | $0                                                                                        | $500                                                  |

Evaluation of Conjoint Analysis
        - Allows the evaluation of each attribute separately
        - Provides estimation of willingness to pay; standardized approach to quantifying preferences for economic evaluation
        - Complex and difficult for respondents
        - Vulnerable to simplification of judgments
        - Possible inconsistent responses

Phillips, Johnson, Maddala, 2002

Integrative Approaches
        - Bockenholt has suggested that using a nested approach to understanding value judgments
        - Comparative judgments are integrated with absolute judgments
        - Scores contain information on both relative value and scale origin

Importance Ratings
| How important is...                                                                    | ExtremelyImportant | VeryImportant | ModeratelyImportant | A LittleImportant | NotImportant |
|--------------------------------------------------------------------------------------|--------------------|---------------|---------------------|-------------------|--------------|
| Avoiding a treatment that causes problems for my relationship with my spouse/partner |                   |              |                    |                  |             |

Best Worst Scaling

Circle the concern that is most important to

you in making your decision about your

prostate cancer treatment:

Urinary Function

Responsibilities

Survival

Developing Measures of Preferences
| Step 1 | Identify a priori constructs                                       |
|--------|--------------------------------------------------------------------|
| Step 2 | Establish comprehensive range of constructs and exemplars for each |
| Step 3 | Develop representative content for each construct                  |
| Step 4 | Generate items for each content area (items and subscales)         |
| Step 5 | Select appropriate response choices                                |
| Step 6 | Refine measure based on pilot testing                              |
| Step 7 | Test measure properties                                            |

Construct Development
        - Ideally should be based on the purpose of the effort to understand preferences
        - Economic analysis?
        - Individual decision making?
        - Cost analyses may require a different conceptual framework for understanding preferences and values and a different method for assessment than needed for the measurement of patient preferences in shared decision making

Literature Review
        - Descriptions of treatment or test alternatives
        - Existing guidelines for care
        - Patient education material
        - Case studies
        - Outcomes research
        - Cost and utilization studies

Expert Judgments
        - Identify key stakeholders
        - Clinicians
        - Investigators
        - Patient advocates
        - Health administrators
        - Methods
        - Key informant interviews
        - Panel discussions
        - Delphi process
        - Conceptual mapping

Focus Groups
        - Used to generate broad range of constructs
        - Often 8 to10 participants; fewer can be justified
        - Experienced moderator needed
        - Stratify on key variables: treatment choice, ethnicity, gender
        - Structured exercises
        - Differences among constructs
        - Rankings among constructs
        - Characteristics that influence choice

Analytic Approach
        - Content analysis using a priori constructs as a starting point
        - Grounded theory approach may be used where an a priori model does not exist
        - Software 
        - NVivo
        - Ethnograph

Cognitive Interviews
        - Think aloud
        - Well, it depends on what you mean by family responsibilities. Do you mean money? Helping my wife with my grandkids? My wife says she’s OK with any treatment as long as I’m still alive.

Structured Cognitive Interviews
        - Pretesting of measures
        - Administration
        - Item stems
        - Response scales
        - Order 
        - Methods
        - Thinkalouds
        - Retrospective protocols
        - Behavioral observations

Psychometric measures versus utilities?
        - Psychometric measures of attitudes yield scores  interpreted from a population reference point and utility elicitation yields judgments scaled in terms of an absolute reference point (death, perfect health)
        - Psychometric concepts of content validity and predictive validity may not be applicable to utilities where construct-irrelevant variance and construct under representation may be more relevant concepts

Lenert and Kaplan, 2000

Evaluating Measurement Properties:Accumulating Evidence
        - Reliability
        - Internal Consistency
        - Test-retest
        - Validity
        - Face
        - Content
        - Criterion
        - Construct
        - Construct irrelevant variance
        - Construct under-representation

Consistency within

Consistency over time

Correct concept

Adequate coverage of concept

External criterion

Construct being measured

Minimal spurious influence

Representative constructs

Reliability: Internal Consistency
| Interpretation     | The consistency of responses within a scale or within a subscale                                                                                                                                                                            |
|--------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Statistical Method | Cronbach’s alpha (α) is the most common method. It is based on the average correlation over items weighted by variances                                                                                                                     |
| Interpretation     | It is the percent of variance that the current scale explains of  the unidimensional underlying construct being measured                                                                                                                    |
| Acceptable values  | Values range from 0-1.0 with higher scores reflecting more accurate measure of the underlying trait.  Values below .70 suggest a need for evaluation of each item within the scale, values above .90 for use of a measure with individuals. |

Reliability: Test-Retest
| Interpretation      | The stability in responses across assessment occasions                                                                                                                                                                           |
|---------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Experimental Method | The same instrument is administered twice to the same sample on two occasions usually 1-2 weeks or up to one month apart                                                                                                         |
| Statistical Method  | Intraclass reliability coefficient for continuous valued measures, kappa for discrete valued measures                                                                                                                            |
| Acceptable values   | Values range from -1.0 - 1.0 with higher scores reflecting more accurate measure of the underlying trait.  Values below .60 suggest low consistency and potential future difficulty in detecting differences due to real effects |

Validity: Face
| Interpretation            | Determination that the instrument is measuring the construct of interest                                                                                                                         |
|---------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Assessment Method         | Prior to development, focus groups can inform the content. After the development of the instrument, cognitive interviews can be used to assess understanding of the content of the questionnaire |
| Statistical Method        | No specific statistical method is associated with documenting face validity                                                                                                                      |
| Population considerations | The face validity of an instrument may vary across patient populations                                                                                                                           |

Validity: Content
| Interpretation           | Determining the adequacy of coverage of the instrument                                                                     |
|--------------------------|----------------------------------------------------------------------------------------------------------------------------|
| Experimental Method      | Experts in the area judge whether the instrument captures all domains of the construct of interest                         |
| Statistical Method       | Though rarely reported, the Content Validity Index or the Content Validity Ratio indicates the extent of expert agreement. |
| Determination of Experts | Choose a panel of experts that informs rather than limits your interpretation of the breadth of your construct.            |

Validity: Criterion
| Interpretation       | The association between the construct and some external criterion                                                                                                         |
|----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Experimental Methods | Concurrent validity: both validation and construct tools are administered on one occasion.  Predictive validity: construct is measured first; criterion is measured later |
| StatisticalMethod    | Tests of association are used to assess construct validity such as a Pearson r for continuous valued scale scores, or Chi-square for categorical scale scores.            |
| Acceptable values    | coefficients of .70 or greater                                                                                                                                            |

Validity: Construct
| Interpretation       | Establishes the ability of the instrument to measure the construct and to distinguish varying levels of the presence of that construct                                                                                                                                                                                                                                                          |
|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Experimental Methods | Collection of data from many subjects from the patient population of interest to determine latent trait(s) underlying the assessment instrument. Collection of data from two groups known to vary on the construct (known-groups technique).Multi-Trait Multi-Method—measure should be more strongly related to other measures of similar constructs and to measures of dissimilar constructs   |
| Statistical Methods  | A statistical method to explain variability in responses (e.g. factor analysis, component analysis). A statistical method to detect differences between groups (e.g. t-test)                                                                                                                                                                                                                    |