International Patient Decision AidStandards [IPDAS] Collaboration
reaches consensus on indicators
for judging the quality of patientdecision aids
A O’Connor, G Elwyn, D Stacey, R Volk, R Thomson,
A Barratt, M Barry, A Coulter, M Holmes-Rovner,
H Llewellyn-Thomas, N Moumjid, T Whelan & IPDAS

27th Annual Meeting of the Society for Medical Decision Making(October 21-24, 2005) San Francisco, CA, USA

Co Chairs: A O’Connor (CA) , G Elwyn (UK)
Steering Committee: A Barratt (AU), M Barry (US), A Coulter (UK),M Holmes-Rovner (US), H Llewellyn-Thomas (US), N Moumjid (FR) M O’Kane(US), D Stacey (CA), R Thomson (UK), T Whelan (CA)
Quality criteria panels: J Austoker (UK), A Barratt (AU), H Bekker (UK), JBelkora (US), C Braddock (US), P Butow (AU), E Chan (US), M Barry (US), A
Charvet (Switz), A Clarke (UK), J Davison (CA), J Dolan (US), A Edwards (UK), V
Entwistle (UK), A Fagerlin (US), D Feldman-Stewart (CA), J Fowler (US), D
Frosch (US), P Hewitson (UK), T Hope (UK), MJ Jacobsen (CA), A Kennedy(Switz), S Knight (US), M Kupperman (US), B Ling (US), H Llewellyn Thomas,(US)T Marteau (UK), K McCaffery (AU), N Moumjid (FR), A Mulley (US), A
O’Connor (CA), M O’Connor (US), E Ozanne (US), M Pignone (US), A Raffle(UK), C Ruland (NO), L Schwartz (US), K Sepucha (US), S Sheridan (US), S
Stableford (US), D Stilwell (US), V Tait (CA), D Timmermans (NL), L Trevena(AU), T Whelan (CA), C Wills (US), S Woloshin (US), S Ziebland (UK)
Stakeholder lead: A Coulter (UK)
Methods group: S Bernstein (US), P Shekelle (US), R Thomson (UK), R Volk(US)Stakeholder 122 voters from 14 countries

Patient Decision Aids (PtDAs)
adjuncts to counseling• information: options, outcomes• outcome probabilities• values clarification• patients’ stories• guidance/coaching
Cochrane Library, 2003; Issue 2

Exponential growth
481

500
400

336

300

DA-RCT
DA's

186

200
100

63

17 17241999

2001

356466

2003

2004

02002

International Patient Decision Aid Standards(IPDAS) Collaboration
Home
What arePatient Decision
Aids?
Who’s Involved?
Contact UsObjective• To establish an internationally approved setof indicators for judging the quality of thedevelopment and evaluation of patientdecision aids [PtDAs]• For use by:– Developers– Users (patients & practitioners)– Providers and policy makers

Methods
Launched 2003 conferences:
ISDM; SMDM; Ix
Committees formed:
1.Steering 2.Quality Criteria 3.Methods;
4.Stakeholder
Background evidence document:
12 quality domains
Discussed on SDM list serve

Methods• nominated sample [familiar with PtDAs]• 4 stakeholder groups:•researcher, practitioner, consumer, policy maker• 2-stage ‘evidence-informed’ Delphi consensusmethod; feedback 2nd round• online voting document•

83 indicators in 12 quality domains•

background summaries

Methods‘Equi-median’ scores [1-9 importance scale]• equalised effect of different numbers instakeholder groups
Indicators classified according to scores• Important:
7-9• Equivocal:
4-6 without disagreement• Invalid:
1-3 ordisagreement[30% scores bottom/top terciles]

Voting Site Layout: 2nd roundResults
Response to invitation to serve on
voter panel# invited on
voting panel# (%) agreeing to beon voting panel

policy makers / healthplan administrators

2614 (54%)

patient / consumers

4321 (49%)

health professionals

1810 (56%)

Researchers/developers

12577 (62%)

Total

212122 (58%)

Results
1st round voters participating in 2ndround%
Policy maker

9/14

64%

Patient

19/2190%

HealthProfessional
Researcher

9/1090%

66/77

85%

103/12285%

Results after two votes
N

Equimedian = 941

Equimedian = 8

26

Equimedian = 77

Equimedian = 4to6 without disagreement

8

Equimedian = 3 or less

0Disagreement : 30% bottom/top terciles

1Total83Consensus: highlights
Development process
Providing information about options
Probabilities
Values clarification
Guiding/Coaching
Disclosure
Delivery on InternetBalance
Plain language
Up-to-date scientific evidence
EffectivenessPresenting probabilities
Medianevent rates, specific pop’n, time period
compare same denominatorcompare same period of time, scales
uncertainty e.g. our best estimate

9visual diagrams e.g. faces, stick figures, bar charts> one method e.g.words numbers diagrams
patient can select method
tailored to patient e.g. age
in context e.g. chances other diseases

8both positive & negative frames

89988787

Clarifying & expressing values
Mediandescribes option features to help patients imagine
what it is like to experience …physical,emotional, social effects
asks patients to consider which positive &negative features matter most to themsuggests ways to share what matters most tothem with others

897

Establishing effectiveness
DECISION PROCESSMedian8
recognize …a decision needs to be madeknow available options

9know different features of options

9

understand …values affect decision

9clear …which features matter most to them9

discuss values with health practitioners

7

become involved in dm…in ways they prefer

9

DECISION QUALITY
improves ..match between features that matter mostto the informed patient & chosen option

9

Excluded indicators• Content of PtDAs– including stories– offering option of trained coaches• Documentation evidence/tools– estimate probabilities– establish plain language– select stories

Is there a stakeholder effect?
16/83 items but only 5/83 cross threshold
98

7

6

PolicyPatientsPractitioner
Research

5

4

3

2

1aXI.4XI
.5
P2.

P2.

XI
.2
P2.

X.
2
P2.

X.
1P2.

VI
.5

VI
II.
2
P2.

P2.

VI
.4
P2.

VI
.3
P2.

VI
.2
P2.

10

VI
.1
P2.

III
.9

III.
P2.

P2.

III
.8

I.6
P2.

P2.

I.1
P2.

0

5 Differences affect Threshold10

9

Evidence: steps used to select included

Probabilities: select way to view

8

7

6

Update: reports how
often updated

5

Probabilities: place chance in context
4Internet: search key words
3Policy
Patients
Practitioner

2

Research

10
P2.I.1P2.I.6

P2.III.8

P2.III.9 P2.III.10 P2.VI.1

P2.VI.2

P2.VI.3

P2.VI.4

P2.VI.5 P2.VIII.2 P2.X.1

P2.X.2P2.XI.2

P2.XI.4 P2.XI.5a

Limitations• 58% participation (e.g. not available within 3
week time frame; thought we were SPAM)• nominated sampling• positive rating bias
• minimum versus gold standardsÆ feasibility

Conclusions• Users now have achecklist version2005.1 toassess quality• Strong endorsement
among stakeholders• Content elements notendorsed had weakestevidence

Research Agenda• Effects of stories &coaching on decisionquality• Application ofindicators in quality
measurement tool

www.ipdas.ohri.ca
Co Chairs: A O’Connor (CA) , G Elwyn (UK)
Steering Committee: A Barratt (AU), M Barry (US), A Coulter (UK),M Holmes-Rovner (US), H Llewellyn-Thomas (US), N Moumjid (FR) M O’Kane(US), D Stacey (CA), R Thomson (UK), T Whelan (CA)
Quality criteria panels: J Austoker (UK), A Barratt (AU), H Bekker (UK), J
Belkora (US), C Braddock (US), P Butow (AU), E Chan (US), M Barry (US), A
Charvet (Switz), A Clarke (UK), J Davison (CA), J Dolan (US), A Edwards (UK), V
Entwistle (UK), A Fagerlin (US), D Feldman-Stewart (CA), J Fowler (US), DFrosch (US), P Hewitson (UK), T Hope (UK), MJ Jacobsen (CA), A Kennedy(Switz), S Knight (US), M Kupperman (US), B Ling (US), H Llewellyn Thomas,(US)T Marteau (UK), K McCaffery (AU), N Moumjid (FR), A Mulley (US), A
O’Connor (CA), M O’Connor (US), E Ozanne (US), M Pignone (US), A Raffle(UK), C Ruland (NO), L Schwartz (US), K Sepucha (US), S Sheridan (US), S
Stableford (US), D Stilwell (US), V Tait (CA), D Timmermans (NL), L Trevena(AU), T Whelan (CA), C Wills (US), S Woloshin (US), S Ziebland (UK)
Stakeholder lead: A Coulter (UK)
Methods group: S Bernstein(US), P Shekelle(US), R Thomson(UK), R Volk(US)