Table 2 Results after two rating rounds: stakeholder group medians and equimedian for responses toPatients were asked what they need to prepare them to discuss a
specific decision.
Practitioners were asked what they need to discuss a specificdecision with patients.
Patients who were facing the decision field tested the decision aid.
Practitioners who counsel patients on the options field tested thedecision aid.
Field testing showed that the decision aid was acceptable topatients.

Researcher Group

1.2

Practitioner Group

Using a systematic development process
The patient decision aid has information about the credentials of thepeople who developed it.

Patient Group

11.1

Policy Group

Domain and Quality Criteria

Equimedian Rating

questions where differences between groups were significant (F and p-values are shown)

87

997

F= 3.37, p value 0.02
1.3
1.4
1.5
1.6

98
98
998.5

9

F = 3.67, p value 0.01
1.7
1.8a
1.8b2
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8

Field testing showed that the decision aid was acceptable topractitioners.
The decision aid was reviewed by outside experts [healthprofessionals] who were not involved in its development or fieldtesting
The decision aid was reviewed by outside experts [patients whopreviously faced the decision] who were not involved in its
development or field testing
Providing information about options
The patient decision aid describes the health condition related tothe decision.
The patient decision aid lists the health care options.
The option of choosing none of the health care options [e.g. doing
nothing] is included
The patient decision aid describes what happens in the natural
course of a health condition if none of the health care options ischosen.
The patient decision aid has information about the procedures
involved (e.g. what is done before, during, and after the health careoption)
The patient decision aid has information about the positive featuresof the options (e.g. benefits, advantages)
The patient decision aid has information about the negative
features of the options (e.g. harms, side effects, disadvantages)
The information about [outcomes] of options (positive andnegative) includes the chances they [may] happen.

889999999999Table 2 (continued)
For decision aids focused on tests
2.9
2.10
2.11
2.12
2.13

3
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
3.10
3.11
3.12

3.13
4
4.1

The patient decision aid has information about what the test is
supposed to measure.
The patient decision aid has information about the chances ofreceiving a true positive, true negative, false positive and false
negative test result.
The patient decision aid describes possible next steps based on
the test results.
The patient decision aid has information about the chances of
disease being found with and without screening.
The patient decision aid has information about detection andtreatment of disease that would never have caused problems if
screening had not been done.
Presenting probabilities
The patient decision aid presents probabilities using event rates ina defined group of patients for a specified time.
The patient decision aid compares probabilities of options using
the same denominator.
The patient decision aid compares probabilities of options over thesame period of time.
The patient decision aid describes the uncertainty around theprobabilities (e.g. by giving a range or by using phrases such as‘our best guess is’).
The patient decision aid uses visual diagrams to show theprobabilities (e.g. faces, stick figures, or bar charts).
The patient decision aid uses the same scales in the diagrams
comparing options.
The patient decision aid provides more than one way of explaining
the probabilities (e.g. words, numbers, diagrams).
The patient decision aid allows patients to select a way of viewing
the probabilities (e.g. words, numbers, diagrams).
The patient decision aid allows patients to see the probabilities ofwhat might happen based on their own individual situation. (e.g.
specific to their age or severity of their disease).
The patient decision aid places the chances of what might happen
in the context of other situations (e.g. chances of developing otherdiseases, dying of other diseases, or dying from any cause).
The way the probabilities were calculated is described [in areference section or accessible technical document]
If the chance of disease is provided by sub-groups [e.g., younger,middle-age, or older people], the tool that was used to estimate
these risks is described [in a reference section or accessible
technical document]
The patient decision aid presents probabilities using both positive
and negative frames (e.g. showing both survival and death rates).
Clarifying and expressing values
The patient decision aid describes the features of options to help
patients imagine what it is like to experience their physical,
emotional, and social effects.

98999999889877788

F = 3.92, p value 0.01
89977

F = 5.81, p value 0.00
87
77

F = 3.4, p value 0.02
5688

6

Table 2 (continued)
4.2
4.3
5
5.1
5.2
5.3
5.4
5.5
6
6.1

The patient decision aid asks patients to think about which positive
and negative features of the options matter most to them.
The patient decision aid suggests ways for patients to share [whatmatters most to them when] others are involved in the decision.
Using patient stories
The patient decision aid provides stories of other patients’experiences.
If stories are used in a patient decision aid, the stories represent arange of experiences (positive and negative).
If stories are used in a patient decision aid, the steps used toselect these stories are described [in a reference section oraccessible technical document].
If stories are used in a patient decision aid, the steps that experts
used to review the information contained in these stories isincluded [in a reference section or accessible technical document]
If stories are used in a patient decision aid, a statement that thepatients gave informed consent to include their stories is included.
[in a reference section or accessible technical document]
Guiding / coaching in deliberation and communication
The patient decision aid provides a step-by-step way to make adecision.

97686678988786.2

The patient decision aid suggests ways to talk about the decision
with a health practitioner.

8F = 3.91, p value 0.01
8896.3The patient decision aid includes tools like worksheets or lists ofquestions to use when discussing options with a practitioner.

8F = 3.53, p value 0.02
98876.4

The patient decision aid offers the option of working with a trained‘coach’ to help patients consider the options.

5F = 2.82, p value 0.04
66
65

6.5

The patient decision aid offers the option of working with a trained‘coach’ to help patients prepare to talk about the decision with apractitioner.
Disclosing conflicts of interest
The patient decision aid reports where the money came from todevelop the decision aid
The patient decision aid reports where the money came from tocopy and distribute the decision aid.
The patient decision aid reports whether the authors of thedecision aid stand to gain or lose by the choices patients makeafter using a decision aid.
The patient decision aid reports whether the affiliations of theauthors stand to gain or lose by the choices patients make afterusing a decision aid.
If the patient decision aid includes stories of other patients’experiences, it reports if there was some financial or other reason
why patients decided to share them.
Delivering patient decision aids on the Internet
If the patient decision aid is used on the Internet, it provides a stepby-step way to move through the web pages (screens) on theInternet.

5F = 4.81, p value 0.01
576

5

7
7.1
7.2
7.3
7.4
7.5
88.1

F = 7.21, p value 0.00
98
9
989

Table 2 (continued)
8.2

If the patient decision aid is used on the Internet, it allows patientsto search for key words in the decision aid.

78.3

If the patient decision aid is used on the Internet, it provides
feedback on personal health information that is entered into thedecision aid. [e.g. the chances you may get a complication]
If the patient decision aid is used on the Internet, the website
provides security for personal health information entered into thedecision aid.
If the patient decision aid is used on the Internet, it easy forpatients to find their way back to the point they were at in thedecision aid when they clicked on links to other web pages.
[If the patient decision aid is on the internet, it can also be printed
as a single document (e.g., pdf document)
Balancing the presentation of options
The patient decision aid makes it possible to compare the positive
and negative features of the available options.
The patient decision aid shows the negative and positive featuresof options with equal detail (for example using similar fonts, order,
display of statistical information).
Field testing showed that undecided patients felt the information
was presented in a balanced way.
Using plain language
The patient decision aid describes the ‘professional standards forplain language materials’ that guided its development (e.g. Plain
Language Association International)
The patient decision aid identifies the reading level at which it iswritten and the formula [method] used to determine the level.

77876

F = 8.35, p value 0.00

8.4
8.5
8.6
99.1
9.2
9.3
1010.110.2

99899955665

F = 5.12, p value 0.00
55644

F = 3.24, p value 0.03
10.3
10.4
10.5
10.6
1111.1
11.2
11.3
11.4

The patient decision aid is written at a level that can be understoodby at least half of the patients for whom it is intended.
The patient decision aid is written at a level no higher than grade 8[or equivalent] according to a readability formula (e.g., SMOG or
FRY).
The patient decision aid provides ways to help patients understandinformation other than reading (e.g. audio, video, or in-person
discussion).
Field testing showed that the patient decision aid was understoodby patients with limited reading skills.
Basing information on up-to-date scientific evidence
The patient decision aid provides references to scientific evidenceused.
The steps used to select the scientific evidence (e.g. finding,appraising, summarizing) is included [in a reference section oraccessible technical document]
The patient decision aid reports the date when it was last updated.
The patient decision aid reports how often the information in thedecision aid is updated.

978898788

6.5

F = 2.85, p value 0.04
98

98

97

F = 4.58, p value 0.04
11.5a

The patient decision aid describes the quality of the scientificevidence (e.g. quality of research studies).

8999

F = 3.3, p value 0.02

8

Table 2 (continued)
11.5b
11.6
1212.1
12.2
12.3
12.4
12.5
12.6
12.7
12.8

The patient decision aid describes the quality of the scientificevidence (e.g. quality of research studies) [including lack ofevidence].
The patient decision aid uses evidence taken from studies onpatients that are similar to the patients who would use the decisionaid (e.g. age, gender).
Establishing effectiveness
There is evidence that the patient decision aid helps patientsrecognize that a decision needs to be made.
There is evidence that the patient decision aid helps patients know
about the available options.
There is evidence that the patient decision aid helps patients know
about different features of the options.
There is evidence that the patient decision aid helps patients
understand that values affect the decision.
There is evidence that the patient decision aid helps patients beclear about which features of options matter most to them.
There is evidence that the patient decision aid helps patients
discuss values with their health practitioners.
There is evidence that the patient decision aid helps patients
become involved in decision making in ways they prefer.
There is evidence that the patient decision aid improves the match
between the features that matter most to the informed patient andthe option that is chosen.

8899999899

A. Statistical appendix [posted as supplied by author]
What is an Equimedian?
Alan M Sykes, BSc DPhil FSS

Acadvent Ltd
171 Gower Road
Swansea
Email: alan.sykes@which.net

Suppose that the population of interest consisted of equal numbers of three groups of people –call them A, B, and C. Suppose also that in a survey, 50 from group A had a mean value of 2.5,
10 from group B had a mean value of 4.3 and 10 from group C had a mean of 8.2 (for somemeasurable characteristic). Whilst the overall mean of all data is 3.614 the overall population
mean would be better estimated by an equally weighted mean of51112.5 u  4.3 u  8.2 u .
333

What would be the equivalent procedure for calculating the ‘equimedian’?
As an example, suppose that the actual values in the three samples are given in the frequency
table:
Value
123
45
6
7
89Frequency in Sample from A

Frequency in Sample from B
101515
1000000

Frequency in Sample from C
000073000000000244

A cumulative percentage table may then be constructed – see the first four columns below. To
this, we can estimate what the cumulative percentage might be in the whole population bytaking the average cumulative percentage for each value across the three samples.
Value
12
3
4
56
789

Cumulative percentage insample from A
20
50
80
100
100
100
100
100
100

Cumulative percentage insample from B
000070
100
100
100
100

Cumulative percentage insample from C0000002060
100

Average Cumulative
Percentage
6.66
16.66
26.66
56.66
66,66
66.66
73.33
86.66
100

From the cumulative percentage tables (or the graph) it is clear that the medians for the threesamples are 2.5, 4 and 8 respectively (whilst the overall median is 3). It is also clear that theestimated cumulative percentage curve for the whole population crosses the 50% line at thevalue 4 – the ‘equimedian’.

Cumulative Percentage

Example Equimedian
100
90
80
7060
Median f or A

50

Median f or C
Median f or B & Equimedian

403020100012345678910

Value axis
Sample A

Sample B

Sample C

Overall