Delivering Decision Aids on the Internet
2012 UPDATED CHAPTER H:
DELIVERING DECISION AIDS ON THE INTERNETSECTION 1:
AUTHORS/AFFILIATIONS
Aubri Hoffman (lead)

Dartmouth Centers for Health and Aging

USA

Robert J. Volk (co-lead)

The University of Texas MD Anderson Cancer Center

USA

Martin Härter

University Medical Center Hamburg-Eppendorf

Germany

Linda Li

University of British Columbia

CanadaHilary Llewellyn-Thomas

Geisel School of Medicine at Dartmouth

USA

Anton Saarimaki

Ottawa Hospital Research Institute & University ofOttawaThe University of Tasmania, School of Nursing

CanadaChristine Stirling

Australia

SECTION 2:
CHAPTER SUMMARY
What is this dimension?
Delivering decision aids on the Internet is defined here as the process of using the Internet toprovide some or all components of a patient decision aid (PtDA) to individuals (e.g., patients,
caregivers, clinicians, etc.) involved in the process of choosing between two or more medicallyappropriate healthcare options. This definition is intended to differentiate this process fromInternet-delivered general health education for patients (e.g., health information websites), andInternet-delivered clinical practice guidelines for clinicians.
What is the theoretical rationale for including this dimension?
Several theories in cognitive psychology, decision psychology, and communication emphasize
the value of using the Internet to provide broad and long-term dissemination of information thatcan be targeted and tailored to patients’ needs and preferences. In addition, several theories ofactive, discovery, and social learning inform the design of interactive PtDAs to effectively
provide information and support. Furthermore, theories of dissemination and implementationemphasize the need for the right information (rapidly updated), delivered to the right person(tailored), at the right time (in advance, during, and after a decision needs to be made).

Suggested Citation:
Hoffman A, Volk R, Härter M, Li L, Llewellyn‐Thomas H, Saarimaki A, Stirling, C. (2012). Delivering decision aids on the internet.
In Volk R & Llewellyn‐Thomas H (editors). 2012 Update of the International Patient Decision Aids Standards (IPDAS)
Collaboration’s Background Document. Chapter H. http://ipdas.ohri.ca/resources.html.

Delivering Decision Aids on the Internet
What is the evidence to support including or excluding this dimension?
In 2011, Internet usage ranges from 74% to 85% in developed countries, with 80% of users
searching for health information. Most individuals search for information about a specificdisease (66%) or treatment procedure (56%). However, it is unknown how many individuals
seek decision support on the Internet, due largely to the breadth of terms used to describe PtDAs
and the variety of formats available. Among the 86 studies in the 2011 update of the Cochrane
Collaboration’s Review of Patient Decision Aids assessed in randomized controlled clinical
trials, only 2 studies included PtDAs that were delivered on the Internet.

SECTION 3:
DEFINITION (CONCEPTUAL/OPERATIONAL) OF THIS QUALITY DIMENSIONa)

Updated Definition

Delivering decision aids on the Internet is defined here as the process of using the Internet toprovide some or all components of a patient decision aid (PtDA) to individuals (e.g., patients,
caregivers, clinicians, etc.) involved in the process of choosing between two or more medicallyappropriate healthcare options.
This definition is intended to differentiate this process from Internet-delivered general healtheducation for patients (e.g., health information websites), and Internet-delivered clinical practiceguidelines for clinicians.
This broad conceptual definition includes a range of operational definitions—from providing adownloadable copy of a paper-based PtDA, to streaming a video-based PtDA online, toproviding interactive decision support websites that tailor information and support to the needs
and preferences of each individual decision maker.
b)

Changes from Original Definition

The updated definition differentiates the process of providing PtDAs on the Internet from general
health education websites.
c)Emerging Issues/Research Areas in Definition

Given this broad definition, delivery of a PtDA using the Internet could be further categorizedaccording to the development methods, design features, and implementation approaches used.
Development Methods
For example, the development of PtDAs for delivery on the Internet could be categorized in threeways:Internet-available PtDAs may include PtDAs that were initially developed and tested in
paper, audio, or video formats, and then made available on the Internet. For example, several

2

Delivering Decision Aids on the Internet
PtDA pamphlets were originally created as paper worksheets, and then made available online
for individuals to download, print, and complete.
Internet-adapted PtDAs may include paper- or video-based PtDAs that were purposefully
adapted to allow individuals to use them directly on the Internet. Examples include adapting
paper worksheets into interactive questionnaires, and adapting text and video components ofPtDA DVDs into websites. While Internet-adapted PtDAs may have been rigorously tested
and evaluated in their original format, it is important to consider whether the adapted version
has been tested and evaluated as used on the Internet (see Usability, below).

Internet-based PtDAs may include PtDAs that were specifically designed and tested for useon the Internet. Examples include websites designed to help patients with specific healthcare
decisions by interactively tailoring information and support to their needs, or by providing
opportunities for family members to participate in the discussion. Internet-based designs may
also include e-mail, discussion forums, blogs, or social media sites (e.g., Facebook, patientcommunity websites) facilitated by clinicians, decision coaches, peers, or patient advocacy
groups.

It is important to note that delivery of a PtDA on the Internet may be only one component of alarger decision support program. For example, a patient may view a PtDA website coupled witha phone call from a clinician or decision coach. For such programs, evaluation measures should
be carefully considered for each component and for the overall decision support program.
Design Features
PtDAs delivered on the Internet could also be categorized using terms from bioinformatics andgraphic design, according to the features available:Linear vs. Open Format – A PtDA is considered linear if the content is presented in the sameorder every time and progression through the aid requires completion of each previous
section. An open format allows the individual to navigate the website and choose which
information to view and in what order.



Non-Interactive vs. Interactive – PtDAs may range from non-interactive if the person
viewing the site simply reads textual content, to minimally-interactive if they can select andview videos (see Chapter E, “Using Personal Stories”), to fully-interactive if individuals cannavigate the content and/or respond to interactive questions.



Static vs. Dynamic – A PtDA is static if it is programmed to provide all viewers with thesame design and content, and dynamic if it provides different information depending on what
the individual selects.

Text-heavy or Graphics-heavy – PtDAs delivered on the Internet may present content using avariety of media, including text, figures, graphs, photos, animations, audio, and videos.
Worksheets and brochures may rely on text-heavy or text-based presentations, while videos
may be more graphic-heavy. Mixed media refers to the use of more than one medium (e.g.,3Delivering Decision Aids on the Internet
text and photos), while multimedia refers specifically to the addition of audio (e.g., narrationand videos). Rich media specifies the inclusion of interactive components, and hypermedia
refers to interactive media that direct the presentation of information (e.g., hyperlinks thatallow open navigation between pages).
Passive vs. Active Deliberative Support – A PtDA may provide passive decision support by
merely providing content about the process of making a well-informed healthcare decision,or active decision support by actually leading the individual through the process of preparing
to make their decision.



Anonymous, De-identified, and Identifiable – PtDAs may be designed for anonymous use,
where it is impossible to identify individual users connection with a PtDA. De-identified
means that, if any data that might directly or indirectly identify a user are collected, thosedata are obscured so that there is only a very small chance the user could be linked to theinformation entered by a user. Identifiable means that the PtDA collects data that are
directly linked to identify the user. Examples include PtDAs that collect users’ email
addresses. If identifiable information is collected, it is important to determine who will haveaccess to that information. For example, password-protected accounts may be used to allow
patients to revisit their information over time and/or clinicians to collect information directly
into electronic health records. However, care (i.e., data safety) must be used to ensure
secure hosting to avoid access by third party companies and/or Internet service providers.



Tailored vs. Non-tailored – PtDAs delivered on the Internet may modify (i.e. “tailor”) the
content provided according to the patients’ characteristics, needs, and preferences in three
ways. First, they can tailor clinical content to reflect the risk / benefit levels implied by apatient’s unique clinical characteristics. Second, they can tailor decision support to patients’decision-making needs and preferences (i.e., preferred “deliberative style”), such as including
more/less detailed information, passive/active deliberative support, or an emphasis on
information-provision versus values-clarification, etc. Third, they can tailor how content isdisplayed to meet the needs of all audiences (see “Accessibility of the Design”, below).



Accessibility of the Design – This refers to the degree to which the PtDA is accessible to all
people, particularly those with disabilities. PtDAs provided on the Internet may be designed
to optimally work with assistive technologies, such as screen reading, screen magnifying,
Braille, or speech recognition programs. Visual accessibility may be increased by providing
larger font sizes, optimizing graphics for color-blindness, limiting flashing lights in videos,and by providing meaningful descriptions for images and links that can be read aloud byscreen reading programs. Providing captions/subtitles for videos may increase audio
accessibility. To increase motor accessibility, larger buttons, touch screens, or voicecommands may be used. Providing options for high and low literacy levels may increasecognitive accessibility.

Evaluation Measures
In addition to evaluation measures used to assess the quality of the decision support (e.g.,
preparation for decision making, information comprehension, decision quality, etc.), the fields of

4

Delivering Decision Aids on the Internet
bioinformatics and implementation science provide several measures of equitable disseminationof the technology.


Accessibility of the technology refers to the degree to which all people can access theInternet, regardless of available service (e.g. cable, mobile, wireless, or satellite broadband,
dial-up vs. high-speed, etc.). Dissemination strategies to maximize technology accessibility
include providing both text-heavy and graphics-heavy versions of a PtDA, as well as versions
for multiple screen sizes and smart phones (known as responsive design).

Universality refers to the degree to which the PtDA is appropriate for men and women of allages, races, ethnicities, religions, and cultures.

Usability “…refers to how well users can learn and use a product to achieve their goals andhow satisfied they are with that process” (U.S. ISO/AWI TR 9241-11, 1998). This definition
includes a combination of factors, primarily focusing on five areas:
1. Ease of Learning - how easy it is to do basic tasks the first time you see the website,
2. Efficiency of Use - how quickly you can use the website once you know how,
3. Memorability - how well you can remember how to use it the next time you visit,
4. Error Frequency - how many errors are typically made in looking for information, and
5. Satisfaction - how much you like using the website.

Because the provision of PtDAs on the Internet allows for multiple levels of interaction among
these sub-types, it will be important to distinguish meaningful conceptual and operational
definitions from these multiple perspectives. Furthermore, effective delivery of a PtDA using
the Internet may be defined differently for patients and/or caregivers making an acute or chronic
decision, than for clinicians helping individual patients, researchers studying groups of patients,
public health programs serving communities of patients, or policy makers serving populations ofpatients. In addition, these definitions will need to be placed in the context of the timing of thePtDA provision (before, during, or after consultation with a clinician) and the device used (e.g.,
computer, tablet, or smart phone phone). Finally, additional evaluation and dissemination
approaches will be needed for mixed-modal decision support, where delivery on the Internet iscoupled with in-person or phone consultations by clinicians or decision coaches.

SECTION 4:
THEORETICAL RATIONALE FOR INCLUDING THIS QUALITY DIMENSIONa)

Updated Theoretical Rationale

Several theories in cognitive psychology, decision psychology, and communication emphasize
the value of using the Internet to provide broad and long-term dissemination of information thatcan be targeted and tailored to patients’ needs and preferences. The Health Belief Modelemphasizes the importance of providing tailored information to motivate active engagement inhealthcare (Becker 1974). Similarly, interactive deliberative tasks foster self-efficacy and lead toincreased engagement (Social Cognitive Theory; Bandura 1977). Within communication
theories, the Elaboration Likelihood Model proposes that people attend to and actively process5Delivering Decision Aids on the Internet
information more if it is perceived as personally-relevant (Petty 1981). The theory of Goal
Setting and Performance proposed by Locke and Latham (1990) supports the role of interactivity
in producing tailored actionable personal health goals. Finally, the Stages of Change Theory
supports the value of having up-to-date information and accessibility over time, so that patientscan revisit it as needed in their progression of both disease changes and decision-making changes(Prochaska 1983).
Theories of active, discovery, and social learning also inform the design of Internet-delivered
decision support tools that effectively inform and educate individuals (Ally 2004). Behaviorism
emphasizes the need for measurable behaviors to confirm learning. For example, optional
activities such as information quizzes can reinforce awareness and facilitate realistic expectationsof outcomes (Thorndike 1913, Pavlov 1927, Skinner 1974). Cognitive psychologists emphasize
the internal processes of memory, motivation, thinking, and reflection on how new informationfits within one’s existing knowledge and experience (Craik & Lockart 1972, Ausubel 1974).
Therefore, interactive activities such as clarifying one’s personal attitudes about the individual
risks and benefits of each treatment option can reinforce comprehension and personalization.
Constructivism builds on these theories to discuss the roles of observation, processing, andinterpretation of information to build onto one’s personal reality (Cooper 1993, Wilson 1997).
Ally (2004) integrates these three theoretical bases to describe how information provided online
can and should be designed to help individuals learn about the “what” (behaviorist), “how”(cognitive psychologist), and “why” (constructivist) of their options (Ertmer and Newby 1993).
Alongside theories that inform the optimal design of Internet-delivered PtDAs for individual
patients, dissemination and implementation theories emphasize the importance of effectivenessfor larger populations. Theories of innovation, knowledge translation, and organizational change
may inform strategies for effective use of web-based PtDAs within and alongside medical
centers (Brownson 2012). Health equity frameworks support the development of technologysupported interventions to extend the reach of medical centers into rural or underserved
communities. RE-AIM, Precede-Proceed, and other community organizing models can guideconsumer-informed design approaches, particularly for chronic disease self-management and for
implementation within community programs (Glasgow 2001).
b)

Changes from the Original Theoretical Rationale

The updated theoretical rationale incorporates theories from multiple disciplines that support
both the use of the Internet to deliver PtDAs, and the potential design components of Internetdelivered PtDAs.
c)Emerging Issues/Research Areas in Theoretical Rationale

As decision support health technologies begin to include modes of social connectivity,
researchers will need to consider how established theories (e.g., Social Development Theory;
Vygotsky, 1962) inform their study of how people perceive, value, and use personal experiences
shared online in making healthcare decisions. Emerging websites such as PatientsLikeMe.org
allow patients and family members to document and share their healthcare treatment experiencesin order to help others make informed decisions. While this content carries a high potential for6Delivering Decision Aids on the Internetpresenting biased information to consumers, it does represent an emerging social experimenthighlighting the importance of shared information, connectivity, and outcomes that are of thegreatest importance to patients. Further research is needed to clarify what types of sociallyshared content are needed, preferred, and beneficial to patients, as well as whether sociallygenerated information can be collected and presented in neutral, unbiased, culturally-relevant
formats.
Similarly, the increasing use of “patient testimonials” on the Internet raises the need for research
into the effects, ethical issues, and appropriate use of this experiential information. On the one
hand, some communication and education theories support the use of written or video “patient
testimonials” because of their saliency, and qualitative studies confirm patient requests for such
experiential information; on the other hand, others advocate further study due to concerns aboutsocial matching, misrepresentation of risks/benefits, and biases (see Chapter E, “Using Personal
Stories”). Additional research is needed to determine which formats (written, audio, video),functions (narration, guidance, content provision), and what content (information about thedisease, the treatments, the outcomes of treatments, the process of decision making, and/or theoutcomes of decisions) are optimal within the various types of decision support (implicit
information, explicit guidance, automated coaching) that may be provided on the Internet.
As decision support interventions begin to be designed and used for long-term health behavior
change (e.g., managing chronic pain, quitting smoking, weight loss, dementia), the theories
related to stages of change will need to be tested in the context of preference-based decision
making. For example, the Trans-Theoretical Model related to Stages of Change has been used in
the development of Internet-delivered resources for patients seeking to improve their healthy
behaviors (Prochaska 1983). In situations in which decision support is nested withinmotivational counseling (for example, when choosing which weight loss diet and exercise
strategies work for you), this model will need to be tested in the context of providing online
decision support tools.
Until Internet access and usage becomes universal, several issues in implementation theory andhealth equity need to be considered. On the one hand, delivery of PtDAs on the Internet iscurrently less accessible for communities that lack either high-speed access of cellular phoneservice. However, telehealth clinics in rural communities can extend healthcare access forpatients who otherwise may be limited by travel times/costs (Fox 2012). Other communities
may benefit specifically from PtDAs delivered on the Internet, such as working adults who
cannot attend daytime doctor visits, home-bound individuals, and older adults who no longer
drive. In addition, emerging evidence (see below) suggests a shift from personal computers tomobile phones, with disadvantaged groups becoming the largest users of smart phones forInternet access. New efforts are being made to address health equity gaps by developing
culturally-tailored mobile health websites. Lastly, current trends indicate that retired adults are
the fastest growing group of new Internet users, but it remains unknown whether the working
generation will increase their Internet use once they retire. During this transition period inInternet use, theories of implementation and health equity will need to be operationalized withconsideration of potential short-term versus long-term approaches for an evolving patient
population.

7

Delivering Decision Aids on the Internet
There is a rising need for multidisciplinary theories and frameworks to address the multifacetedand changing needs and preferences of patients with chronic illnesses (Rogers 1983, Krueter
1999, Li 2012). In addition to traditional healthcare decisions, such as surgical versus medical
management, patients are increasingly offered self-management strategies, family-based
interventions, and community programs. Self-management theories support the use oftechnology for documentation of personal experiences over time, and social learning theories
discuss the saliency of shared experiences. As patients are openly reporting their testimonials on
the Internet, issues remain about how to consider these rapidly-growing banks of patient-reported
data and testimonials within the theoretical frameworks associated with decision support andevidence-based medicine. The expanding market of health technologies shapes the expectationsof patients, raising issues around incorporating user-directed marketing strategies within
implementation frameworks (Brownson 2012). An integrated framework of chronic care,
implementation, marketing, and knowledge translation theories may be needed to create effective
decision support programs that assist with self-management of progressive illnesses, theinclusion of family decision makers, and rapidly-evolving online health technologies.

SECTION 5:
EVIDENCE BASE UNDERLYING THIS QUALITY DIMENSIONa)

Updated Evidence BaseUse of The Internet For Decision Support
For 2007-2011, Worldbank.org indicates that an average of 74.2% of individuals in the UnitedStates, 76% in Australia, 81% in Canada, 83% in Germany, and 85% in the United Kingdom
used the Internet. A 2011 survey in the United States indicates that 80% of Internet users searchfor health information, making it the third most popular online activity (behind using e-mail andsearch engines) (Fox 2011). Most individuals search for information about a specific disease(66%) or treatment procedure (56%).
In recent years, studies by the Pew Internet & American Life Project reported a continuation in
the upward trend in mobile phone usage (Smith 2011, Fox 2012). Over 83% of U.S. adults
reported owning a mobile phone, with 35% using a smart phone to access the Internet. Mobile
Internet usage was most common among individuals who were 18-29 years old, African
American, and Latino. Qualitative analyses suggested a growing preference for social
connectivity (e.g., high rates of texting and social websites), and for searching for healthinformation on the Internet prior to making an appointment with a doctor.
However, it is unknown how many individuals seek decision support on the Internet, due largely
to the breadth of terms used to describe patient DAs and the variety of formats available.
Inventories of Decision Aids Delivered on the Internet
The Ottawa Hospital Research Institute maintains a Decision Aid Library Inventory (available at:
http://decisionaid.ohri.ca/AZinvent.php) that provides searchable and browseable lists of PtDAs

8

Delivering Decision Aids on the Internet
by: clinical topic; contact and availability information; and scoring on each of the IPDASChecklist criteria. As of October 2012, this library contained 270 PtDAs, 198 of which wereavailable in some form on the Internet. A systematic review is currently being conducted toa) identify which of these PtDAs are Internet-available, Internet-adapted, or Internet-based, andb) report the extent to which their effectiveness has been evaluated. Results will be incorporated
into this IPDAS chapter as they become available.
Randomized Controlled Trials of Decision Aids Delivered Using the Internet
The 2011 update of the Cochrane Collaboration’s Review of Patient Decision Aids searched for
randomized controlled trials published between 2006 and 2009 that compared a PtDA with usual
care or an alternative intervention with individuals who were actively making a treatment orscreening decision (Stacey 2011). Among the 86 studies that met these criteria, 2 studies
focused on PtDAs that were delivered on the Internet.
In the first study, Frosch and colleagues (2003) randomly assigned 226 men to view a 23-minutevideotaped PtDA (n = 112) or to access a website-based PtDA (n = 114), prior to deciding
whether they wanted to be screened for prostate cancer at their next physical examination. The
video-based PtDA had been developed and evaluated in several previous studies. The Internetbased version was created by adapting the video content into a series of slides (e.g., selected
images and audio from the video) to maximize download speed. Individuals randomized to thevideo group were offered the opportunity to watch the video at the clinic prior to their scheduled
appointment. Individuals randomized to the Internet group were asked to access the slides on a
website using a personal or public computer prior to their appointment. Internet access was notmade available at the clinic and 24.7% had to download software to view the slides.
Results indicated that men rated the video and Internet presentations equivalently in terms ofeffort required, perceived convenience, and satisfaction. Men who viewed the video were more
likely (χ2 (1) = 73.29, p < .001) to view the entire video than those who viewed the Internet site.
However, knowledge scores were equivalent between the video group and the Internet group
who viewed the entire set of slides. These results must be interpreted cautiously, given that no
usability or accessibility evaluation was reported for the Internet version prior to evaluation for
effectiveness. There also exists potential confounding by location and timing of availability
relative to the clinical appointment. This study highlights the importance of purposefully
integrating health technology and decision support research methods and outcome measures.
In the second study, Frosch and colleagues (2008) randomly assigned 611 men to one of 4
versions of an Internet-delivered PtDA: a) traditional didactic PtDA providing information aboutprostate-specific antigen screening options and outcomes; b) chronic disease trajectory model for
prostate cancer followed by a time–trade-off exercise; c) both the didactic PtDA and the chronic
disease trajectory model; or d) links to public prostate cancer–specific websites from credible
sources. Reminders were sent prior to the patients’ appointments, which increased website usage
to 84.3% from the previous trial. On a 10-item information comprehension quiz, men whoviewed the traditional didactic version scored the highest (mean 8.65; SD.18), and those whoviewed the public websites scored the lowest (mean 7.49; SD 0.19); p = .005. Differences wereobserved between groups in preferences for screening (p = .047). The authors concluded that9Delivering Decision Aids on the Internet
guiding patients to a structured PtDA on the Internet was more effective than health information
websites.
Additional Studies in 2011-2012
In addition, the authors are aware of at least three additional studies conducted in 2011-2012.
First, Li and colleagues (2011) developed and field-tested an Animated Self-serve Web-based
Research (ANSWER) tool to provide decision support for rheumatoid arthritis patients
considering methotrexate therapy. The PtDA consisted of narrated information about
methotrexate, 6 patient stories, and interactive questionnaires to clarify treatment preferences.
Participants (n = 15) in the field test took an average of 56.1 minutes (SD = 34.8) to complete theprogram and provided positive ratings using the System Usability Scale (mean 81.2 out of 100;
SD13.5). The Think Aloud technique was used and content analysis revealed three themes: 1)
user engagement (e.g., relevance of the design and content to the user), 2) information quality(e.g., clarity and credibility), and 3) user-tool interaction (e.g., ease of use). This study illustrates
the importance of quantitative and qualitative usability testing. An ongoing follow-up study(ANSWER-DC) is investigating the potential added value of combining the web-based
ANSWER PtDA with a 20-minute structured telephone consultation with an arthritis therapist.
In the second study, Simon and colleagues (2012) evaluated the effectiveness of a web-based,
individually tailored PtDA (known as Patient Dialogue) on depression or acute low back pain for
insurees of a German sickness fund. The Patient Dialogue system was compared to the nontailored Static Patient Information in an online randomized controlled trial. The primary outcome
was decisional conflict; secondary outcomes included knowledge, preparation for decisionmaking, preference for participation, involvement in decision-making, decision regret, andadherence. Out of 2,480 randomized participants, 657 (26.5%) provided analyzable data
immediately after using the system. Three months later, data from 131 (5.3%) participants could
be included in the analysis. The group who viewed Patient Dialogue reported significantly lower
levels of overall decisional conflict than the group who viewed the Static Patient Information(38.7 vs. 45.1; p = 0.028 via multiple imputation estimator). The largest standardized effect(Cohen’s d = 0.56) was observed on the preparation for decision-making scale (PD = 59.4 vs.
SPI = 46.8; p < 0.001). The authors concluded that the interactive Patient Dialogue PtDA may be
an effective tool to reduce decisional conflict and prepare participants for treatment decision
making. However, the large dropout rate needs to be taken into account. Furthermore, this study
shows how a health insurance fund can support shared decision making, and how a web-based
PtDA can be evaluated in a randomized controlled trial under routine care conditions.
Third, at the 2012 Society of Medical Decision Making Annual Meeting, Hoffman andcolleagues presented the results of the ENGAGE study of an Internet-based PtDA to assist
patients in choosing between non-surgical and surgical therapies for knee pain from chronic
osteoarthritis. Using a multidisciplinary theoretical framework, they developed the PtDA toprovide up-to-date clinical evidence about the treatment options, using a rich media format thatallowed viewers to interactively select “More Information” to learn more about each clinical
topic. The PtDA then provided guidance in 4 deliberative steps. To investigate the role of
implicit versus explicit deliberative guidance, 2 versions were created. The implicit guidance

10Delivering Decision Aids on the Internet
version presented the clinical evidence followed by standardized statements describing theprocess of becoming well-informed, considering which risks/benefits are most important
personally, considering one’s resources, and forming an action plan. The second version
explicitly led viewers through each deliberative step, offering an optional interactive personal
activity at each step.
One hundred and twenty-six participants from diverse background and locations in the UnitedStates were randomly assigned to either version. Notably, 100% of participants preferred to view
the PtDA on their home or local public computer rather than at the clinic. Overall, participants
reported that: the Internet-based PtDA was easy to use (98%), the information was clear (90%),
the length was appropriate (100%), it was appropriately detailed (90%), and it held their interest(97%). Participants scored an average of 75% correct responses (min. 60%; max. 100%) on 5
Information Comprehension items. The median Preparation for Decision Making score was 74(interquartile range = 30). The median Decision Self-efficacy score was 100 (interquartile range= 13.6). Viewing the PtDA reduced mean Decisional Conflict scores from 31.1 to 19.53 (p <0.01). These results were comparable to the performance of the video-based PtDA currently
used in clinical care.
Overall, no differences were observed between the group who viewed the implicit version andthose who viewed the explicit version. However, 46% of viewers in both groups chose tointeractively engage with the clinical information or with the deliberative steps. Multiple
regression analyses indicated that interactive engagement was significantly associated withincreased self-efficacy (b = -9.08; 95% CI = -15.95 - -2.21; p = 0.01) and decreased decisional
conflict (b = -13.29, 95% CI = -22.75 - -3.83; p = 0.007). These results suggest that a) an
Internet-based PtDA is acceptable to users and feasible for research, and b) patients may exhibit
different “deliberative styles” that benefit more/less from different opportunities for interactivity,
engagement, and support. Therefore, Internet-based PtDAs may offer additional gains whencareful attention is given to their use of interactive and tailored information and support.
b)Changes from Original Evidence Base

The updated evidence base provides additional research studies up to July 2012.
c)Emerging Issues/Research Areas in Evidence Base

As patients and families increasingly seek healthcare information and decision support on theInternet, additional research will be needed to address several fundamental questions:

What are the best methods for providing decision support using the Internet? Whichtechnology features are most helpful for patients and families?



What are the advantages and disadvantages of different learning environments, such as linear
and open designs?

Should Internet PtDAs “stand alone” or do they need to be supplemented with othermaterials, such as paper-based tools? For which populations?

11Delivering Decision Aids on the InternetWhat are the potential harms of providing decision support on the Internet?

What are the costs of developing and maintaining Internet PtDAs (e.g., updating
graphic/video components when new evidence emerges), and how do they compare to aids
developed and delivered using other media?

How can the features of Internet PtDAs be used to facilitate informed decision making by
patients from disadvantaged groups or with low health literacy?

Which technology features are most usable? Which are most preferred? By whom?



What are the strengths and weaknesses of delivering PtDAs on the Internet, and how can they
be adapted for other platforms such as smart phones, as applications for devices such astablets, smart phones, and e-book readers (e.g., iPads, iPhones, Kindles), or via social media
applications such as Facebook, Twitter, etc.?



How might delivery of decision support using the Internet foster long-term engagement ofpatients and families facing chronic diseases? For example, how could decision support via
websites best support communication with caregivers in other locations?



How can Internet-based PtDAs be designed to adapt to the rapidly-evolving expectations ofan increasingly technology-savvy population?

REFERENCES
Ally, M. (2004). Chapter 1: Foundations of Educational Theory for Online Learning. Theory andPractice of Online Learning. In Thorndike, E. L. (1913), Educational psychology: The
psychology of learning. New York: Teachers College Press.
Ausubel, D. P. (1960). The use of advance organizers in the learning and retention of meaningfulverbal material. Journal of Educational Psychology, 51, 267-272.
Bandura, A (1977). Toward a unifying theory of behavior change. Psychological Review, 184(2),191-215.
Becker, M.H. (1974) The health belief model and personal health behavior. Health Education
Monographs, 2,324-473.
Brownson, R.C., Colditz, G.A., Proctor, E.K. (2012) Dissemination and implementation
research in health: Translating science to practice. New York, NY: Oxford University Press.
Cole, R. A. (2000). Issues in web-based pedagogy: A critical primer. Westport, CT: Greenwood
Press.

12

Delivering Decision Aids on the Internet
Craik, F.I.M., Lockhart, R.S. (1972). Levels of processing: A framework for memory research".
Journal of Verbal Learning & Verbal Behavior 11 (6): 671–84.
Ertmer, P. A., & Newby, T. J. (1993). Behaviorism, cognitivism, constructivism: Comparing
critical features from an instructional design perspective. Performance Improvement
Quarterly, 6(4), 50-70.
Fishbein, M., Azjen, I. (1975). Beliefs, attitudes, intention, and behavior: An introduction totheory and research. Reading, MA: Addison-Wesley.
Fox, S. (2011). Health Topics Report of the Pew Internet and American Life Project.
[Accessed July 20, 2012 at http://www.pewinternet.org/reports]
Fox, S. (2012). Health, technology, and communities of color report of the Per Internet &American Life Project. [Accessed September 25, 2012 at
http://pewinternet.org/Commentary/2012/February/Health-Technology-Communities-ofColor.aspx]
Frosch, D.L., Kaplan, R.M., Felitti, V.J. (2003). A randomized controlled trial comparing
internet and video to facilitate patient education for men considering the prostate specificantigen test. Journal of General Internal Medicine, 18(10), 781-787.
Frosch, D.L., Bhatnagar, V., Tally, S., Hamori, C.J., Kaplan, R.M. (2008). Internet patientdecision support: A randomized controlled trial comparing alternative approaches for men
considering prostate cancer screening. Archives of Internal Medicine, 168(4), 363-369.
Glasgow , R.E., McKay, H.G., Piette, J.D., Reynolds, K.D. (2001). The RE-AIM framework forevaluating interventions: What can it tell us about approaches to chronic illness
management? Patient Education and Counseling, 44(2):119-127.
Hoffman, A.S., Llewellyn-Thomas, H.A., Tosteson, A.N.A., O’Connor, A.M., Volk, R.J.,
Tomek, I., Bartels, S.J., Andrews, S.A. (2012) Development of an Internet-based patients’decision aid for surgical versus nonsurgical management of knee osteoarthritis. The 34th
Annual Meeting of the Society for Medical Decision Making [Abstract available at
http://smdm.confex.com/smdm/2012az/webprogram/Session1601.html, manuscript in
preparation]
Hoffman, A.S., Llewellyn-Thomas, H.A., Tosteson, A.N.A., O’Connor, A.M., Volk, R.J.,
Tomek, I., Bartels, S.J., Andrews, S.A. (2012). The effect of implicit versus explicit
deliberative guidance and role of patients’ deliberative styles in online patients’ decision
aids. The 34th Annual Meeting of the Society for Medical Decision Making [Abstract
available at http://smdm.confex.com/smdm/2012az/webprogram/Session1601.html,
manuscript in preparation]"ISO/AWI TR 9241-1". International Organization for Standardization. Accessed July 20, 2012.

13

Delivering Decision Aids on the Internet
Keller, J. M. (1983). Motivational design of instruction. In C. M. Reigeluth (Ed.), Instructional
design theories and instruction: An overview of their current status (pp. 383-429). Hillsdale,
NJ: Lawrence Erlbaum.
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning anddevelopment. Englewood Cliffs, NJ: Prentice-Hall.
Kozma, R. B. (2001). Counterpoint theory of “learning with media.” In R. E. Clark (Ed.),
Learning from media: Arguments, analysis, and evidence (pp. 137-178). Greenwich, CT:
Information Age Publishing Inc.
Kreuter, M.W., Strecher, V.J., Glassman, B. (1999). One size does not fit all: The case for
tailoring print materials. Annals of Behavioral Medicine, 21(4), 276-283.
Li, L.C. Adam, P.M., Townsend, A. F., Lacaille, D., Yousefi, C., Turnau, S., Stacey, D. (2011)
Development and usability testing of ANSWER: A web-based methotrexate decision aid forpatients with rheumatoid arthritis [abstract] Arthritis & Rheumatism, 63(10): 1669.
Locke, E.A., Latham, G.P. (1990). A theory of goal setting and performance. Englewood Cliffs,
NJ: Prentice Hall.
Lorig, K.R., Laurent, D.D., Deyo, R.A., Marnell, M.E., Minor, M.A., Ritter, P.L. (2002).
Can a Back Pain E-mail Discussion Group improve health status and lower health carecosts? A randomized study. Archives of Internal Medicine, 162(7), 792-796.
Marlatt, G.A. (1985). Relapse prevention: Theoretical rationale and overview of the model.
In Marlatt G.A., Gordon J.R. (eds), Relapse Prevention. New York: Guilford Press.
Ottawa Hospital Research Institute. (2011). The A to Z Inventory of Decision Aids.
(Accessed July 20, 2012 at http://decisionaid.ohri.ca/AZinvent.php).
Pavlov, I. P. (1927). Conditioned reflexes. London: Clarendon Press.
Petty, R.E., Cacioppo, J.T. (1986). The elaboration likelihood model of persuasion in advanced
experimental social psychology. Orlando, FL: Academic Press.
Prochaska J.O., DiClemente C.C. (1983). Stages and processes of self-change for smoking:Toward an integrative model of change. Journal of Consulting and Clinical Psychology, 51,
390-395.
Rogers E.M. (1983) Diffusion of innovations (3rd Ed.). New York: The Free Press.
Rovai, A. (April, 2002). Building sense of community at a distance. International Review of
Research in Open and Distance Learning (IRRODL), 3, 1.

14

Delivering Decision Aids on the Internet
Schwitzer, G. (2002). A review of features in Internet consumer health decision-support
tools. Journal of Medical Internet Research. (Available
at http://www.jmir.org/2002/2/e11/ , accessed July 19, 2012)
Shaw, B.R., Mctavish, F., Hawkins, R., Gustafson, D.H., Pingree, S. (2000). Experiences ofwomen with breast cancer: exchanging social support over the CHESS computer
network. Journal of Health Communication, 5(2), 135-159.
Simon, D., Kriston, L., von Wolff, A., Buchholz, A., Vietor, C., Hecke, T., Loh, A., Zenker, M.,
Weiss, M., Härter, M. (2012). Effectiveness of a web-based, individually tailored decision
aid for depression or acute low back pain: A randomized controlled trial. Patient Education
and Counseling, 87, 360-368.
Skinner, C.S., Siegfried, J.C., Kegler, M.C., Strecher, V.J. (1993). The potential of computers inpatient education. Patient Education and Counseling, 22(1), 27-34.
Skinner, B. F. (1974). About behaviorism. New York: Knopf.
Smith, A., (2011) Smartphone adoption and usage. Pew Internet & American Life Project.
[Accessed September 25, 2012 at http://pewinternet.org/Reports/2011/Smartphones.aspx]
Stacey D., Bennet C.L., Barry M.J., Col N.F., Eden K.B., Holmes-Rovner M., LlewellynThomas H.A., Lyddiatt A., Légaré F., Thomspon R. (2011). Decision aids for people facing
health treatment or screening decisions. Cochrane Database of Systematic Reviews,
Published Online: 5 OCT 2011 http://www2.cochrane.org/reviews/en/ab001431.html
Strecher, V.J., DeVellis, B.M., Becker M.H., Rosenstock, I.M. (1986). The role of self-efficacy
in achieving behavior change. Health Education Quarterly, 13(1), 73-92.
Street, R.L., Rimal, R.N. (1997). Health promotion and interactive technology: A conceptual
foundation. In Street, R.L., Gold, W.R., Manning, T. (eds), Health Promotion and Interactive
Technology. Theoretical Applications and Future Directions. Mahwah, NJ: Lawrence
Erlbaum Associates, 1997.
Thorndike, E. L. (1913). Educational psychology: The psychology of learning. New York:
Teachers College Press.
Vygotsky, L.S. (1962). Thought and language. Cambridge, MA: MIT Press.
Wilson, B. G. (1997). Reflections on constructivism and instructional design. In C. R. Dills & A.
J. Romiszowski (Eds.), Instructional development paradigms (pp. 63-80). Englewood Cliffs,
NJ: Educational Technology Publications.

15

Delivering Decision Aids on the Internet
APPENDIX:
ORIGINAL CHAPTER H
Original Authors
Cornelia Ruland (lead)

Rikshospitalet National Hospital

NOR

Dominick Frosch

University of California

USA

Original Definition
The Internet is one of several ways in which patient decision aids are made available. Other
methods include video-cassettes, computer programs, decision boards, booklets, and audioguided workbooks.
Original Rationale/Theory
The use of the Internet for making patient decision aids available is becoming more popular
because of the increased accessibility for patients and the decreased costs for producing anddistributing them. Patients go increasingly to the Internet to find answers to their health concerns
and to get assistance in making health related decisions. However, information on the Internet
needs not only to be valid and reliable, but also easily accessible and comprehensible. The
design, usability, interface, logical organisation, and interactivity of an Internet site are crucial
factors to enable patients to obtain the information they need.
Original Evidence BaseInventory of Available Patient Decision Aids
Although patient decision aids have been available since the early 1980’s, the use of the Internet
as the medium of delivery has dramatically increased over the last 5 years. However, there has
been little evaluation of the effectiveness of the Internet as the medium of delivery.
Of 131 patient decision aids that were available, updated within the last 5 years, and registeredin the Cochrane Review of patient decision aids inventory (O’Connor et al., 2003):
 73% (of 131) were available only on the Internet. Several other patient decision aids
developed and evaluated as booklets or leaflets were also available on the Internet.
RCTs Involving Patients Facing Actual Choices
None of the 29 individual patient decision aids that were evaluated in 34 RCTS included in the
Cochrane review used the Internet as the medium of delivery in the study (O’Connor et al.,2003).
A recently published trial by Frosch and colleagues (2003) compared men considering prostatecancer antigen (PSA) screening a) who viewed a patient decision aid video in the clinic
immediately prior to their practitioners’ appointment with b) those who accessed the Internet16

Delivering Decision Aids on the Internet
based patient decision aid at home. There was no difference between the groups in terms ofparticipants' ratings of the convenience, effort, or satisfaction related to using the patient decision
aids. Compared to the Internet group, men in the video group were more likely to use the patientdecision aid (98.2% versus 53.5%), had significantly improved knowledge, and were more likely
to decline PSA screening. Men in the Internet group who reviewed the entire online patientdecision aid showed knowledge improvements similar to the video group.
Other Relevant Literature
The National Cancer Institute (NCI) provides Web design and usability guidelines based on
research studies and supporting information from the field. Each guideline provides:






A brief statement of the overarching principle that is the foundation of the guideline
Comments that further explain the research/supporting information
Source(s) of the research/supporting information
A score indicating the "Strength of the Evidence" that supports the guideline
One or more graphic examples of the guideline in practice.
(http://usability.gov/guidelines/intro.html#1)

Original References
Criteria for Evaluating Web Resources from Kent State University including A Website evaluation form.
Available at : http://www.library.kent.edu/internet/evalform.html;
http://www.library.kent.edu/internet/criteria.html.
EU Joins Standards Setting Movement with Publication of Quality Criteria and HON Code of Conduct(HONcode) for medical and health Web sites. Available at:
http://www.hon.ch/HONcode/HON_CCE_en.htm,
http://www.hon.ch/HONcode/HON_CCE_intro.htm.
Frosch DL, Kaplan RM, Felitti VJ. A randomized controlled trial comparing internet and video tofacilitate patient education for men considering the prostate specific antigen test. J Gen Intern Med
2003;18(10):781-7.
Health Summit Working Group: Criteria for Assessing the Quality of Health Information on the Internet –Policy Paper. Available at: http://hitiweb.mitretek.org/docs/policy.html
Inaccurate consumer health information on the Internet. Available at:
http://www.amia.org/pubs/symposia/D005366.PDF.
Lenert LA, Cher DJ. Use of meta-analytic results to facilitate shared decision-making. J Am Med Inform
Assoc 1999; 6: 412-419.
National Cancer Institute Usability Guidelines. Available at: http://usability.gov.
http://usability.gov/basics/index.html.
Nielsen, J. Usability Engineering. New York, Academic Press, 1993.
Schneiderman, B. Designing the User Interface: Strategies for Effective Human-Computer Interaction:
Third Edition, Addison-Wesley Publ. Co., Reading, MA, 1998.
Smith AG. “Testing the Surf: Criteria for Evaluating Internet Information Resources." The Public-Access
Computer Systems Review 1997; 8 (no.3).

17