Online platforms are using warning messages to counter disinformation, but current approaches are not evidence-based and appear ineffective. We designed and empirically evaluated new disinformation warnings by drawing from the research that led to effective security warnings. In a laboratory study, we found that contextual warnings are easily ignored, but interstitial warnings are highly effective at inducing subjects to visit alternative websites . We then investigated how comprehension and risk perception moderate warning effects by comparing eight interstitial warning designs. This second study validated that interstitial warnings have a strong effect and found that while warningdesign impacts comprehension and risk perception, neither attribute resulted in a significant behavioral difference . Our work provides the first empirical evidence that disinformation warnings can have a strong effect on users ' information-seeking behaviors, shows a path forward for  effective warnings, and contributes scalable, repeatable methods for establishing evidence on the effectsof disinformation warnings .
Disinformation is proliferating on the internet, and platforms are responding by attaching warnings to content. There is little evidence, however, that these warnings help users identify or avoid disinformation. In this work, we adapt methods and results from the information security warning literature in order to design and evaluate effective disinformation warnings. In an initial laboratory study, we used a simulated search task to examine contextual and interstitial disinformation warning designs. We found that users routinely ignore contextual warnings, but users notice interstitial warnings---and respond by seeking information from alternative sources . We then conducted a follow-on crowdworker study with eight interstitial warning designs. We confirmed a significant impact on user information-seeking behavior, and we found that a warning's design could effectively inform users or convey a risk of harm. We also found, however, that neither user comprehension nor fear of harm moderated behavioral effects . Our work provides evidence that disinformation warnings can---when designed well---help users identify and avoid disinformation. We show a path forward for designing effective warnings, and we contribute repeatable methods for evaluating behavioral effects. We also surface a possible dilemma: disinformation warnings might be able to inform users and guide behavior, but the behavioral effects might result from user experience friction, not informed decision making .
R
Online platforms are using warning messages to counter disinformation, but current approaches are not evidence-based and appear ineffective. We designed and empirically evaluated new disinformation warnings by drawing from the research that led to effective security
Disinformation is proliferating on the internet, and platforms are responding by attaching warnings to content. There is little evidence, however, that these warnings help users identify or avoid disinformation. In this work, we adapt methods and results from the information security warning literature in order to design and evaluate effective disinformation
R
a
an initial
R
found that contextual warnings are easily ignored, but interstitial warnings are highly effective at inducing subjects to visit alternative websites
used a simulated search task to examine contextual and interstitial disinformation warning designs. We found that users routinely ignore contextual warnings, but users notice interstitial warnings---and respond by seeking information from alternative sources
R
investigated how comprehension and risk perception moderate warning effects by comparing
conducted a follow-on crowdworker study with
R
This second study validated that interstitial warnings have a strong effect and found that while warningdesign impacts comprehension and risk perception, neither attribute resulted in a significant behavioral difference
We confirmed a significant impact on user information-seeking behavior, and we found that a warning's design could effectively inform users or convey a risk of harm. We also found, however, that neither user comprehension nor fear of harm moderated behavioral effects
D
the first empirical
R
can have a strong effect on users ' information-seeking behaviors, shows
can---when designed well---help users identify and avoid disinformation. We show
A
designing
R
contributes scalable,
we contribute
R
establishing evidence on the effectsof disinformation warnings
evaluating behavioral effects. We also surface a possible dilemma: disinformation warnings might be able to inform users and guide behavior, but the behavioral effects might result from user experience friction, not informed decision making