INFORMATION DISORDER :Toward an interdisciplinary framework
for research and policy making

Council of Europe report
DGI(2017)09

Claire Wardle, PhD
Hossein Derakhshan

Information Disorder
Toward an interdisciplinary framework for research and policymaking
By Claire Wardle, PhD and Hossein Derakhshan
With research support from Anne Burns and Nic Dias
September 27, 2017

The opinions expressed in this work are
the responsibility of the authors and do
not necessarily reflect
the official policy of theCouncil of Europe.

All rights reserved.
No part of this publication may be translated, reproduced or transmitted
in any form or by any means without the prior permission in writing from the
Directorate of Communications (F-67075 Strasbourg Cedex or publishing@coe.int).

Photos © Council of Europe
Published by the Council of Europe
F-67075 Strasbourg Cedex
www.coe.int© Council of Europe, October, 20171

Table of content
Author Biographies

3

Executive Summary

4

Introduction

10

Part 1: Conceptual Framework

20

The Three Types of Information Disorder

20

The Phases and Elements of Information Disorder

22

The Three Phases of Information Disorder

23

The Three Elements of Information Disorder25

1) The Agents: Who are they and what motivates them?

292) The Messages: What format do they take?

383) Interpreters: How do they make sense of the messages?

41

Part 2: Challenges of filter bubbles and echo chambers

49Part 3: Attempts at solutions

57

Part 4: Future trends

75Part 5: Conclusions

77

Part 6: Recommendations

80

Appendix: European Fact-checking and Debunking Initiatives

86

References

902Authors’ Biographies
Claire Wardle, PhD
Claire Wardle is the Executive Director of First Draft, which is dedicated to finding solutions tothe challenges associated with trust and truth in the digital age. Claire is also a Research
Fellow at the Shorenstein Center on Media, Politics and Public Policy at the Harvard Kennedy
School where First Draft is now a project. She sits on the World Economic Forum’s Global
Agenda Council on the Future of Information and Entertainment. She was previously the
Research Director at the Tow Center for Digital Journalism at Columbia Journalism School,
head of social media for the UN Refugee Agency and Director of News Services for Storyful.
She is one of the world’s experts on user generated content, and has led two substantial
research projects investigating how it is handled by news organizations. In 2009 she was
asked by the BBC to develop a comprehensive social media training curriculum for theorganization and she led a team of 19 staff delivering training around the world. She holds a
PhD in Communication and an MA in Political Science from the University of Pennsylvania,
where she won a prestigious Thouron Scholarship. She started her career as a professor at
Cardiff University’s School of Journalism, Media and Cultural Studies.
Hossein Derakhshan
Hossein Derakhshan is an Iranian-Canadian writer and researcher. The pioneer of blogging in
Iran in the early 2000s, he later spent six years in prison there over his writings and web
activities. He is the author of ‘The Web We Have to Save’ (Matter, July 2015), which was
widely translated and published around the world. His current research is focused on thetheory and socio-political implications of digital and social media. His writings have appearedin Libération, Die Zeit, the New York Times, MIT Technology Review, and The Guardian. He
studied Sociology in Tehran and Media and Communication in London.

3

Executive Summary
This report is an attempt to comprehensively examine information disorder and its related
challenges, such as filter bubbles and echo chambers. While the historical impact of rumours
and fabricated content have been well documented, we argue that contemporary social
technology means that we are witnessing something new: information pollution at a global
scale; a complex web of motivations for creating, disseminating and consuming these‘polluted’ messages; a myriad of content types and techniques for amplifying content;
innumerable platforms hosting and reproducing this content; and breakneck speeds ofcommunication between trusted peers.
The direct and indirect impacts of information pollution are difficult to quantify. We’re only atthe earliest of stages of understanding their implications. Since the results of the ‘Brexit’ votein the UK, Donald Trump’s victory in the US and Kenya’s recent decision to nullify its national
election result, there has been much discussion of how information disorder is influencing
democracies. More concerning, however, are the long-term implications of dis-information
campaigns designed specifically to sow mistrust and confusion and to sharpen existing sociocultural divisions using nationalistic, ethnic, racial and religious tensions.
So, how do we begin to address information pollution? To effectively tackle the problems of
mis-, dis- and mal- information, we need to work together on the following fronts:
1. Definitions. Think more critically about the language we use so we can effectively
capture the complexity of the phenomenon;
2. Implications for democracy. Properly investigate the implications for democracy whenfalse or misleading information circulates online;
3. Role of television. Illuminate the power of the mainstream media, and in particular
television, in the dissemination and amplification of poor-quality information thatoriginates online;
4. Implications of weakened local media. Understand how the collapse of local
journalism has enabled mis-and dis-information to take hold, and find ways to support
local journalism;
5. Micro-targeting. Discern the scale and impact of campaigns that use demographic
profiles and online behavior to micro-target fake or misleading information1;
6. Computational amplification. Investigate the extent to which influence is bought
through digital ‘astroturfing’—the use of bots and cyborgs to manipulate the outcomeof online petitions, change search engine results and boost certain messages on social
media;
1 Hendrix, J. and Carroll, D. (2017) Confronting a Nightmare for Democracy. Available at:https://medium.com/@profcarroll/confronting-a-nightmare-for-democracy-5333181ca675

4

7. Filter bubbles and echo chambers. Consider the implications of the filter bubbles andecho chambers that have emerged because of media fragmentation, both offline(mediated via partisan talk radio and cable news) and online (mediated via hyperpartisan websites, algorithmically derived feeds on social networks and radical
communities on WhatsApp, Reddit and 4chan.) 2
8. Declining trust in evidence. Understand the implications of different communities
failing to share a sense of reality based on facts and expertise.
In this report, we refrain from using the term ‘fake news’, for two reasons. First, it is woefully
inadequate to describe the complex phenomena of information pollution. The term has also
begun to be appropriated by politicians around the world to describe news organisations
whose coverage they find disagreeable. In this way, it’s becoming a mechanism by which thepowerful can clamp down upon, restrict, undermine and circumvent the free press.
We therefore introduce a new conceptual framework for examining information disorder,
identifying the three different types: mis-, dis- and mal-information. Using the dimensions ofharm and falseness, we describe the differences between these three types of information:▪ Mis-information is when false information is shared, but no harm is meant.
▪ Dis-information is when false information is knowingly shared to cause harm.
▪ Mal-information is when genuine information is shared to cause harm, often by
moving information designed to stay private into the public sphere.

We also argue that we need to separately examine the ‘elements’ (the agent, messages andinterpreters) of information disorder. In this matrix we pose questions that need to be askedof each element.
2 Beran, Dale. “4chan: The Skeleton Key to the Rise of Trump.” Medium.com, February 14, 2017.

https://medium.com/@DaleBeran/4chan-the-skeleton-key-to-the-rise-of-trump-624e7cb798cb

5

We also emphasise the need to consider the three different ‘phases’ (creation, production,
distribution) of information disorder.

As we explain, the ‘agent’ who creates a fabricated message might be different to the agent
who produces that message—who might also be different from the ‘agent’ who distributes
the message. Similarly, we need a thorough understanding of who these agents are and what
motivates them.
We must also understand the different types of messages being distributed by agents, so that
we can start estimating the scale of each and addressing them. (The debate to date has been

6overwhelmingly focused on fabricated text news sites, when visual content is just as
widespread and much harder to identify and debunk.)
Finally, we need to examine how mis-, dis- and mal-information are being consumed,interpreted and acted upon. Are they being re-shared as the original agent intended? Or are
they being re-shared with an oppositional message attached? Are these rumours continuing
to travel online, or do they move offline into personal conversations, which are difficult tocapture?
A key argument within this report, which draws from the work of the scholar James Carey, isthat we need to understand the ritualistic function of communication. Rather than simply
thinking about communication as the transmission of information from one person toanother, we must recognize that communication plays a fundamental role in representing
shared beliefs. It is not just information, but drama — “a portrayal of the contending forces in
the world.”3
The most ‘successful’ of problematic content is that which plays on people’s emotions,encouraging feelings of superiority, anger or fear. That’s because these factors drive resharing among people who want to connect with their online communities and ‘tribes’.
When most social platforms are engineered for people to publicly ‘perform’ through likes,
comments or shares, it’s easy to understand why emotional content travels so quickly andwidely, even as we see an explosion in fact-checking and debunking organizations.
In addition to our conceptual framework, we provide a round-up of related research, reports
and practical initiatives connected to the topic of information disorder, as well as filter
bubbles and echo chambers. We examine solutions that have been rolled out by the social
networks and consider ideas for strengthening existing media, news literacy projects andregulation. We also introduce some key future trends, particularly in terms of the rise of
closed messaging apps and the implications of artificial intelligence technology for
manufacturing as well as detecting dis-information.
The report ends with an explanation of thirty-four recommendations, targeted at technology
companies, national governments, media organisations, civil society, education ministries andfunding bodies. They are explained in detail after the report’s conclusions.
What could technology companies do?
1. Create an international advisory council.

3 Carey, J. (1989), Communication as Culture: Essays on Media and Society, London: Routledge.

p.16

7

2. Provide researchers with the data related to initiatives aimed at improving public
discourse.
3. Provide transparent criteria for any algorithmic changes that down-rank content.
4. Work collaboratively.
5. Highlight contextual details and build visual indicators.
6. Eliminate financial incentives.
7. Crack down on computational amplification.
8. Adequately moderate non-English content.
9. Pay attention to audio/visual forms of mis- and dis-information.
10. Provide metadata to trusted partners.
11. Build fact-checking and verification tools.
12. Build ‘authenticity engines’.
13. Work on solutions specifically aimed at minimising the impact of filter bubbles:a. Let users customize feed and search algorithms.
b. Diversify exposure to different people and views.
c. Allow users to consume information privately.
d. Change the terminology used by the social networks.
What could national governments do?
1. Commission research to map information disorder.
2. Regulate ad networks.
3. Require transparency around Facebook ads.
4. Support public service media organisations and local news outlets.
5. Roll out advanced cybersecurity training.
6. Enforce minimum levels of public service news on to the platforms.
What could media organisations do?
1. Collaborate
2. Agree policies on strategic silence.
3. Ensure strong ethical standards across all media.
4. Debunk sources as well as content.
5. Produce more news literacy segments and features.
6. Tell stories about the scale and threat posed by information disorder.
7. Focus on improving the quality of headlines.
8. Don’t disseminate fabricated content.
What could civil society do?
1. Educate the public about the threat of information disorder.
2. Act as honest brokers.

8What could education ministries do?
1. Work internationally to create a standardized news literacy curriculum.
2. Work with libraries.
3. Update journalism school curricula.
What could funding bodies do?
1. Provide support for testing solutions.
2. Support technological solutions.
3. Support programs teaching people critical research and information skills.

9Introduction
Rumours, conspiracy theories and fabricated information are far from new. 4 Politicians haveforever made unrealistic promises during election campaigns. Corporations have always
nudged people away from thinking about issues in particular ways. And the media has long
disseminated misleading stories for their shock value. However, the complexity and scale ofinformation pollution in our digitally-connected world presents an unprecedented challenge.
While it is easy to dismiss the sudden focus on this issue because of the long and varied
history of mis- and dis-information5, we argue that there is an immediate need to seek
workable solutions for the polluted information streams that are now characteristic of ourmodern, networked and increasingly polarised world.
It is also important to underline from the outset that, while much of the contemporary furor
about mis-information has focused on its political varieties, ‘information pollution’ 6
contaminates public discourse on a range of issues. For example, medical mis-information has
always posed a worldwide threat to health, and research has demonstrated how incorrect
treatment advice is perpetuated through spoken rumours 7, tweets8, Google results9 and
Pinterest boards10. Furthermore, in the realm of climate change, a recent study examined theimpact of exposure to climate-related conspiracy theories. It found that exposure to suchtheories created a sense of powerlessness, resulting in disengagement from politics and areduced likelihood of people to make small changes that would reduce their carbon
footprint.11
In this report, we hope to provide a framework for policy-makers, legislators, researchers,
technologists and practitioners working on challenges related to mis-, dis- and malinformation—which together we call information disorder.

4 Sunstein, Cass R., and Adrian Vermeule. “Conspiracy Theories: Causes and Cures”. Journal of Political

Philosophy 17, no. 2 (2009): 202–227.
5 Uberti, D. (2016) The Real History of Fake News, Columbia Journalism Review, December 15, 2016.
https://www.cjr.org/special_report/fake_news_history.php
6 The term information pollution was first used by Jakob Nielsen in 2003 as a way to describe irrelevant,redundant, unsolicited and low-value information.
7
Smith, L. C., Lucas, K. J., & Latkin, C. (1999). Rumor and gossip: Social discourse on HIV and AIDS.
Anthropology & Medicine, 6(1), 121-131.
8 Oyeyemi, S. et al., (14 Oct., 2014), Ebola, Twitter, and mis-information: a dangerous combination, British
Medical Journal, 349
9 Venkatraman A. et al., (2016) Zika virus mis-information on the internet, Travel Medicine and Infectious
Disease, Vol 14: 4, pp 421-422
10 Guidry, J. et al., (2015) On pins and needles: How vaccines are portrayed on Pinterest, Vaccines, Vol 33 (39),pp.5051-5056
11 Jolley, D. and K. Douglas, (2014) The Effects of Anti-Vaccine Conspiracy Theories on Vaccination Intentions,
PLOS ONE 9(2)10But first, how did we get to this point? Certainly, the 2016 US Presidential election led to an
immediate search for answers from those who had not considered the possibility of a Trump
victory—namely the major news outlets, pundits and pollsters. And while the US election
result was caused by an incredibly complex set of factors – socio-economic, cultural, politicaland technological – there was a desire for simple explanations, and the idea that fabricatednews sites could provide those explanations drove a frenzied period of reporting, conferences
and workshops.12
Reporting by Buzzfeed News’ Craig Silverman provided an empirical framework for these
discussions, offering evidence that the most popular of these fabricated stories were shared
more widely than the most popular stories from the mainstream media: “In the final threemonths of the US presidential campaign, 20 top-performing false election stories from hoax
sites and hyper-partisan blogs generated 8,711,000 shares, reactions, and comments on
Facebook. Within the same time period, the 20 best-performing election stories from 19
major news websites generated a total of 7,367,000 shares, reactions, and comments on
Facebook.”13
In addition, research on referral data shows that “fake news” stories relied heavily on social
media for traffic during the election14. Only 10.1% of traffic to the top news sites came fromsocial media, compared with 41.8% for ‘fake news sites’. (Other traffic referral types weredirect browsing, other links and search engines.)
While we know that mis-information is not new, the emergence of the internet and social
technology have brought about fundamental changes to the way information is produced,
communicated and distributed. Other characteristics of the modern information environment
include:a) Widely accessible, cheap and sophisticated editing and publishing technology has
made it easier than ever for anyone to create and distribute content;

12 See write-ups of some of the most important events here: Shorenstein Center’s ‘Fake News Agenda for

Research and Action: (https://shorensteincenter.org/combating-fake-news-agenda-for-research/); and Yale
University’s Information Society Project’s Fighting Fake News Workshop
https://law.yale.edu/system/files/area/center/isp/documents/fighting_fake_news_-_workshop_report.pdf and theWestminster Media Forum Keynote Seminar: Fake news - scope, public trust and options for policy,
http://blogs.lse.ac.uk/mediapolicyproject/2017/08/10/the-evolving-conversation-around-fake-news-and-potentialsolutions/
13 Silverman, C. (2016b) This Analysis Shows How Viral Fake Election News Stories Outperformed Real NewsOn Facebook, Buzzfeed News, November 16, 2016. https://www.buzzfeed.com/craigsilverman/viral-fake-electionnews-outperformed-real-news-on-facebook
14 Allcott, H., & Gentzkow, M. (2017). Social Media and Fake News in the 2016 Election. National Bureau of
Economic Research. Retrieved from http://www.nber.org/papers/w23089

11b) Information consumption, which was once private, has become public because ofsocial media;c) The speed at which information is disseminated has been supercharged by anaccelerated news cycle and mobile handsets;d) Information is passed in real-time between trusted peers, and any piece of
information is far less likely to be challenged.
As Frederic Filloux explained: “What we see unfolding right before our eyes is nothing lessthan Moore’s Law applied to the distribution of mis-information: an exponential growth ofavailable technology coupled with a rapid collapse of costs.”15
A study conducted in eighteen countries by the BBC World Service in September 2017 foundthat 79% of respondents said they worried about what was fake and what was real on theinternet.16 Brazilians were most troubled, with 92% of respondents from that country
expressing some concern about the issue. The least concerned were Germans, where 51% of
respondents indicated that they were worried. Unfortunately, we don’t have similar datafrom previous years to understand whether concern has increased in light of recent
discussions about the phenomenon. But one thing to bear in mind is that when the purposeof Russian dis-information campaigns is to sow mistrust and confusion about what sources of
information are authentic, it is important that we continue to track attitudes about the
information people source from the internet.
Another critical point is that popular social networks make it difficult for people to judge thecredibility of any message, because posts from publications as unlike as the New York Times
and a conspiracy site look nearly identical. This means that people are increasingly reliant on
friends and family members to guide them through the information ecosystem. As Messing
and Westwood have argued, “social media has had two effects: by collating stories frommultiple sources, the focus is on the story, and not on the source; secondly, endorsements
and social recommendations guide readership” 17 rather than traditional gatekeepers or
ingrained reading habits.

15 Filloux, F. (2017) You can’t sell news for what it costs to make, The Walkley Magazine on Medium,https://medium.com/the-walkley-magazine/you-cant-sell-news-for-what-it-costs-to-make-7a4def964ffa
16 Cellan-Jones, R. (Sept 22, 2017) Fake news worries 'are growing' suggests BBC poll, BBC News,
http://www.bbc.com/news/technology-41319683
17 Messing, S., & Westwood, S. J. (2014). Selective exposure in the age of social media: Endorsements trump
partisan source affiliation when selecting news online. Communication Research, 41(8), 1042-1063.

12

Daily, we spend twice as much time online compared with 2008. During that protracted
amount time, we consume incredible amounts of information 18 and inevitably make mistakes.
Recent research by Filippo Menczer and colleagues shows we are so utterly inundated that
we share untruths. Parsing information and judging the credibility of sources on Facebook orother social platforms will require our brains to adapt with new cognitive strategies forprocessing information. But Facebook is only 13 years old.19
Social networks are driven by the sharing of emotional content. The architecture of thesesites is designed such that every time a user posts content—and it is liked, commented uponor shared further— their brain releases a tiny hit of dopamine. As social beings, we intuit thetypes of posts that will conform best to the prevailing attitudes of our social circle. 20 And so,on this issue of information disorder, this performative aspect of how people use social
networks is critical to understanding how mis- and dis-information spreads.
However, we must also recognize the role of television in spreading dis-information.21 While
much has been written about the growing influence of Sputnik and Russia Today22, as well as
its new youth channel, In the Now, the unintentional amplification of dis-information by themainstream media across the world needs to be acknowledged. From the New York Times’
inaccurate reporting on Iraq’s weapons of mass destruction, to the wall-to-wall coverage of
Hillary Clinton’s leaked emails (now known to be carried out by Russian hackers), or thealmost daily amplification of Trump’s tweets (some including information from conspiracy
sites23), getting the mainstream media to amplify rumour and dis-information is the ultimate
goal of those who seek to manipulate. Without amplification, dis-information goes nowhere.
It is within this context that we have to study information disorder. These technology
platforms are not neutral communication pipelines. They cannot be, as they are inherently
social, driven by billions of humans sharing words, images, videos and memes that affirm theirpositions in their own real-life social networks.

18 Meeker, M (2017) Internet Trends, 2017. P.9. Available at:

http://dq756f9pzlyr3.cloudfront.net/file/Internet+Trends+2017+Report.pdf
19 Qiu, X. et al. (2017) Limited individual attention and online virality of low-quality information, Nature Human
Behaviour, Vol 1
20
Derakhshan blames web 2.0 for starting this ‘tyranny of the novel and the popular’ which then moved on tosocial platforms. See Derakhshan H. (July 14, 2015) The Web We Have to Save. Matter.
https://medium.com/matter/the-web-we-have-to-save-2eb1fe15a426
21 Derakhshan H. (Nov 29, 2016) Social Media Is Killing Discourse Because It’s Too Much Like TV, MIT
Technology Review. https://www.technologyreview.com/s/602981/social-media-is-killing-discourse-because-itstoo-much-like-tv/
22
Rutenberg, J. (Sept. 13, 2017) RT, Sputnik and Russia’s New Theory of War, New York Times,
https://www.nytimes.com/2017/09/13/magazine/rt-sputnik-and-russias-new-theory-of-war.html?
23 Benkler Y. et al (March 3, 2017) Study: Breitbart-led right-wing media ecosystem altered broader media
agenda. Columbia Journalism Review. https://www.cjr.org/analysis/breitbart-media-trump-harvard-study.php

13Currently, Western democracies are worried about how, in a diverse global media system,
potential rivals like ISIS and Russia can use dis-information to further expand their reach into
legislative and executive branches of power. The shock of the Brexit referendum, the US
election, Le Pen reaching the run-off vote in the French election and the overturning of the
Kenyan election have been used as examples of the potential power of systematic disinformation campaigns. However, empirical data about the exact influence of such campaigns
does not exist.
As danah boyd argues about recent responses to fears about mis- and dis-information, “It’spart of a long and complicated history, and it sheds light on a variety of social, economic,
cultural, technological, and political dynamics that will not be addressed through simplistic
solutions.”24 Certainly, we have to look for explanations for how societies, particularly in theWest, have become so segregated in terms of terms of age, race, religion, class and politics.25
Recognizing the impact of factors such as the collapse of the welfare state, the failure of
democratic institutions to provide public services, climate change and miscalculated foreigninterventions are required. We cannot see the phenomenon of mis- and dis-information in
isolation, but must consider its impact amid the new-media ecosystem. This ecosystem isdominated by increasingly partisan radio, television and social media; exaggerated emotional
articulations of the world; quick delivery via algorithmically derived feeds on smartphones
and audiences that skim headlines to cope with the floods of information before them.
Making sense of mis-, dis- and mal-information as a type of information disorder, and learning
how it works, is a necessity for open democracies. Likewise, neglecting to understand thestructural reasons for its effectiveness is a grave mistake.
Communication as Ritual
One of the most important communication theorists, James Carey, compared two ways ofviewing communication - transmission and ritual - in his book Communication as Culture:
Essays on Media and Society.26
Carey wrote, “The transmission view of communication is the commonest in our culture—
perhaps in all industrial cultures… It is defined by terms such as ‘imparting,’ ‘sending,’‘transmitting,’ or ‘giving information to others.’”27 The ‘ritual view of communication’, by24 boyd, d. (March 27,2017) “Google and Facebook can’t just make fake news disappear”, Backchannel,

https://medium.com/backchannel/google-and-facebook-cant-just-make-fake-news-disappear-48f4b4e5fbe8
25 Iyengar S. and S. J. Westwood (2015) Fear and Loathing across Party Lines: New Evidence on Group
Polarization, American Journal of Political ScienceVol. 59, No. 3 (July 2015), pp. 690-707
26 Carey, J. (1989) Communication as Culture: Essays on Media and Society, London: Routledge.
27 Carey, (1989), p.18

14

contrast, is not about “the act of imparting information but the representation of sharedbeliefs.”
Under a transmission view of communication, one sees the newspaper as an instrument for
disseminating knowledge. Questions arise as to its effects on audiences—as enlightening or
obscuring reality, as changing or hardening attitudes or as breeding credibility or doubt.
However, a ritual view of communication does not consider the act of reading a newspaper tobe driven by the need for new information. Rather, it likens it to attending a church service.
It’s a performance in which nothing is learned, but a particular view of the world is portrayedand confirmed. In this way, news reading and writing is a ritualistic and dramatic act. 28
In this report, we pay close attention to social and psychological theories that help to makesense of why certain types of dis-information are widely consumed and shared. Considering
information consumption and dissemination from merely the transmission view is unhelpful
as we try and understand information disorder.
Four Key Points
The term ‘fake news’ and the need for definitional rigour
Before we continue, a note on terminology. One depressing aspect of the past few months isthat, while it has resulted in an astonishing number of reports, books, conferences andevents, it has produced little other than funding opportunities for research and thedevelopment of tools. One key reason for this stagnation, we argue, is an absence ofdefinitional rigour, which has resulted in a failure to recognize the diversity of mis- and disinformation, whether of form, motivation or dissemination.
As researchers like Claire Wardle29, Ethan Zuckerman30, danah boyd31 and Caroline Jack32 andjournalists like the Washington Post’s Margaret Sullivan33 have argued, the term ‘fake news’ iswoefully inadequate to describe the complex phenomena of mis- and dis-information. As
Zuckerman states, “It’s a vague and ambiguous term that spans everything from false balance(actual news that doesn’t deserve our attention), propaganda (weaponized speech designed
28 Carey (1989), pp.20-21
29 Wardle, C. (Feb 16, 2017) Fake News. It’s Complicated, First Draft, https://firstdraftnews.com/fake-news-

complicated/
30 Zuckerman, E. (Jan 30. 2017) Stop Saying Fake News, It’s not Helping, My Heart is in Accra,
http://www.ethanzuckerman.com/blog/2017/01/30/stop-saying-fake-news-its-not-helping/
31 boyd, d. (March 27, 2017) Google and Facebook can’t just make Fake News Disappear, Wired,https://www.wired.com/2017/03/google-and-facebook-cant-just-make-fake-news-disappear/
32 Jack, C. (2017) Lexicon of Lies, Data & Society,https://datasociety.net/pubs/oh/DataAndSociety_LexiconofLies.pdf
33
Sullivan, M (Jan 6, 2017,) It’s Time To Retire the Tainted Term Fake News, Washington Post,
https://www.washingtonpost.com/lifestyle/style/its-time-to-retire-the-tainted-term-fakenews/2017/01/06/a5a7516c-d375-11e6-945a-76f69a399dd5_story.html

15

to support one party over another) and disinformatzya (information designed to sow doubt
and increase mistrust in institutions).”34
A study by Tandoc et al., published in August 2017, examined 34 academic articles that used
the term ‘fake news’ between 2003 and 2017.35 The authors noted that the term has beenused to describe a number of different phenomena over the past 15 years: news satire, news
parody, fabrication, manipulation, advertising and propaganda. Indeed, this term has a long
history, long predating President Trump’s recent obsession with the phrase.
The term “fake news” has also begun to be appropriated by politicians around the world todescribe news organisations whose coverage they find disagreeable. In this way, it’s becominga mechanism by which the powerful can clamp down upon, restrict, undermine andcircumvent the free press. It’s also worth noting that the term and its visual derivatives (e.g.,
the red ‘FAKE’ stamp) have been even more widely appropriated by websites, organisations
and political figures identified as untrustworthy by fact-checkers to undermine opposing
reporting and news organizations.36 We therefore do not use the term in this report andargue that the term should not be used to describe this phenomenon.
Many have offered new definitional frameworks in attempts to better reflect the complexities
of mis- and dis-information. Facebook defined a few helpful terms in their paper on
information operations:
1. Information (or Influence) Operations. Actions taken by governments or organized
non-state actors to distort domestic or foreign political sentiment, most frequently toachieve a strategic and/or geopolitical outcome. These operations can use acombination of methods, such as false news, dis-information or networks of fakeaccounts aimed at manipulating public opinion (false amplifiers).
2. False News. News articles that purport to be factual, but contain intentional
misstatements of fact to arouse passions, attract viewership or deceive.
3. False Amplifiers. Coordinated activity by inauthentic accounts that has the intent ofmanipulating political discussion (e.g., by discouraging specific parties fromparticipating in discussion or amplifying sensationalistic voices over others).
In ‘Fake News. It’s Complicated’, Wardle outlines seven types of mis- and dis-information,
revealing the wide spectrum of problematic content online, from satire and parody (which,

34 Zuckerman, (2017).
35 Tandoc, Jr., E. C, Lim, Z. W., and Ling, R. (Aug. 2017) Defining ‘Fake News’: A Typology of Scholarly

Definitions, Digital Journalism, 5 (7): 1-17
36 Haigh et al, (2017) Stopping Fake News: The work practices of peer-to-peer counter propaganda. Journalism
Studies, 1-26.

16

while a form of art, can become mis-information when audiences misinterpret the message)
to full-blown fabricated content.

Figure 1: 7 Types of Mis- and Dis-information (Credit: Claire Wardle, First Draft)
While these seven classifications are helpful in encouraging people to see beyond theinfamous ‘Pope endorses Trump’-type news sites that received so much attention after the
US election, the phenomenon requires an even more nuanced conceptual framework—
particularly one that highlights the impact of visuals in perpetuating dis-information. We havetherefore created such a framework, and we will use as the organizing structure for thereport.
While we work through terms and descriptions, it’s important that we recognise theimportance of shared definitions. As Caroline Jack argued in the introduction to her recent
report, Lexicon of Lies, for Data & Society:“Journalists, commentators, policymakers, and scholars have a variety of words attheir disposal — propaganda, dis-information, mis-information, and so on — todescribe the accuracy and relevance of media content. These terms can carry a lot ofbaggage. They have each accrued different cultural associations and historicalmeanings, and they can take on different shades of meaning in different contexts.
These differences may seem small, but they matter. The words we choose to describe
media manipulation can lead to assumptions about how information spreads, who17spreads it, and who receives it. These assumptions can shape what kinds ofinterventions or solutions seem desirable, appropriate, or even possible.”37

Visuals, Visuals, Visuals
As well as the other problematic aspects of the popular term ‘fake news’ outlined above, it
has also allowed the debate to be framed as a textual problem. The focus on fabricated news‘sites’ means the implications of misleading, manipulated or fabricated visual content,
whether that’s an image, a visualization, a graphic, or a video are rarely considered. The
solutions by the technology companies have been aimed squarely at articles, and while
admittedly that is because natural language processing is more advanced, and therefore text
is easier to analyse computationally, the framing of the debate as ‘fake news’, has not helped.
As we describe in this report, visuals can be far more persuasive than other forms ofcommunication38, which can make them much more powerful vehicles for mis- and disinformation. In addition, over the past couple of months, we’ve been confronted with thetechnological implications whereby relatively limited audio or video clips of someone can act
as very powerful ‘training data’ allowing for the creation of completely fabricated audio orvideo files, making it appear that someone has said something that they have not. 39
Source-Checking vs Fact-Checking
There is much discussion of fact-checking in this report. There has been an explosion ofprojects and initiatives around the world, and this emphasis on providing additional context
to public statements is a very positive development. Many of these organisations are focusedon authenticating official sources: politicians, reports by think tanks or news reports (a list ofEuropean fact-checking organization are listed in Appendix A), but in this age of disinformation where we are increasingly seeing information created by unofficial sources (fromsocial media accounts we don’t know, or websites which have only recently appeared), we
argue that we need to be doing source-checking as well as fact-checking.
Increasingly, when assessing the credibility of a piece of information, the source who
originally created the content or first shared it, can provide the strongest evidence about
whether something is accurate. Newsrooms, and people relying on social media for
information, need to be investigating the source, almost before they look at the content itself.
For example, routinely people should be researching the date and location embedded in
37 Jack, C. (August 2017) Lexicon of Lies, Data & Society,https://datasociety.net/pubs/oh/DataAndSociety_LexiconofLies.pdf
38 Birdsell, D. S., & Groarke, L. (1996). Toward a theory of visual argument. Argumentation and Advocacy,33(1), 1-10
39 WNYC Radio Lab (July 27, 2017) Breaking News, http://www.radiolab.org/story/breaking-news/

18

domain registration information of a supposed ‘news site’ to seeing whether it was created
two weeks ago in Macedonia. Similarly, people should be instinctively checking whether aparticular tweeted message has appeared elsewhere, as it could be that the same message
was tweeted out by ten different accounts at exactly the same time, and six of them werelocated in other countries. Newsrooms in particular need more powerful tools to be able tovisually map online networks and connections to understand how dis-information is being
created, spread and amplified.
Strategic Silence
Newsrooms also need more powerful tools to help them understand how dis-information ismoving across communities. In election monitoring projects First Draft has been involved within France, the UK and Germany, Newswhip (a platform which helps newsrooms discovercontent before it goes viral) was used as a way of monitoring whether a piece of misleading,manipulated or fabricated content was predicted to be shared widely. Newswhip has a
prediction algorithm which allows the user to see how many social interactions a piece ofcontent has received at any given moment and to offer a prediction about how many
interactions it would have twenty-four hours later. First Draft used this technology to inform
decisions about what stories to debunk and which ones to ignore. If certain stories, rumoursor visual content, however problematic, were not gaining traction, a decision was made notto provide additional oxygen to that information. The media needs to consider that publishing
debunks can cause more harm than good, especially as agents behind dis-information
campaigns see media amplification as a key technique for success. Debunks themselves canbe considered a form of engagement. The news industry needs to come together to think
about the implications of this type of reporting and the philosophical and practical aspects ofincorporating these ideas related to strategic silence.
The Report
The report starts with a new conceptual framework for talking about information disorder,
including three types, three phases and three elements. We then consider the specificchallenges of filter bubbles and echo chambers, before moving on to examine the solutions
that have been put it place to date (including those by the technology companies, education
initiatives, the media and regulatory bodies). We end the report with a look at future trends,
before wrapping up with some conclusions, and additional details about the thirty-fourrecommendations we are proposing.

19

Part 1: Conceptual Framework
Our conceptual framework has three components, each of which is also broken down into
three parts:
1) The Three Types of Information Disorder: Dis-information, Mis-information and
Mal-information
2) The Three Phases of Information Disorder: Creation, Production and Distribution
3) The Three Elements of Information Disorder: Agent, Message and Interpreter

The Three Types of Information Disorder
Much of the discourse on ‘fake news’ conflates three notions: mis-information, disinformation and mal-information. But it’s important to distinguish messages that are true
from those that are false, and messages that are created, produced or distributed by “agents”
who intend to do harm from those that are not:● Dis-information. Information that is false and deliberately created to harm a person,social group, organization or country.
● Mis-information. Information that is false, but not created with the intention of
causing harm.
● Mal-information. Information that is based on reality, used to inflict harm on a
person, organization or country.

Figure 2: Examining how mis-, dis- and mal-information intersect around the concepts offalseness and harm. We include some types of hate speech and harassment under the malinformation category, as people are often targeted because of their personal history or

20

affiliations. While the information can sometimes be based on reality (for example targetingsomeone based on their religion) the information is being used strategically to cause harm.
The 2017 French Presidential election 40 provides examples that illustrate all three types of
information disorder.
1) Examples of dis-information:
One of the most high profile hoaxes of the campaign, was the creation of a sophisticatedduplicate version of the Belgian newspaper Le Soir, with a false article claiming that Macron
was being funded by Saudi Arabia.41 Another example was the circulation of documents
online claiming falsely that Macron had opened an offshore bank account in the Bahamas. 42
And finally, dis-information circulated via ‘Twitter raids’ in which loosely connected networks
of individuals simultaneously took to Twitter with identical hashtags and messages to spreadrumours about Macron (e.g., that he was in a relationship with his step-daughter).
2) Examples of mis-information:
The attack on the Champs Elysees on 20 April 2017 inspired a great deal of mis-information43,
as is the case in almost all breaking news situations. Individuals on social media unwittingly
published a number of rumours, for example the news that a second policeman had been
killed. The people sharing this type of content are rarely doing so to cause harm. Rather, they
are caught up in the moment, trying to be helpful, and fail to adequately inspect the
information they are sharing.
3) Examples of mal-information:
One striking example of mal-information occurred when Emmanuel Macron’s emails wereleaked the Friday before the run-off vote on 7 May. The information contained in the emails
was real, although Macron’s campaign allegedly included false information to diminish theimpact of any potential leak. 44 However, by releasing private information into the public

40 For an in depth analysis of Dis-information and the French President Election see also: Bakamo (2017a) The

Role and Impact of Non-Traditional Publishers in the 2017 French Presidential Election. Available at:https://www.bakamosocial.com/frenchelection/ andBakamo (2017b) Patterns of Dis-information in the 2017 French Presidential Election. Available at: Available at:https://www.bakamosocial.com/frenchelection/
41 CrossCheck, (March 2, 2017) Was Macron’s Campaign for the French Presidency Funded by Saudi Arabia?
CrossCheck, https://crosscheck.firstdraftnews.com/checked-french/macrons-campaign-french-presidencyfinanced-saudi-arabia/
42 CrossCheck (May 5, 2017) Did Emmanuel Macron Open an Offshore Account? CrossCheck,
https://crosscheck.firstdraftnews.com/checked-french/emmanuel-macron-open-offshore-account/
43 One example was the rumour that London Muslims were celebrating the attack on the Champs Elysee, which
was debunked by the CrossCheck project: CrossCheck, (April 22, 2017) Did London Muslims ‘celebrate’ aterrorist attack on the Champs-Elysees? CrossCheck, https://crosscheck.firstdraftnews.com/checkedfrench/london-muslims-celebrate-terrorist-attack-champs-elysees/
44 Rachel Donadio, (May 8, 2017) Why the Macron Hacking Attack Landed with a Thud in France, The New YorkTimes, https://www.nytimes.com/2017/05/08/world/europe/macron-hacking-attack-france.html

21sphere minutes before the media blackout in France, the leak was designed to cause
maximum harm to the Macron campaign.
In this report, our primary focus is mis- and dis-information, as we are most concerned about
false information and content spreading. However, we believe it’s important to consider thisthird type of information disorder and think about how it relates to the other two categories.
However, hate speech, harassment and leaks raise a significant number of distinct issues, andthere is not space in this report to consider those as well. The research institute Data &Society is doing particularly good work on mal-information and we would recommend reading
their report Media Manipulation and Disinformation Online.45

The Phases and Elements of Information Disorder
In trying to understand any example of information disorder, it is useful to consider it in threeelements:
1) Agent. Who were the ‘agents’ that created, produced and distributed the example,
and what was their motivation?
2) Message. What type of message was it? What format did it take? What were thecharacteristics?
3) Interpreter. When the message was received by someone, how did they interpret themessage? What action, if any, did they take?

Figure 3: The Three Elements of Information Disorder
We argue that it is also productive to consider the life of an example of information disorderas having three phases:
1. Creation. The message is created.
2. Production. The message is turned into a media product.
3. Distribution. The message is distributed or made public.
45 Marwick, A and R. Lewis (May 2017) Media Manipulation and Dis-information Online, Data & Society,https://datasociety.net/output/media-manipulation-and-disinfo-online/,

22

Figure 4: The Three Phases of Information DisorderIn particular, it’s important to consider the different phases of an instance of information
disorder alongside its elements, because the agent that creates the content is often
fundamentally different from the agent who produces it. For example, the motivations of themastermind who ‘creates’ a state-sponsored dis-information campaign are very different
from those of the low-paid ‘trolls’ tasked with turning the campaign’s themes into specificposts. And once a message has been distributed, it can be reproduced and redistributed
endlessly, by many different agents, all with different motivations. For example, a social
media post can be distributed by several communities, leading its message to be picked up
and reproduced by the mainstream media and further distributed to still other communities.
Only by dissecting information disorder in this manner can we begin to understand thesenuances.
In the next two sections, we will examine these elements and phases of information disorderin more detail.

The Three Phases of Information Disorder
To examine how the phases of creation, production and distribution help us understandinformation disorder, let’s use the example of the article ‘Pope Francis Shocks World,
Endorses Donald Trump for President, Releases Statement’ published on the self-proclaimed
fantasy news site WTOE 5 in July 2016. For an in-depth analysis of this article and the networkof sites connected to it, we would recommend reading ‘The True Story Behind The Biggest
Fake News Hit Of The Election’ from Buzzfeed.46
46

Craig Silverman (Dec 2016) The True Story Behind The Biggest Fake News Hit Of The Election, Buzzfeed,
https://www.buzzfeed.com/craigsilverman/the-strangest-fake-newsempire?utm_term=.yrzPyEpLXq#.nqXD9N7opO

23

Figure 5: Screenshot of the fabricated news article published in July 2016 on WTOE5News.com(The site no longer exists).

If we think about the three phases in this example, we can see how different agents wereinvolved in creating the impact of this content.

24

Figure 6: Using the example of the ‘Pope Francis Shocks World, Endorses Donald Trump for
President, Releases Statement’ fabricated news articles to test the Three Phases ofInformation Disorder
The role of the mainstream media as agents in amplifying (intentionally or not) fabricated or
misleading content is crucial to understanding information disorder. Fact-checking has always
been fundamental to quality journalism, but the techniques used by hoaxers and those
attempting to disseminate dis-information have never been this sophisticated. With
newsrooms increasingly relying on the social web for story ideas and content, forensic
verification skills and the ability to identify networks of fabricated news websites and bots ismore important than ever before.

The Three Elements of Information Disorder
The Agent
Agents are involved in all three phases of the information chain – creation, production anddistribution – and have various motivations. Importantly, the characteristics of agents canvary from phase to phase.
We suggest seven questions to ask about an agent:1) What type of actor are they?
Agents can be official, like intelligence services, political parties, news organizations.
They can also be unofficial, like groups of citizens that have become evangelized about25

an issue.
2) How organized are they?
Agents can work individually, in longstanding, tightly-organized organizations (e.g., PR
firms or lobbying groups) or in impromptu groups organized around common
interests.
3) What are their motivations?
There are four potential motivating factors: Financial: Profiting from information
disorder through advertising; Political: Discrediting a political candidate in an election
and other attempts to influence public opinion; Social: Connecting with a certain
group online or off; and Psychological: Seeking prestige or reinforcement.
4) Which audiences to they intend to reach?
Different agents might have different audiences in mind. These audiences can varyfrom an organization’s internal mailing lists or consumers, to social groups based on
socioeconomic characteristics, to an entire society.
5) Is the agent using automated technology?
The ability to automate the creation and dissemination of messages online has
become much easier and, crucially, cheaper. There is much discussion about how todefine a bot. One popular definition from the Oxford Internet Institute is an accountthat posts more than 50 times a day, on average. Such accounts are often automated,but could conceivably be operated by people. Other accounts, known as cyborgs, areoperated jointly by software and people.
6) Do they intend to mislead?
The agent may or may not intend to deliberately mislead the target audience.
7) Do they intend to harm?
The agent may or may not intend deliberately to cause harm
The Message
Messages can be communicated by agents in person (via gossip, speeches, etc.), in text
(newspaper articles or pamphlets) or in audio/visual material (images, videos, motiongraphics, edited audio-clip, memes, etc.). While much of the current discussion about ‘fakenews’ has focused on fabricated text articles, mis- and dis-information often appears in visual
formats. This is important, as technologies for automatically analysing text are significantly
different from those for analysing still and moving imagery.
We offer five questions to ask about a message:26

1. How durable is the message?
Some messages are designed to stay relevant and impactful for the long term
(throughout an entire war or in perpetuity). Others are designed for the short term
(during an election) or just one moment, as in the case of an individual message during
a breaking news event.
2. How accurate is the message?
The accuracy of a message is also important to examine. As discussed earlier, malinformation is truthful information used to harm (either by moving private information
into the public arena or using people’s affiliations, like their religion, against them).
For inaccurate information, there is a scale of accuracy from false connection (aclickbait headline that is mismatched with its article’s content) to 100% fabricatedinformation.
3. Is the message legal?
The message might be illegal, as in the cases of recognised hate speech, intellectual
property violations, privacy infringements or harassment. Of course, what messages
are legal differs by jurisdiction.
4. Is the message ‘imposter content’, i.e. posing as an official source?
The message may use official branding (e.g., logos) unofficially, or it may steal thename or image of an individual (e.g., a well-known journalist) in order to appear
credible.
5. What is the message’s intended target?
The agent has an intended audience in mind (the audience they want to influence) but
this is different to the target of the message (those who are being discredited). The
target can be an individual (a candidate or a political or business leader), an
organisation (a private firm or a government agency), a social group (a race, ethnicity,
the elite, etc.) or an entire society.
The Interpreter
Audiences are very rarely passive recipients of information. An ‘audience’ is made up of many
individuals, each of which interprets information according to his or her own socio-cultural
status, political positions and personal experiences.
As outlined earlier, understanding the ritualistic aspect of communication is critical forunderstanding how and why individuals react to messages in different ways. The types of
information we consume, and the ways in which we make sense of them, are significantly
impacted by our self-identity and the ‘tribes’ we associate with. And, in a world where what
we like, comment on and share is visible to our friends, family and colleagues, these 'social'
and performative forces are more powerful than ever.

27

Having to accept information that challenges our sense of self can be jarring. Irrespective of
how persuasive a message may appear to a neutral observer, it is easier to ignore or resist
information that opposes our own worldview. Certainly, evidence suggests that fact-checks
do tend to nudge individuals’ knowledge in the direction of the correct information, but it
certainly doesn’t replace the mis- or dis-information entirely.
This reality complicates our search for solutions to information disorder. If we accept thathuman brains do not always work rationally, simply disseminating more quality information is
not the answer. Solutions must grapple with the social and performance characteristics thathave helped make certain fabricated content so popular on Facebook. How, for example, canwe make sharing false information publicly shameful and embarrassing? What can we learn
from the theories of performativity, particularly in performance and identity management inan online setting that could help us experiment with some potential solutions?
What the ‘interpreter’ can do with a message highlights how the three elements of
information disorder should be considered parts of a potential never-ending cycle. In an eraof social media, where everyone is a potential publisher, the interpreter can become the next
‘agent,’ deciding how to share and frame the message for their own networks. Will they show
support for the message by liking or commenting on it, or will they decide to share themessage? If they do share the message, have they done so with the same intent as theoriginal agent, or will they share it to, for example, show their disagreement?

Figure 7: Questions to ask about each element of an example of information disorder28

Figure 8: Using the ‘Three Elements of Information Disorder’ to examine the ‘Pope Endorses
Trump’ article
In the next section, we will review literature that helps provide a deeper historical andtheoretical understanding of the three elements of information disorder.
1) The Agents: Who are they and what motivates them?
In this section, we explore the role of agents, or those who create, produce and distributemessages. Again, the motivations of the person who creates and posts a meme on an inviteonly chat group on Discord could be different from the person who sees the meme on their
Facebook feed and shares it with a WhatsApp group.
Official vs Unofficial Actors?
When official actors are involved, the sophistication, funding and potential impact of amessage or campaign of systematic messages is far greater. Much has been written about the
impact of Russian propaganda on information ecosystems in Europe and further afield. One of
the most notable is the Rand Corporation’s report from July 2016, entitled “The Russian‘Firehose of Falsehood’ Propaganda Model,” 47 which identified four characteristics of modern
Russian propaganda:
1.
47

Voluminous and multi-channeledPaul, Christopher and Miriam Matthews, (June 20, 2016) The Russian “Firehose of Falsehood” Propaganda
Model: Why It Might Work and Options to Counter It, Santa Monica, Calif.: RAND Corporation,
https://www.rand.org/pubs/perspectives/PE198.html

29

2.
3.
4.

Rapid, continuous and repetitive
Noncommittal to objective reality
Inconsistent in its messaging

The EU Stratcomm Taskforce provides regular analysis of Russian propaganda messaging
across the European Union.48. Likewise, their research shows that a key strategy of Russia isto spread as many conflicting messages as possible, in order to persuade audiences that there
are too many versions of events to find the truth. As they explain, “Not only (are) big media
outlets like Russia Today or Sputnik … deployed, but also seemingly marginal sources, like
fringe websites, blog sites and Facebook pages. Trolls are deployed not only to amplify disinformation messages but to bully those... brave enough to oppose them. And the network
goes wider: NGOs and “GONGOs” (government organised NGOs); Russian government
representatives; and other pro-Kremlin mouthpieces in Europe, often on the far-right and farleft. In all, literally thousands of channels are used to spread pro-Kremlin dis-information, allcreating an impression of seemingly independent sources confirming each other’s
message.”49
In April 2017, Facebook published a paper by three members of its Security team, entitled“Information Operations and Facebook,” that outlines the use of the platform by state actors.
They define information operations as “actions taken by organized actors (governments or
non-state actors) to distort domestic or foreign political sentiment, most frequently toachieve a strategic and/or geopolitical outcome. These operations can use a combination of
methods, such as false news, dis-information or networks of fake accounts aimed atmanipulating public opinion (we refer to these as ‘false amplifiers’).”50
While Russian propaganda techniques are the current focus of much concern, digitalastroturfing campaigns – that is, campaigns that use troll factories, click farms and automatedsocial media accounts – have been used by other state actors for years. A recent report by theComputational Propaganda Research Project tracked this activity across twenty-eight
countries, showing the scale of these operations. 51
Perhaps the most notable of these state actors is China, which has paid people to post
millions of fabricated social media posts per year, as part of an effort to “regularly distract the48 European Union’s East StratCom Task Force, https://euvsdisinfo.eu/
49 EU East StratCom Task Force, (Jan 19, 2017), Means, goals and consequences of the pro-Kremlin dis-

information campaign, ISPI http://www.ispionline.it/it/pubblicazione/means-goals-and-consequences-prokremlin-dis-information-campaign-16216
50 Jen Weedon, William Nuland and Alex Stamos (April 27, 2017) Information Operations and Facebook, p. 4
https://fbnewsroomus.files.wordpress.com/2017/04/facebook-and-information-operations-v1.pdf
51
Bradshaw, S. and P. Howard (August 2017) Troops, Trolls and Troublemakers: A Global
Inventory of Organized Social Media Manipulation
http://comprop.oii.ox.ac.uk/wp-content/uploads/sites/89/2017/07/Troops-Trolls-and-Troublemakers.pdf

30

public and change the subject” from any policy-related issues that threaten to incite
protests.52 In countries like Bahrain and Azerbaijan, there is evidence of PR firms creating fakeaccounts on social media to influence public opinion.53 Duterte’s government has usedsophisticated ‘astroturfing’ techniques to target individual journalists and news
organizations.54
Additionally, in South Africa, an email leak in May exposed large-scale dis-information efforts
by the powerful Gupta family to distract attention from its business dealings with thegovernment. These efforts included paying Twitter users to abuse journalists and spread disinformation and the use of bots to amplify fabricated stories.55
In contrast to official actors, unofficial actors are those who work alone or with loose
networks of citizens, and create false content to harm, make money, or entertain other likeminded people.
Following the outcry about the role of fabricated websites in the 2016 US election, journalists
tracked down some of these ‘unofficial’ agents. One was Jestin Coler, who, in an interview
with NPR, admitted that his “whole idea from the start was to build a site that could kind of
infiltrate the echo chambers of the alt-right, publish blatantly [false] or fictional stories andthen… publicly denounce those stories and point out the fact that they were fiction.” As NPR
explains, “[Coler] was amazed at how quickly fake news could spread and how easily people
believe[d] it.”56
How organised are the agents?
Trolls have existed since the internet was invented.57 Definitions vary, but one aspect is key:trolls provoke emotions by publicly offending their targets. Trolls are humans who post
behind a username or handle. Yet, similar to bots, they can amplify dis-information incoordinated ways to evoke conformity among others. What they do better than bots is target

52 King, G, J. Pan & M. Roberts, (May 2016) How the Chinese Government Fabricates Social Media Posts for

Strategic Distraction, not Engaged Argument, Harvard University,
http://gking.harvard.edu/files/gking/files/50c.pdf?m=1463587807
53 Woolley, S and P. Howard (2017) Social Media, Revolution and the Political Bot, in Routledge Handbook orMedia, Conflict and Security, edited by Piers Robinson, Philip Seib, Romy Frohlich, London:Routledge.
54
Julie Posetti, This is why Online Harassment Still Needs Attention, MediaShift July 2017,
http://mediashift.org/2017/07/online-harassment-still-needs-attention/
55 Eliseev, A (July 20, 2017) The Gupta scandal: how a British PR firm came unstuck in South Africa, The New
Statesman, http://www.newstatesman.com/culture/observations/2017/07/gupta-scandal-how-british-pr-firm-cameunstuck-south-africa
56 Sydell, L. (2017) (Nov 23, 2016) We Tracked Down A Fake-News Creator In The Suburbs. Here’s What We
Learned, NPR, http://www.npr.org/sections/alltechconsidered/2016/11/23/503146770/npr-finds-the-head-of-acovert-fake-news-operation-in-the-suburbs
57 De Seta, G., “Trolling, and Other Problematic Social Media Practices,” in The SAGE Handbook of Social
Media, ed. Jean Burgess, Alice E. Marwick, and Thomas Poell (Thousand Oaks, CA: SAGE Publications, 2017).

31

those who question the veracity of a piece of information. Trolls work efficiently to silence
naysayers in the early stages of dis-information distribution by posting personal attacks toundermine that person’s position on the board. And we know that some governments
organize agents to pursue specific messaging goals on social media, whether through bots,
cyborgs or ‘troll factories.’58
In the report entitled ‘Media Manipulation and Disinformation Online’ from Data & Society,
Alice Marwick and Rebecca Lewis analyzed ‘Gamergate’, an online campaign of bullying andharassment that took place in late 2014. They identified organized brigades, networked andagile groups, men’s rights activists and conspiracy theorists as exploiting “young men’s
rebellion and dislike of ‘political correctness’ to spread white supremacist thought,
Islamophobia, and misogyny through irony and knowledge of internet culture”.59
Buzzfeed’s Ryan Broderick examined similar, loosely-affiliated groups of Trump supporters in
the US who were active during the French election. 60 Using technologies like Discord (a set ofinvite-only chat rooms), Google documents, Google forms and Dropmark (a file-sharing site
like Dropbox), they organized ‘Twitter raids’ where they would simultaneously bombard
Twitter accounts they hoped to influence with messages using the same hashtags.
Analysis of mis-information during the French election by Storyful and the Atlantic Council
showed that such loose, online networks of actors push messages across different platforms.
Anyone wishing to understand their influence needs to monitor several closed and open
platforms. For example, in the context of the US election, Trump supporters produced and“audience-tested many anti-Clinton memes in 4Chan and fed the ones with the best
responses into the Reddit forum ‘The_Donald.’ The Trump campaign also monitored theforum for material to circulate in more mainstream social media channels.”61
Finally, it’s worth mentioning ‘fake tanks’, or partisan bodies disguised as think tanks. As
Transparify, the group that provides global ratings on the financial transparency of thinktanks, has explained, “[T]hese [fake tanks] range from essentially fictitious entities
purposefully set up to promote the very narrow agenda and vested interests of (typically one
single) hidden funder at one extreme, to more established organisations that work on
multiple policy issues but (occasionally or routinely) compromise their intellectual

58 Benedictus, L., “Invasion of the troll armies: from Russian Trump supporters to Turkish state stooges,”,

Guardian https://www.theguardian.com/media/2016/nov/06/troll-armies-social-media-trump-russian
59 Marwick, A and R. Lewis (May 2017) Media Manipulation and Dis-information Online, Data & Society,https://datasociety.net/output/media-manipulation-and-disinfo-online/, p.29
60 Broderick, R. (Jan 24, 2017) Trump Supporters Online Are Pretending To Be French To Manipulate France's
Election, Buzzfeed, https://www.buzzfeed.com/ryanhatesthis/inside-the-private-chat-rooms-trump-supporters-areusing-to?utm_term=.skLbZaN42#.xnOE5JybL
61 Shaffer, K. et al. (2017) Democracy Hacked. Available at: https://medium.com/data-for-democracy/democracyhacked-a46c04d9e6d1

32

independence and research integrity in line with multiple funders’ agendas and vested
interests...”Representatives from fake tanks “regularly appear on television, radio, or in newspaper
columns to argue for or against certain policies, their credibility bolstered by the abuse of thethink tank label and misleading job titles such as “senior scholar.” 62What is the motivation of the agent?
Looking into what motivates agents not only provides a deeper understanding of how dis- or
mal-information campaigns work, it also points to possible ways to resist them.
It is a mistake to talk generally about agents’ motivations, since they vary in each phase. It isquite likely that publishers (e.g., an editor of a cable news show) or distributors (e.g., a user
on a social network) of a message may not even be fully aware of the real purpose behind apiece of dis-information.
As we illustrate above, if a message is partly or entirely false, but no harm is intended by its
producer, it doesn’t fall under the definition of dis-information. For this reason, it’s important
to differentiate between mis-information (false, but not intended to harm) and malinformation (true, but intended to harm).
i) Political
Producers of dis-information campaigns, from Russia and elsewhere, sometimes have politicalmotivations. A great deal has been written about Russian dis-information activity in Europe,
but it’s worth quoting at length from a statement given by Constanze Stelzenmüller to the US
Senate Committee on Intelligence, in June 2017, on the subject of potential interference by
Russia in the German Federal elections:“Three things are new about Russian interference today. Firstly, it appears to be
directed not just at Europe’s periphery, or at specific European nations like Germany,
but at destabilizing the European project from the inside out: dismantling decades ofprogress toward building a democratic Europe that is whole, free, and at peace.
Secondly, its covert and overt “active measures” are much more diverse, larger-scale,and more technologically sophisticated; they continually adapt and morph inaccordance with changing technology and circumstances. Thirdly, by striking at Europe
and the United States at the same time, the interference appears to be geared
towards undermining the effectiveness and cohesion of the Western alliance assuch—and at the legitimacy of the West as a normative force upholding a global order62

Written evidence provided to the UK Parliamentary Inquiry on Fake News by Transparify
http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/culture-media-and-sportcommittee/fake-news/written/47967.html

33

based on universal rules rather than might alone. That said, Russia’s active measures
are presumably directed at a domestic audience as much as towards the West: They
are designed to show that Europe and the U.S. are no alternative to Putin’s Russia. Life
under Putin, the message runs, may be less than perfect; but at least it is stable.”63
In terms of Russian dis-information, one of the best sources of information is the EU East
StratCom Task Force,64 which has a site called ‘euvsdisinfo.eu’ that provides regular updates
about Russian dis-information campaigns across Europe. As they explain, “the dis-information
campaign is a non-military measure for achieving political goals. Russian authorities are
explicit about this, for example through the infamous Gerasimov doctrine and throughstatements by top Russian generals that the use of ‘false data’ and ‘destabilising propaganda’
are legitimate tools in their tool kit.” Elsewhere, the Task Force wrote, “The Russian Ministerof Defence describes information as ‘another type of armed forces.’”65
One critical aspect to understanding Russian dis-information, as noted by information warfare
expert Molly McKew, is that “information operations aim to mobilize actions and behavioral
change. It isn't just information.” 66 As the recent revelations about Russian operatives
purchasing dark ads on Facebook 67 and organizing protests via Facebook’s Events feature68
show, the aim of these acts is to create division along socio-cultural lines.
ii) Financial
Some of those who produce or distribute dis-information may do so merely for financial gain,
as in the case of PR firms and fabricated news outlets. Indeed, entire businesses might bebased on dis-information campaigns.69

63 Stelzenmüller, C. (June 28, 2017) Testimony to the US Senate Committee on Intelligrance: The impact ofRussian interference on Germany’s 2017 elections, Available at: https://www.brookings.edu/testimonies/theimpact-of-russian-interference-on-germanys-2017-elections/
64 For those interested in the subject of Russian dis-information, there is great deal of excellent analysis including
the recent report by Flemming Splidsboel Hansen of the Danish Institute for International study.
65 EU East StratCom Task Force, (Jan 19, 2017), Means, goals and consequences of the pro-Kremlin disinformation campaign, ISPI http://www.ispionline.it/it/pubblicazione/means-goals-and-consequences-prokremlin-dis-information-campaign-16216
66
Molly McKew on Twitter
https://twitter.com/MollyMcKew/status/907585015915171840?t=1&cn=ZmxleGlibGVfcmVjc18y&refsrc=email&iid=618ad44aaddf4116ac68a52cd832ee09&uid=20131383&nid=244+293670929
67 Stamos, A. (September 6, 2017) An Update On Information Operations On Facebook, Facebook Newsroom,https://newsroom.fb.com/news/2017/09/information-operations-update/
68 Facebook Events to Organize Anti-Immigrant Rallies on U.S. Soil, The Daily Beast,
http://www.thedailybeast.com/exclusive-russia-used-facebook-events-to-organize-anti-immigrant-rallies-on-ussoil
69 Tambini, D. (2017) How advertising fuels fake news. LSE Media Policy Project Blog,
http://blogs.lse.ac.uk/mediapolicyproject/2017/02/24/how-advertising-fuels-fake-news/

34

Fabricated ‘news’ websites created solely for profit have existed for years. Craig Silverman
documented some of the most prolific in the US in his 2015 report 70 for the Tow Center for
Digital Journalism. However, the US election shone a light on how many of these sites arelocated overseas, but aimed at US audiences. Buzzfeed was one of the first news
organisations to detail the phenomenon of English-language websites created by
Macedonians to capitalise on US readers’ enthusiasm for sensationalist stories.71 The smallcity of Veles in Macedonia produced “an enterprise of cool, pure amorality, free not only ofideology but of any concern or feeling about the substance of the election. These
Macedonians on Facebook didn’t care if Trump won or lost the White House. They only
wanted pocket money to pay for things.”72
This example from Veles also underscores the difficulty of assessing the true motivation of
any particular agent. The dominant narrative has been that these young people were
motivated by the financial benefits. We can assume this is true, as they undoubtedly mademoney, but we will unlikely ever know whether there was any coordinated attempt toencourage these teenagers to start this type of work in the first place.
‘Fake news’ websites make money through advertising. While Google and Facebook havetaken steps to prevent these sites from getting money through their ad networks, there arestill many other networks through which site owners can make money.
French startup Storyzy alerts brands when they appear on dubious websites. In a August 2017
write-up of their work, Frederic Filloux explains that over 600 brands had advertisements onquestionable sites. When they were approached for comment, Filloux concluded that few
cared, as long as their “overall return on investment was fine.” They certainly did not seem toconsider the ethical implications of helping to fuel a ‘vast network of mis-information.’73iii) Social and Psychological
While much of the debate around dis-information has focused on political and financial
motivations, we argue that understanding the potential social and psychological motivations
for creating dis-information is also worth exploring.

70

Silverman, C. (Feb. 2015) Lies, Damn Lies and Viral Content, Tow Center for Digital Journalism,
http://towcenter.org/research/lies-damn-lies-and-viral-content/
71 Silverman, C. and L. Alexander (Nov. 3, 2016) How Teens In The Balkans Are Duping Trump Supporters With
Fake News, Buzzfeed, https://www.buzzfeed.com/craigsilverman/how-macedonia-became-a-global-hub-for-protrump-misinfo?utm_term=.vxz8qAVJj#.vukajzr2R
72 Subramanian, S. (2017) Inside the Macedonian Fake News Complex, Wired, February 15, 2017. Wired.
Available at: https://www.wired.com/2017/02/veles-macedonia-fake-news
73
Filloux, F. (August 21, 2017) More than 600 global brands still feed the fake news ecosystem, The Monday
Note on Medium, https://mondaynote.com/more-than-600-global-brands-still-feed-the-fake-news-ecosystemd1ddfbd80458

35

For example, consider the motivation to simply cause trouble or entertain. There have always
been small numbers of people trying to ‘hoax’ the news media—from Tommaso Debenedetti,
who frequently uses fake Twitter accounts to announce the death of high profile people, 74 to
the person behind the ‘Marie Christmas’ account, who fooled CNN into thinking he or she wasa witness to the San Bernardino shooting. 75
Some share mis-information as a joke, only to find that people take it seriously. Most
recently, during Hurricane Harvey, Jason Michael McCann tweeted the old, already-debunked
image from Hurricane Sandy of a shark swimming in a flood highway. When Craig Silverman
reached out to him for comment, he explained, “Of course I knew it was fake, it was part of
the reason I shared the bloomin' thing... What I had expected was to tweet that and have my
1,300 followers in Scotland to laugh at it.”On more serious matters, the previously-mentioned research by Marwick and Lewis76 takes a
deep dive into alt-right communities and discusses the importance of considering their shared
identity in understanding their actions online.
Examining the audiences of hyper-partisans sites, such as Occupy Democrats in the US and
The Canary in the UK, we can also see the influence of political tribalism and identity. Thesetypes of sites do not peddle 100% fabricated content, but they are very successful in using
emotive (and some would argue misleading) headlines, images and captions – which are often
all of an article that’s read on platforms like Facebook – to get their audiences to share theirmessages.
In August 2017, Silverman and his colleagues at Buzzfeed published the most comprehensive
study to date of the growing universe of US-focused, hyper-partisan websites and Facebook
pages. They revealed that, in 2016 alone, at least 187 new websites launched, and that thecandidacy and election of Donald Trump “unleashed a golden age of aggressive, divisive
political content that reaches a massive amount of people on Facebook.” 7774 Kington, T. (March 30, 2010) Twitter hoaxer comes clean and says: I did it to expose weak media, The

Guardian, https://www.theguardian.com/technology/2012/mar/30/twitter-hoaxer-tommaso-de-benedetti
75 Buttry, S. (2015) ‘Marie Christmas:’ Some journalists fell for San Bernardino prank; others backed away, The
Buttry Diary, https://stevebuttry.wordpress.com/2015/12/03/the-case-of-marie-christmas-verifying-eyewitnessesisnt-simple-or-polite/
76 Marwick, A and R. Lewis (May 2017) Media Manipulation and Dis-information Online, Data & Society,https://datasociety.net/output/media-manipulation-and-disinfo-online/, p.29
77 Silverman et al. (August 8, 2017) Inside the Partsan Fight for your NewsFeed, Buzzfeed News,https://www.buzzfeed.com/craigsilverman/inside-the-partisan-fight-for-your-news-feed/36

Is the agent using automation?
Currently, machines are poor at creating dis-information, but they can efficiently publish anddistribute it. Recent research by Shao and colleagues concluded that “[a]ccounts that actively
spread mis-information are significantly more likely to be bots.” They also found that bots are“particularly active in the early spreading phases of viral claims, and tend to target influential
users.”78
Bots can manipulate majority-oriented platform algorithms to gain vast visibility and cancreate conformity among human agents who would then further distribute their messages.79
Many bots are designed to amplify the reach of dis-information80 and exploit thevulnerabilities that stem from our cognitive and social biases. They also create the illusionthat several individuals have independently come to endorse the same piece of information .81
As a recent report on computational amplification by Gu et al. concluded: “A properly
designed propaganda campaign is designed to have the appearance of peer pressure—bots
pretending to be humans, guru accounts that have acquired a positive reputation in social
media circles—these can make a propaganda campaign-planted story appear to be more
popular than it actually is.”82 Despite the platforms’ public commitment to stifle automatedaccounts, bots continue to amplify certain messages, hashtags or accounts, creating theappearance of certain perspectives being popular and, by implication, true.83
A recent report by NATO StratCom entitled ‘Robotrolling’ found that two in three Twitter
accounts posting in Russian about the NATO presence in the Baltics and Poland were bots.
They also found the density of bots is 2 to 3 times greater among Russian-tweeting accounts
than in English-tweeting accounts. The authors conclude that foreign-language sources onsocial networks are policed and moderated much less effectively than English-language
sources.84

78 Shao, C., G.L. Ciampaglia, O. Varol, A. Flammini and F. Menczer, (July 24, 2017) The spread of fake news

by social bots, https://arxiv.org/pdf/1707.07592.pdf
79 Woolley, S & P. Howard, (2017b) Computational Propaganda Worldwide: An Executive Summary,
http://comprop.oii.ox.ac.uk/wp-content/uploads/sites/89/2017/06/Casestudies-ExecutiveSummary.pdf
80 Shao, C. et al., (2016) Hoaxy: A Platform for Tracking Online Mis-information, Proceedings of the 25th
International Conference Companion on World Wide Web, pp. 745-750
81 Ratkiewicz et al., (2011) Detecting and Tracking Political Abuse in Social Media, Proceedings of the Fifth
International AAAI Conference on Weblogs and Social Media,
https://www.aaai.org/ocs/index.php/ICWSM/ICWSM11/paper/view/2850
82 Gu, L., V. Kropotov & F. Yarochkin, (June 2017), How Propagandists Abuse the Internet and Manipulate the
Public. Trend Micro, https://documents.trendmicro.com/assets/white_papers/wp-fake-news-machine-howpropagandists-abuse-the-internet.pdf
83 Ferrara, E. et al. (2016). The rise of social bots. Communications of the ACM, 59(7), p.101
84 NATO Strategic Communications Centre of Excellence (September 2017) Robotrolling,
http://www.stratcomcoe.org/robotrolling-20171

37

It also seems possible that there may be a black market for social bots. Ferrara found thatmany bots who supported Trump in the 2016 election also engaged with the #MacronLeaks
trend, but made few posts in-between.85
Important research has been done on bots recently, particularly in terms of thinking about
their definition, scale and influence. The most comprehensive body of research has been
carried out by the Oxford Internet Institute’s Computational Propaganda Research Project. 86
They define high-frequency accounts as those that tweet more than 50 times per day on
average. While often these accounts are bots, we also need to realise there are some humans
who tweet that frequently. There are also cyborg accounts 87, which are jointly operated bypeople and software. As Nic Dias argues, looking at an account’s posting frequency can be
more useful than a fixation on whether an account is fake or not.88
There are certainly highly partisan individuals whose accounts could be mistaken for bots. A
Politico article in August 2017 described how tens of thousands of tweets per day continue toemanate from a very human, grassroots organisation. Using Group Direct Messages on
Twitter, they organize people into “invite-only rooms with names like ‘Patriots United’ and‘Trump Train. Many rooms have accompanying hashtags to track members’ tweets as they
propagate, and each can accommodate as many as 50 people.” 89
Returning to our discussion around agents’ motivations, these examples show the power ofsocial and psychological motivations for creating and disseminating mis- and dis-information.
Being a part of the tribe is a powerful, motivating force.
2) The Messages: What format do they take?
In the previous section we examined the different characteristics of the ‘agents’ those whoare involved in creating, producing or disseminating information disorder. We now turn ourattention to the messages themselves.
There are four characteristics that make a message more appealing and thus more likely to be
consumed, processed and shared widely:1) It provokes an emotional response.
85 Ferrara, E. et al. (2016)
86 Woolley, S & P. Howard, (2017)
87 Chu, Z., S. Gianvecchio, H. Wang, S. Jajodia, (Nov/Dec 2012) Detecting Automation of Twitter Accounts: Are

You a Human, Bot, or Cyborg? IEEE Transactions on Dependable and Secure Computing, Vol. 9, No. 6
88 Dias, N. (June 22, 2017), Reporting on a new age of digital astroturfing, First
Draft,https://firstdraftnews.com/digital-astroturfing/
89 Musgrave, S. (August 9, 2017) I Get Called a Russian Bot 50 Times a Day, Politico,
http://www.politico.com/magazine/story/2017/08/09/twitter-trump-train-maga-echo-chamber-215470

38

2) It has a powerful visual component.
3) It has a strong narrative.
4) It is repeated.
Those who create information campaigns, true or false, understand the power of thisformula. Identifying these characteristics helps us to recognize dis-information campaigns
which are more likely to be successful, and to inform our attempts to counter dis-information(see more in Part Three).
Verbal, text or audio?
While much of the conversation about mis- and dis-information has focused on the role of theinternet in propagating messages, we must not forget that information travels by word of
mouth. The offline and online worlds are not separate, although the challenges researchers
face in effectively studying the effects of different forms of communication simultaneouslymeans it’s easier to think about these elements separately.
The ‘fake news’ conversation has also focussed on text-based, fabricated news websites. As
Nausicaa Renner argues, “the fake news conversation has taken place in the realm of words,
but that’s missing a big part of the story. Much of the content that circulates on Facebook areimages, often memes. They’re not attached to an article, and there’s often no way to trace
their source. And while Facebook’s algorithm is notoriously elusive, it seems to favor images
and video over text. As such, images have the potential to reach more readers than articles —whether fake, real, non-partisan or hyper-partisan.”90
Certainly, in the election-based projects First Draft led in France and the U.K., visuals were
overwhelmingly the most shared and the most difficult to debunk of misleading content. In
both cases, while there were almost no examples of fabricated news sites as we saw in the US
context, there were large numbers of highly shareable images, infographics and memes (i.e.,compelling images with large block text layered over top.) 91

90 Renner, N. (Jan. 30, 2017) Memes trump articles on Breitbart’s Facebook page, Columbia Journalism Review.

https://www.cjr.org/tow_center/memes-trump-articles-on-breitbarts-facebook-page.php
91 Matteo Moschella and Ryan Watts, (June 19, 2017) What we Learned Fact-Checking the UK Election, First
Draft, https://firstdraftnews.com/joint-venture-learnings/

39

Figure 9: An example of a ‘meme’ shared widely during the UK election
As scholarship of visuals92 has shown, the way we understand imagery is fundamentallydifferent to how we understand text. 93 Our brains process images at an incredible speed
when compared with text.94 As a result, our critical reasoning skills are less likely to engage
with what we’re seeing.
Technology that could identify manipulated or fabricated images lags behind technology forparsing and analysing text. While Google’s Reverse Image Search engine (see also TinEye andYandex) is a good starting point for identifying when images have circulated before, we still
don’t have publicly available reverse-video search engines or OCR (Optical Character
Recognition) tools capable of reading the text on memes in a timely manner. We need moresophisticated, widely accessible tools to help identify problematic visual content.
Over the next few years, we will certainly see the development of artificial intelligence
technologies to create as well as identify dis-information. (Simply understood, artificial
intelligence is the ability of computers to undertake tasks that we previously needed humanbrains to work, like speech recognition or visual identification.) It is critical that the engineers
who develop the new products, tools and platforms have been provided ethical training on
the unintended consequences of the algorithms they write.
Upon whom are the messages focused?
92 Sontag S. (1977) On Photography, New York: Farrar Straus and Giroux.
93Postman, N. (1985). Amusing ourselves to death: Public discourse in the age of television. New York: Viking.
94

A team of neuroscientists from MIT has found that the human brain can process entire images that the eye sees
for as little as 13 milliseconds. See: Potter, M. C. (2014). Detecting and remembering briefly presented pictures.
In K. Kveraga & M. Bar (Eds.), Scene Vision (pp. 177-197). Cambridge, MA: MIT Press

40

While agents have particular audiences in mind when they create dis-information, thetargeted subject of the message will be different. Dis-information often deliberately highlights
differences and divisions, whether they be between supporters of different political parties,
nationalities, races, ethnicities, religious groups, socio-economic classes or castes. As
Greenhill argues, these types of messages enable discriminatory and inflammatory ideas toenter public discourse and to be treated as fact. Once embedded, such ideas can in turn be
used to create scapegoats, normalize prejudices, harden us-versus-them mentalities and, in
extreme cases, even catalyze and justify violence.95
Most discussion around dis-information in the US and European contexts has focused onpolitical messages, which, while worrying from a democratic perspective, tend not to inciteviolence. However, in other parts of the world, dis-information directed toward people due totheir religious, ethnic or racial identities has led to violence. As Samantha Stanley explained,“perhaps the most obvious example of how mis-information can lead to violent offline actionis the two-day riots in Myanmar’s second largest city, Mandalay, in July 2014. Following an
unsubstantiated rumor posted on Facebook that a Muslim tea shop owner raped a Buddhist
employee, a mob of almost 500 people wreaked havoc on the city and incited lingering fear
amongst its Muslim citizens. Two people were killed during the riot, one Buddhist and one
Muslim.”963) Interpreters: How do they make sense of the messages?
As Stuart Hall explained in his seminal work on reception theory 97, messages are encoded by
the producer, but then decoded by individual audience members in one of three ways:
1.
2.
3.

Hegemonic. Accepting the message as it was encoded.
Negotiated. Accepting aspects of the message, but not all of it.
Oppositional. Declining the way the message was encoded.

In this section, we outline the work of key cultural and social theorists who have attempted to
explain how audiences make sense of messages.
George Lakoff sees rationality and emotions as being tied together to the extent that, as
human beings, we cannot think without emotions. The emotions in our brains are structured
95 Greenhill, K. M. (forthcoming). Whispers of War, Mongers of Fear: Extra-factual Sources of Threat

Conception and Proliferation and Greenhill, K. M., & Oppenheim B. (forthcoming). Rumor Has It: The Adoptionof Unverified Information in Conflict Zones. International Studies Quarterly.
96 Stanley, S. (May 16, 2017) Mis-information and hate speech in Myanmar, First Draft,
https://firstdraftnews.com/mis-information-myanmar/
97 Hall, S. (1973). Encoding and Decoding in the Television Discourse. Birmingham: Centre for Contemporary
Cultural Studies

41

around certain metaphors, narratives and frames. They help us make sense of things, and,without them, we would become disoriented. We would not know what or how to think.
Lakoff distinguishes two different kinds of reason: ‘False reason’ and ‘real reason.’ 98 False
reason, he says, ‘sees reason as fully conscious, as literal, disembodied, yet somehow fitting
the world directly, and working not via frame-based, metaphorical, narrative and emotional
logic, but via the logic of logicians alone.’ Real reason, alternatively, is an unconscious thoughtthat ‘arises from embodied metaphors.’99 He argues that false reason does not work incontemporary politics, as we’ve become increasingly emotional about our political affiliations.
Understanding how our brains make sense of language is also relevant here. Every word isneurally connected to a particular frame, which is in turn linked together with other frames ina moral system. These ‘moral systems’ are subconscious, automatic and acquired throughrepetition. As the language of conservative morality, for example, is repeated, frames and inturn the conservative system of thought are activated and strengthened unconsciously andautomatically. Thus, conservative media and Republican messaging work unconsciously toactivate and reinforce the conservative moral system, making it harder for fact-checks topenetrate.100
Considering Trump’s success, D’Ancona recently argued, “He communicated a brutal empathy
to [his supporters], rooted not in statistics, empiricism or meticulously acquired information,
but an uninhibited talent for rage, impatience and the attribution of blame.” 101 Ultimately,news consumers “face a tradeoff: they have a private incentive to consume precise andunbiased news, but they also receive psychological utility from confirmatory news.” 102
As we will discuss in Part Two, the emotional allure of situating ourselves within our filter
bubbles and having our worldviews supported and reinforced by ‘confirmatory news’ isincredibly powerful. Finding solutions to this is going to require a mixture of technological andeducational solutions and, ultimately, a psychological shift whereby one-sided media diets are
deemed socially unacceptable.

98 Lakoff, George (1997). Metaphors We Live By. University of Chicago Press and Moral Politics: What

Conservatives Know That Liberals Don't. University of Chicago Press
99 Lakoff, G. (2010) “Why "Rational Reason" Doesn't Work in Contemporary Politics”, http://www.truthout.org/buzzflash/commentary/george-lakoff-why-rational-reason-doesnt-work-in-contemporary-politics/8893george-lakoff-why-rational-reason-doesnt-work-in-contemporary-politics
100 Lakoff, G. (2010)
101 D’Ancona, M. (2017) Post-Truth, Ebury Press.
102 Allcott, H. and M. Gentzkow, (2017) Social Media and Fake News in the 2016 Election, Journal of Economic
Perspectives, 31:2, p.218

42

Communication as Ritual
When town criers announced news to crowds, runners read newspapers aloud incoffeehouses and families listened to or watched the evening news together, news
consumption was largely a collective experience. However, news consumption has slowly
evolved into an individual behavior with the emergence of portable radios and television and,more recently, the ubiquitous adoption of laptops, tablets and smartphones.
But while we might physically consume the news alone, what we choose to consume isincreasingly visible because of social media. The posts that we like or comment on and thearticles, videos or podcast episodes we share are all public. Borrowing from Erving Goffman’s
metaphor of life as theatre, invariably, when we use social media to share news, we become
performers.103 Whatever we like or share is often visible to our network of friends, family andacquaintances, and it affects their perceptions of us.104
If social media is a stage, our behaviour is a performance and our circle of friends or followers
are our audience. Goffman thinks our goal for this performance is to manage our audience’s
perception of us.105 Therefore, we tend to like or share things on social media that our friends
or followers would expect us to like or share—or, in other words, what we would normallylike or share.106
Similarly, as Maffesoli argued in his 1996 book The Time of the Tribes107, to understandsomeone’s behavior, one must consider the sociological implications of the many different,
small and temporary groups that he or she is a member of at any given time of day.
Maffesoli’s writings aptly describe the realities of users who have to navigate different online
groups throughout the day, deciding what information to post or share to different ‘tribes’
online and off.
This tribal mentality partly explains why many social media users distribute dis-information
when they don’t necessarily trust the veracity of the information they are sharing: they would
like to conform and belong to a group, and they ‘perform’ accordingly. 108 The pressure toconform can become particularly strong when algorithms on social platforms suppress views

103 Goffman, E, (1956) The Presentation of Self in Everyday Life. Random House.
104

Karlova, N. A., & Fisher, K. E. (2013). Plz RT: a social diffusion model of mis-information and disinformation for understanding human information behaviour. Information Research, 18(1), 1-17.
105 Goffman defines impression management as a conscious or unconscious process in which people try to
influence the perceptions of other people about a person, object or event by regulating and controlling informationin our daily social interaction.
106 Picone, I. (2015) Impression Management in Social Media, Published Online: 11 FEB 2015
http://onlinelibrary.wiley.com/doi/10.1002/9781118767771.wbiedcs071/abstract
107 Maffesoli, M. (1996) The time of the tribes, London:Sage.
108 Social platforms that do not allow anonymity are more prone to this twisted aspect of impression management,
whereas on platforms which permit anonymity, other problems such as trolling and harassment can arise.

43

opposing those of the user. Even if a user has a politically diverse circle of friends or followers,
what she sees in her newsfeed or timeline does not necessarily reflect that diversity.
This connects with the theory of motivated cognition, which refers to the unconscious
tendency of individuals to process information to fit conclusions that suit some internal goal.
The classic example comes from the 1950s, when psychologists asked students from two Ivy
League colleges to watch a film of a football game between their schools that featured a setof controversial officiating calls. The students from each school were more likely to see thereferees’ calls as correct when it favored their school than when it favored their rival. The
researchers concluded that the emotional stake the students had in affirming their loyalty totheir respective institutions shaped what they saw on the tape.109
Yale University’s Dan Kahan and colleagues demonstrated motivated cognition in a politicalcontext. They found that, on issues such as a gun control or climate change, participants
would do mathematical somersaults with available data to ‘prove’ the point of viewsupported by their own politics. 110 Kahan argues that while it’s tempting to fixate on the ‘lazy
brain’ theory – that humans rely heavily on mental shortcuts to compensate for the vast
amount of information they encounter every day – humans are instead making decisions
about what position is most appropriate to publicly support. He concludes: “Work on
motivated cognition and political conflict tends to focus more on the need for maintaining avalued identity, particularly as a member of a group… But the seeming inability of economic
interests to explain who believes what on issues such as climate change, the HPV vaccine,
economic policies that include tax cuts or social welfare spending and the like is in fact the
motivation for examining the contribution that identity-protective forms of motivatedcognition make.”111
Communication as Transmission
In a study by Van Dammes and Smets in 2014, they remind us that the “human memory is nota recording device, but rather a process of (re)construction that is vulnerable to both internaland external influences.”112
The challenge for the human brain today is how these influences work in the context of social
networks that are bombarding us with information, pinging us repeatedly via thesmartphones in our pockets. As WikiMedia testified to the UK Parliament, “Our minds have
109 Kahan, D. (2011) What is Motivated Reasoning and How Does It Work? Science and Religion Today,

http://www.scienceandreligiontoday.com/2011/05/04/what-is-motivated-reasoning-and-how-does-it-work/
110 Kahan, D. et al (2013) Motivated Numeracy and Enlightened Self-Government, Behavioural Public Policy, 1,
54-86
111 Kahan, D. (2011) What is Motivated Reasoning and How Does It Work? Science and Religion Today,
http://www.scienceandreligiontoday.com/2011/05/04/what-is-motivated-reasoning-and-how-does-it-work/
112 Van Damme, I. & K. Smets, 2014, p. 310. The power of emotion versus the power of suggestion: Memory foremotional events in the mis-information paradigm. Emotion. 14 (2): 310

44

always been a battleground for various social forces, but the sheer number of agents andinstitutions vying for control of our thoughts and feelings today is so large that it is confusing
and destabilising for many.”113
Filippo Menczer’s114 most recent research highlights the challenges of our brains to makedecisions about credibility when the streams of information are overwhelming. In otherwords, normal people are too distracted by a deluge of information to find the most accurate
stories: “There are a hundred more stories you’re not seeing that are much better than those
five that you thought were good.” So, according to this research, irrespective of echochambers and confirmation bias, people are not sharing verified stories in part because they
never see them.
According to research115 116 conducted before the heavy use of social media that we take for
granted today, people used a set of key heuristics, or mental shortcuts, when evaluating thecredibility of a source or message:
1.
2.
3.
4.

Reputation. Based on recognition and familiarity
Endorsement. Whether others find it credible
Consistency. Whether the message is echoed by multiple sites
Expectancy violation. Whether a website looks and behaves in the expectedmanner
5. Self-confirmation. Whether a message confirms one’s beliefs
6. Persuasive intent. The intent of the source in creating the message

When we consider these heuristics in the context of our heavy reliance on social media as asource of information, the issues we see in this current age of mis- and dis-information
become less surprising.
A very recent meta-analysis117 of the psychological efficacy of messages countering misinformation provides an excellent overview of the research literature pertaining to debunks
and how they impact people’s perceptions of mis-information. The review of the literature
113 Evidence Provided to the UK Parliamentary Inquiry on Fake News by WIkiMedia UK,

http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/culture-media-and-sportcommittee/fake-news/written/48122.html
114 Qiu, X. et al. (2017) Limited individual attention and online virality of low-quality information, Nature
Human Behaviour, Vol 1.
115 Metzger, M. and A. J. Flanagin (2013) Credibility and trust of information in online environments: The use ofcognitive heuristics, Journal of Pragmatics, 59 pp. 210-220
116 Lewandowsky, S. et al. (2012) Mis-information and Its Correction: Continued Influence and Successful
Debiasing, Psychological Science in the Public Interest, 13(3), pp. 106–131
117 Chan, M.S., C. R.Jones, K.H. Jamieson, D. Albarracín (2017) Debunking: A Meta-Analysis of thePsychological Efficacy of Messages Countering Mis-information, Psychological Science, 1-16.

45

underlined that the effects of a debunking effect were weaker when audiences generated
reasons in support of the initial mis-information, supporting what we know about the powerof confirmation bias and motivated reasoning.
People are not incentivised to click out of social media to view an article in its original form.
As such, the cue of ‘expectancy violation’ (whether the site behaves as expected) and‘consistency’ (whether the information is supported by multiple sites) are unlikely to be
utilized.
A most troubling finding from social media studies is how powerful ‘familiarity’ is as apersuasive factor.118 As Paul and Matthews discuss in their 2016 paper on the methods bywhich Russia effectively creates a ‘firehose of falsehood’, repetition is one of the mosteffective techniques for getting people to accept mal- and dis-information.
The repetition component is particularly problematic on social media due to people trying tomanipulate the platforms through bots that automatically “like” or “share” stories or ‘clickfarms’. These techniques can create false sense of popularity about content, and, by tagging
influential people like celebrities, politicians or even journalists, impact the news cycle. A
disturbing recent report by Trend Micro119 outlines the varied ways that influence is beingbought, and the ways in which click farms are being used to boost hashtags, game online
petitions, skew online comment and create fake accounts.
Cues like ‘endorsement’ also become more salient on social media. Our ability to immediately
see whether friends and family have liked, shared, commented or retweeted a piece ofcontent becomes a powerful influence on our credibility judgments. As researchers haveshown 120, if you find out your friends like a song, you’ll be more likely to like it too. Humanbeings are drawn to follow the masses, particularly when the mass is shown to include yourclosest friends and family. As Jonathan Stray explains, “messages received in greater volume
and from more sources will be more persuasive. Quantity does indeed have a quality all its
own… [R]eceiving a message via multiple modes and from multiple sources increases themessage’s perceived credibility, especially if a disseminating source is one with which an
audience member identifies.”121

118

Pennycook, G. et al (July 5, 2017) Prior Exposure Increases Perceived Accuracy of Fake News, Available at
SSRN: https://ssrn.com/abstract=2958246
119 Gu, L., V. Kropotov & F. Yarochkin, (June 2017), How Propagandists Abuse the Internet and Manipulate the
Public. Trend Micro, https://documents.trendmicro.com/assets/white_papers/wp-fake-news-machine-howpropagandists-abuse-the-internet.pdf
120 Salganik, M. et al. (2006) Experimental Study of Inequality and Unpredictability in an Artificial CulturalMarket, Science, Vol. 311, pp.854-856
121
Stray, J. (Feb 27, 2017), Defense Against the Dark Arts: Networked Propaganda and Counter-Propaganda,
Tow Center for Digital Journalism, Medium. https://medium.com/tow-center/defense-against-the-dark-artsnetworked-propaganda-and-counter-propaganda-deb7145aa76a

46

The heuristic of self-confirmation now is also especially powerful, now that social networks
are the dominant form of information dissemination. Back in 2006, research by Taber andLodge122 showed the powerful effect of prior attitudes upon reasoning. Attitudinallycongruent arguments are evaluated as stronger than attitudinally incongruent arguments.
The algorithmic filtering that makes us much less likely to come across information thatchallenges us (see the section below on filter bubbles and echo chambers) means that the
selective exposure that humans tend toward (as it requires less cognitive ‘work’) is done forus automatically.
In addition to self-confirmation bias, humans are also affected by motivated reasoning and adesire to be vindicated. As Sunstein et al.123 found, people who believed in man-made climatechange updated their beliefs more in response to bad news (e.g. temperatures are going up
more than expected), whereas those who disbelieved man-made climate change were more
responsive to good news. Therefore, beliefs were only changed in ways that cemented what
they already thought to be true.
This is linked to recent research trying to replicate the so-called backfire effect, which was
first proposed in 2010124 to account for fact-checks that appeared to harden people’s beliefs
about false information. The researchers were unable to replicate the backfire effect andfound that corrections and fact-checks do nudge people toward the truth.
Specifically, their research found that, while Trump supporters were more resistant tonudging, they were nudged all the same. And there was another clear pattern of Trump
supporters: corrections didn’t change participants’ feelings about Trump. As one of theresearchers, Brendan Nyhan, explained, “People were willing to say Trump was wrong, but it
didn’t have much of an effect on what they felt about him.” 125 The takeaway is that, while
facts make an impression, they just don’t matter for our decision-making—a conclusion thathas a great deal of support in the psychological sciences.126

122 Taber, C. and M. Lodge, (2006) Motivated Skepticism in the Evaluation of Political Beliefs, American

Journal of Political Science, Vol. 50, No. 3 (Jul., 2006), pp. 755-769
123 Sunstein, C. R., et al. (2016). How People Update Beliefs about Climate Change: Good News and Bad News(SSRN Scholarly Paper No. ID 2821919). Rochester, NY: Social Science Research Network.
124 Nyhan, Brendan, and Jason Reifler. 2010. “When Corrections Fail: The persistence of politicalmisperceptions.” Political Behavior 32 (2): 303–330.
125 Resnick, B. (July 10, 2017) “Trump supporters know Trump lies. They just don’t care.” Vox,
https://www.vox.com/2017/7/10/15928438/fact-checks-political-psychology
126
See this analysis of the 7 most important psychological principles which explain how people make sense ofpolitics today, also by Brian Resnick https://www.vox.com/science-and-health/2017/3/20/14915076/7psychological-concepts-explain-trump-politics

47

The research on how best to word and visualise fact-checks and debunks is varied and attimes contradictory.127 Much of this research is US-focused, concerned with political factchecks and mostly carried out on American undergraduate students. It’s vital that morestudies are replicated in different geographical settings, using mis-information in otherareas—particularly health and science.
There is currently a great deal of discussion about increasing funding for individual news
literacy programs, as well as integrating core elements into national curricula. We would
argue those programs and curricula should include discussions of how to override the humantendency to seek out information that supports our worldview and ‘tribal identifications’,
how to beat confirmation bias and how to be skeptical of information which produces anemotional response.
In this first section, we introduced new conceptual frameworks for discussing and researching
information disorder, outlining the three types, elements and phases of information disorder:
i)
ii)
iii)

The three types: mis-, dis- and mal-information
The three elements: agents, messages and interpreters
The three phases: creation, production and dissemination

We need to be much more precise about the definitions we use to describe the phenomenonof information disorder, if we are to begin understanding how and why it is created, the forms
that it takes, and its impact. We also need to understand how characteristics change as
information flows through the different phases, and how the person who interprets aparticular message can become an agent in their own right as they go on to re-share thatmessage with their own networks. In the following section, we discuss the challenges of filter
bubbles and echo chambers, underlining the importance of considering how people discover
information and share it with their own networks, and the need to study the wider
implications for public discourse.

127

An excellent overview of some of the literature can be found here: Flynn, D.J. & Nyhan, Brendan & Reifler,
Jason. (2017). ‘The Nature and Origins of Misperceptions: Understanding False and Unsupported Beliefs About
Politics’, Political Psychology. 38: 127-150

48

Part 2: Challenges of filter bubbles and echo chambers‘The “public sphere”: the shared spaces – real, virtual or imagined – whereby social issues are
discussed and public opinion is formed’. This theory was first shared by the Germansociologist and philosopher Jurgen Habermas, who argued that a healthy public sphere isessential for democracy and must be inclusive, representative and characterised by respect
for rational argument.128 The most significant challenge to any theory of a shared publicsphere is that humans, when we have a choice about who to connect with or not, tend toestablish and continue relationships with people who have views similar to our own. We areprogrammed to enjoy spending time in ‘echo chambers,’ as it requires less cognitive work.
There is no doubt that digital technologies support us in these tendencies. In his 1998 essay,
Which Technology and Which Democracy?, Benjamin Barber wrote, “Digitalization is, quite
literally, a divisive, even polarizing, epistemological strategy… It creates knowledge niches forniche markets and customizes data in ways that can be useful to individuals but does little forcommon ground… [I]t obstructs the quest for common ground necessary to representative
democracy and indispensable to strong democracy.” 129 At the same time, MIT Media Lab
founder Nicholas Negroponte started a discussion about what this very human set of
behaviours would look like online. In ‘The Daily Me’, a thought experiment, he considered theimplications of completely personalised newspapers. And, in 2006, Habermas acknowledged
the challenge for the public sphere in the era of the internet. He argued, “[T]he rise of millionsof fragmented chat rooms across the world has tended instead to lead to the fragmentationof large but politically focused mass audiences into a huge number of isolated issue
publics.”130
These ideas moved into the mainstream in 2011 with the publication of Eli Pariser’s book, theFilter Bubble131. By that point, Negroponte’s thought experiment had become a reality with
Facebook’s Newsfeed, and Pariser was able to explain how the social technology companies
have engineered personalised experiences. Using algorithms to deliver content that we aremostly likely to enjoy, these platforms reinforce our worldviews and allow us to stay encasedin our safe, comfortable echo chambers.
It is worth recounting here James Carey’s description of the ‘ritual view of communication’,128 Habermas, J. (1962) The Structural Transformation of the Public Sphere: An Inquiry into a Category of

Bourgeois Society
129 Barber, B. (1998) Which Technology and Which Democracy?, Talk given at Democracy and Digital Media
Conference, http://web.mit.edu/m-i-t/articles/barber.html
130 Habermas, J. (2006) Speech to the International Communication Association.
131 Pariser, E. (2011) The Filter Bubble: How the New Personalized Web Is Changing What We Read and How
We Think, Penguin Books.

49

which is not about “the act of imparting information but the representation of sharedbeliefs.” Appreciating this truth helps us explain why echo chambers are so appealing. They
provide safe spaces for sharing beliefs and worldviews with others, with little fear of
confrontation or division. They allow us to ‘perform’ our identities as shaped by ourworldviews with others who share those worldviews. This behaviour is not new, but theplatforms have capitalized on these human tendencies, knowing they would encourage users
to spend more time on their sites.
Agents who are creating dis-information understand that, when people consume and sharethese messages, they will be doing so increasingly from inside these echo chambers, with noone to challenge the ideas. This means the people who will interpret their messages are muchless likely to have an ‘oppositional’ (rejecting the way the message was encoded) or‘negotiated’ (accepting only some aspects of the message) reading. As such, agents target
groups that they know are more likely to be receptive to the message. If they are successful indoing that, it is very likely the message will then be shared by the initial recipient. And, as
research shows, we are much more likely to trust a message coming from someone we
know132, even if we suspect it to be false. This is why dis-information can be disseminated soquickly. It is travelling between peer-to-peer networks where trust tends to be high.133
The fundamental problem is that “filter bubbles” worsen polarization by allowing us to live in
our own online echo chambers and leaving us with only opinions that validate, rather thanchallenge, our own ideas. While confirmation bias occurs offline and the term ‘selective
exposure’ has been used by social scientists for decades to describe how information seekers
use only certain sources that share their views,134 social media are designed to takeadvantage of this innate bias.
The rise in popularity of social networks as sources of news has taken place at the same time
as a decline in local newspapers in some of the largest democracies in the world. In the U.S.,
Canada and the U.K. particularly, the local news ecosystem is struggling as the advertisingmodel for news has collapsed. Many local newsrooms have been forced to make serious staff
cuts, consolidate or close. In the U.K., there has been a net loss of approximately two hundred
local newspaper titles since 2005135 In Canada a study commissioned by Friends of Canadian
Broadcasting warned the Canadian Radio-television and Telecommunications Commission
132 Metzger et al. (2010) Social and Heuristic Approaches to Credibility Evaluation Online, Journal ofCommunication, 60 (3):413-439
133 Granovetter, M.S. (1973) The Strength of Weak Ties, Journal of Sociology, 78(6):1360-1380
134 Prior, M. (2003). Any good news in soft news? The impact of soft news preference on political knowledge.
Political Communication, 20(2), pp.149-171.
135
Pondsford, D. (March, 31, 2017) The decline of local journalism is a far greater threat to media plurality thanRupert Murdoch, The Press Gazette, http://www.pressgazette.co.uk/the-decline-of-local-journalism-is-a-fargreater-threat-to-media-plurality-than-rupert-murdoch/50

that “without intervention, half of Canada’s small and medium-market television stations
could disappear by 2020.”136 And in a Columbia Journalism Review article the scale of US
News deserts was spelled out very clearly, showing how many cities have been left with justone local newspaper, and how many have none at all.137 As ad revenue moves to Google andFacebook (in the mobile ad market these two companies earn early half of all revenue138), it
is expected that the same patterns witnessed in these countries will be felt in many othercountries in the next few years.
In 2009, the U.S.-based Knight Commission on the Information Needs of Communities in a
Democracy concluded that information is “as vital to the healthy functioning of communities
as clean air, safe streets, good schools, and public health.”139 While there no evidence yet to
directly connect the decline of local news media to the rise in information pollution, whenstrong local media do not exist other sources will fill that vacuum.
As Nina Jankowicz, a fellow at the Woodrow Wilson Center’s Kennan Institute wrote in an
opinion piece in the New York Times recently:
Without news that connects people to their town councils or county fair, or
stories that analyze how federal policies affect local businesses, people are left
with news about big banks in New York and dirty politics in Washington….
Readers compare this coverage with their dwindling bank balances andcrumbling infrastructure and feel disconnected and disenfranchised, and latch
onto something — anything — that speaks to them. That might be President
Trump’s tweets. Or dubious “news” from an extreme right- or left-wing site
might ring true. Or they might turn to Russian disinformation, which exploits
this trust gap.140
Local news outlets provide a shared experience for a community. When communities rely on
individual feeds of news from their social networks, these shared experiences disappear. We
need more research to understand the implications of this in terms of people sharing mis- anddis-information.
136 Lindgren, A., J. Corbett & J. Hodson, (Jan. 23, 2017) Canada’s Local News Poverty, Policy Options,

http://policyoptions.irpp.org/magazines/january-2017/canadas-local-news-poverty/
137
Bucay, Y., V. Elliott, J. Kamin, A. Park, America’s Growing News Deserts, Columbia Journalism Review,https://www.cjr.org/local_news/american-news-deserts-donuts-local.php
138 Thompson, D. (Nov. 3, 2016) The Print Apocalypse and How to Survive It, The Atlantic,
https://www.theatlantic.com/business/archive/2016/11/the-print-apocalypse-and-how-to-survive-it/506429/
139 Knight Commission on the Information Needs of Communities in a Democracy, (2009) Informing
Communities: Sustaining Democracy in the Digital Age, The Aspen Institute,
https://production.aspeninstitute.org/publications/informing-communities-sustaining-democracy-digital-age/
140
Jankowicz, N (Sept. 25, 2017) The Only Way to Defend Against Russia’s Information War, The New YorkTimes, https://mobile.nytimes.com/2017/09/25/opinion/the-only-way-to-defend-against-russias-informationwar.html

51

As discussed, the technology companies are commercial entities, and therefore to keep theirshareholders happy need to encourage users to stay on their site for as long as possible tomaximize the number of exposures to advertisements. They do so by tweaking the algorithms
to deliver more of what users have liked, shared or commented on in the past. So, while wehave seen the technology companies take some steps to fight dis-information on theirplatforms (see Part 3), ultimately, it’s difficult to imagine them making substantive changes totheir algorithms to pop these filter bubbles. If the platforms changed the algorithm to provide
us with more challenging material that forced us to reconsider some of our established world
views, we are unlikely to spend as much time on them.
As Wired dramatically concluded in an article just after the US election, “The global villagethat was once the internet has been replaced by digital islands of isolation that are drifting
further apart each day. From your Facebook feed to your Google Search, as your experience
online grows increasingly personalized, the internet’s islands grow farther apart.”141 The
Laboratory for Social Machines at MIT has been investigating filter bubbles, how they form,and how people can try to break out of them. In research published in December 2016,
tweets sent during the US election were visualized in a network that showed almost no
overlap between Trump and Clinton supporters.142 The analysis concluded that, on Twitter,
Trump supporters formed a particularly insular group when talking about politics during thegeneral election, having few connections to Clinton supporters or the mainstream media.

Figure 10: Visualization by the Laboratory for Social Machines at MIT of Donald Trump and Hillary
Clinton supporters on Twitter
141 El-Bermawy, M. (Nov. 18, 2016) Your Filter Bubble is Destroying Democracy, Wired.

https://www.wired.com/2016/11/filter-bubble-destroying-democracy/
142
Thompson, A. (Dec. 8, 2016) Parallel Narratives, Clinton and Trump Supporters Really Don’t Listen to One
Another, Vice. https://news.vice.com/story/journalists-and-trump-voters-live-in-separate-online-bubbles-mitanalysis-shows52Research by Demos in the U.K. that analysed British Twitter accounts found similar patterns
between supporters of different political parties. However, they were able to show thatpeople with more extreme political views tended to engage with a smaller number of people
than those who had more moderate political views.143
Walter Quattrociocchi and his team have studied the dynamics of echo chambers on
Facebook.144 Examining the posts of 1.2 million users, his team analysed mainstream andconspiracy science news and how they are consumed and shaped by communities on
Facebook. They found that polarized communities emerge around distinct types of content,
and consumers of conspiracy news tended to be extremely focused on specific topics likeclimate change.
There has also been research that challenges these ideas about the dangers of echochambers. A survey of 14,000 people in in seven countries that was published in May 2017 145
concluded that “people who are interested and involved in politics online are more likely to
double-check questionable information they find on the internet and social media, including
by searching online for additional sources in ways that will pop filter bubbles and break out ofecho chambers.”146 In addition, the 2017 Digital News Report, published by the Reuters
Institute for the Study of Journalism, concluded, “Echo chambers and filter bubbles areundoubtedly real for some, but we also find that – on average – users of social media,
aggregators and search engines experience more diversity than non-users.”147
Concerns raised since Brexit and the US election have led to new innovations by social
platforms, third-party organizations and academic institutions to help people ‘prick’ theirfilter bubbles. With the renewed emphasis on scaling news literacy programs globally,teaching how social algorithms produce these filter bubbles should be a crucial part of any
standardised curriculum.
Facebook, for example, rolled out a new related-articles feature148 that is designed to show
multiple perspectives on a story. And, during the U.K. and French election, Facebook rolled

143 Krasodomski-Jones, A. (2016) Talking To Ourselves? Political Debate Online and the Echo Chamber Effect.

Demos. Available at: https://www.demos.co.uk/project/talking-to-ourselves/
144 Quattrociocchi, W. (14 Jan, 2016) How does mis-information spread online? World Economic Forum Blog,
https://www.weforum.org/agenda/2016/01/q-a-walter-quattrociocchi-digital-wildfires/
145 Dutton, William H., et al. (2017) "Search and Politics: A Cross-National Survey." Quello Center Working
Paper No. 2944191 Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2944191
146 Dutton, B. (May 2017) Fake news, echo chambers and filter bubbles are an exaggerated threat. Here's why.
World Economic Forum Blog. https://www.weforum.org/agenda/2017/05/fake-news-echo-chambers-and-filterbubbles-are-an-exaggerated-threat-heres-why
147 Newman, N. (2017) Overview and Key Findings: Digital News Report, Reuters Institute for the Study ofJournalism. http://www.digitalnewsreport.org/survey/2017/overview-key-findings-2017/
148 Constine, J. (2017) Facebook shows Related Articles and fact checkers before you open links, TechCrunch,https://techcrunch.com/2017/04/25/facebook-shows-related-articles-and-fact-checkers-before-you-open-links/

53

out the Perspectives149 feature, which allowed users to compare candidates or parties
positions after clicking on an election-related article.
A third-party effort to prick the filter bubble is PolitEcho150, a Chrome extension that allows
you to plot your Facebook friends on a graph based on their political affiliation, which isestimated from the news organizations they have liked. Another is Flipfeed 151, which allows
you to randomly see the Twitter feed of someone with a diametrically oppositional view toyour own. As the app claims, watching the ‘Trump news conference’, in the ‘flipped’ modecan be eye-opening. And yet another Chrome extension, Rbutr152, is a community-driven app
which connects webpages on the basis that one page argues against the other. If you havedownloaded the extension and you visit a ‘rebutted’ page, you will be told there are rebuttals
to the particular page and it will link out to the rebutting articles.

Figure 11: Screenshot of the Blue Feed/Red Feed streams on the topic of immigration(http://graphics.wsj.com/blue-feed-red-feed/#/immigration)
149 McGregor, J. (May 31, 2017) Facebook Wades Into Another Election, Forbes.

https://www.forbes.com/sites/jaymcgregor/2017/05/31/facebook-wades-into-another-election/#37265336635c
150 http://politecho.org/
151 https://flipfeed.media.mit.edu/
152 http://rbutr.com/#

54

The Wall Street Journal was one of the first news outlets to try to find a way for theiraudiences and other people interested in this concept to compare the way different topics—
like Trump, guns, health care, and immigration—were being covered and shared by people
from different political perspectives. They created ‘Blue Feed, Red Feed’ 153 “to demonstrate
how reality may differ for different Facebook users.” If a source appears in the red feed, amajority of the articles shared from the source were classified as ‘very conservatively aligned’in a large 2015 Facebook study. For the blue feed, a majority of each source’s articles aligned‘very liberal.’ These feeds aren’t intended to resemble actual individual news feeds. Instead,they are rare, side-by-side looks at real conversations from different perspectives.
Buzzfeed and the Guardian newspaper have been testing new features that help readersnavigate alternative viewpoints. BuzzFeed’s ‘Outside Your Bubble’ 154 pulls in opinions fromacross the web and gives them a neutral platform. Public comments, which can be the mostemotional and combative part of any online story, are then removed from their normal
context and rephrased as dispassionate bullet points.
The Guardian’s weekly column ‘Burst Your Bubble’ 155 curates “five conservative articles worth
reading" for the site’s liberal audience. Similarly, every week, the Washington Post journalist
Will Sommer publishes a newsletter, ‘Right Richter’,156 which aggregates right-wing
perspectives for left-leaning audiences.
Finally, there is the AllSides site157, whose stated mission is to expose bias and provide“multiple angles on the same story so you can quickly get the full picture, not just one slant.”
Using a mixture of crowd-driven ratings and its patented algorithms, it’s the newest attempt
to provide audiences with a visual guide to politically slanted journalism.

153 http://graphics.wsj.com/blue-feed-red-feed/
154https://www.buzzfeed.com/bensmith/helping-you-see-outside-your-bubble?utm_term=.lhpjZ5lx2#.yqm1R3wE5
155 https://www.theguardian.com/us-news/series/burst-your-bubble
156 http://tinyletter.com/rightrichter
157 http://allsides.com/

55

Figure 12: Screenshot of the Allsides.com website

The ultimate challenge of filter bubbles is re-training our brains to seek out alternative
viewpoints. Some people have likened our informational diet to our nutritional one, claimingthat in the same way as we had to be educated to see the value of a diet rich in fruit andvegetables, we need to give ‘nutritional ‘labels to information so that people understand thevalue of a media diet with a variety of political viewpoints. While we can pressure the social
networks to diversify our diet, we can’t force people to actually click, let alone read thecontent. At public events, Facebook has admitted that when they have attempted to deliver
more content from an opposite view, people tend not to click on it.
As this report underlines, we have to think about information consumption from a ritual aswell as transmission perspective. If we recognise that people seek out and consume content
for many reasons beyond simply becoming informed – like feeling connected to similarpeople or affiliating with a specific identify – it means that pricking the filter bubbler requires
more than simply providing diverse information.

56

Part 3: Attempts at solutions
One week after the election, Eli Pariser, author of The Filter Bubble, created a public Google
document and asked people to contribute solutions to solve the mis-information problem.
Within a few weeks, the document was more than 150 pages long and included commentsfrom over 50 people. The document 158 includes many ideas and can be seen as acomprehensive blueprint of what solutions are possible.
One point we’d like to stress, however, is that much of the debate about solutions
presupposes communication as information transmission. But this cannot explain or solve theproblem of information disorder. As Carey suggests, “under a ritual view [of communication]news is not information but drama” 159 and “a portrayal of the contending forces in theworld.”160 Our conversations about solutions will need to evolve in order to recognise thisrole information plays beyond simply transmitting messages.
Over the past twelve months, potential solutions have been discussed endlessly atconferences and in workshops, but we’ve seen little concrete changes from the platforms.
While there is certainly more foundation money than there was,161 and a myriad of small
projects are underway, the grand ideas are yet to be implemented. These include Apple’s CEO
Tim Cook call for a Public Service Announcement about dis-information, new labels to identify
different types of content on social platforms, systematic programs for taking down bot
accounts, the integration of critical media literacy programs in schools and best practices formaking fact-checks and debunks shareable.
Facebook and Google have announced methods for preventing fabricated sites from making
money through their advertising platforms. However, anecdotally, ‘fake news’ creators have
explained that, while they experienced short term losses in revenue earlier in the year, they
have returned their profits to previous levels using other ad networks that are willing topartner with them.
The only real major development we’ve seen has been the passing of legislation in Germany
that fines platforms for hosting unlawful content, including defamation and incitement to

158 Pariser, E. (2016) Media ReDesign: The New Realities,

https://docs.google.com/document/d/1OPghC4ra6QLhaHhW8QvPJRMKGEXT7KaZtG_7s5UQrw/edit#heading=h.l4uvrs8m75xh
159 Carey, J. (1989), p.17
160 Carey, J. (1989), p.16
161 Yurieff, K. (April 5, 2017) eBay founder commits $100 million to combat 'fake news', CNN,
http://money.cnn.com/2017/04/05/technology/pierre-omidyar-donation-fake-news/index.html

57

hatred, and not removing such posts within 24 hours.162 And we’re seeing serious discussions
in Singapore about passing a similar law.163
In this next section, we discuss potential solutions from a number of different perspectives—technological, social, media-centric, educational and regulatory.
Technological Approaches
Mis-, dis- and mal-information are incredibly complex phenomena, but the impression thatthese problems emerged suddenly during the US election encouraged many to believe that asolution could be found just as quickly. While changing underlying socio-economic andcultural factors takes time, the allure of an easy algorithmic tweak meant it became thepopular solution. Certainly, in the crowd-sourced ‘Design Solutions’ Google document started
by Eli Pariser, discussions about technological solutions dominate. 164 Even Krishna Bharat, theengineer responsible for building Google News, stepped in and wrote a detailed piece about
the technical ways in which the platforms could detect mis- and dis-information in realtime.165
What have the social networks done?
As we have seen, one of the primary motivations for creating dis-information is financial gain.
Google has therefore worked to prevent revenue flowing to the owners of “bad sites, scams
and ads” and has permanently banned nearly 200 publishers from its AdSense advertising
network as of late 2016. 166 Facebook made similar moves, updating its policies with language
stating they would not display ads that show misleading or illegal content. Facebook has also
taken steps to tackle ‘imposter content’, stating, “On the buying side, we’ve taken action
against the ability to spoof domains, which will reduce the prevalence of sites that pretend to
be real publications.”167 And, in late August 2017, Facebook announced they would block ads
from pages that had repeatedly shared false news, stating “Currently, we do not allow
advertisers to run ads that link to stories that have been marked false by third-party fact162 Eddy, M. and M. Scott (June 30 2017), Delete Hate Speech or Pay Up, Germany Tells Social Media

Companies, New York Times, https://www.nytimes.com/2017/06/30/business/germany-facebook-googletwitter.html
163 Yi, S.B. (June 19, 2017) New legislation to combat fake news likely to be introduced next year: Shanmugam,
Straits Times, http://www.straitstimes.com/singapore/new-legislation-to-combat-fake-news-next-year-shanmugam
164 Eli Pariser (document creator) (2016) Media ReDesign: The New Realities,
https://docs.google.com/document/d/1OPghC4ra6QLhaHhW8QvPJRMKGEXT7KaZtG_7s5-UQrw/edit
165 Bharat, K. (April27, 2017) How to Detect Fake News in Real-Time, NewsCo on Medium,https://shift.newco.co/how-to-detect-fake-news-in-real-time-9fdae0197bfd
166 Spencer, S. (Jan. 25, 2017) How we fought bad ads, sites and scammers in 2016, Google Blog,
https://www.blog.google/topics/ads/how-we-fought-bad-ads-sites-and-scammers-2016
167
Written evidence by Facebook to the UK Parliamentary Inquiry on Fake News,
http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/culture-media-and-sportcommittee/fake-news/written/49394.html

58checking organizations. Now we are taking an additional step. If Pages repeatedly sharestories marked as false, these repeat offenders will no longer be allowed to advertise on
Facebook.”168
Google News recently took steps to allow publishers to highlight fact-checked content for
programmatic detection using schema.org, a structured data markup schema supported by
the major search engines. This feature first appeared in the U.K. and the US last October, andhas since been added to Google News in Germany, France, Brazil, Mexico and Argentina.
Google News Lab, which is distinct from Google News and whose mission is to collaborate
with journalists and entrepreneurs to help build the future of media, has been very active inthis space over the past couple of years.169 For example, the News Lab was a founding partnerof First Draft when it began in June 2015.
Facebook moved, from Mark Zuckerberg’s post-election denial170 that ‘fake news’ was a
problem on his platform, to rolling out a third-party fact-checking initiative on December 15,
2016 that includes the International Fact Checking Network, The Associated Press, The
Washington Post and Snopes.171 They expanded the project to France and Germany inFebruary and the Netherlands in March. In this initiative, users flag posts they think might be‘false news’, populating a queue that the affiliated fact-checking organizations can see. Afteran article has been fact-checked, any user who sees that content will see that it has beendisputed by one of the fact-checking organizations. If someone tries to share a disputed
article, they are reminded with a pop-up notice that the content is in dispute.
The initiative was widely welcomed when launched, although there were dissenting voices,such as Data & Society’s Robyn Caplan who raised concerns about the challenge of writing
algorithms to identify this type of content when the definitions are so broad. She also spokeout about the need to financially support this outsourced journalism. 172
While there is an increasing body of evidence suggesting that many types of ‘nudge’

168 Shukla, S. (August 28, 2017) Blocking Ads from Pages that Repeatedly Share False News. Facebook

Newsroom. https://newsroom.fb.com/news/2017/08/blocking-ads-from-pages-that-repeatedly-share-false-news/
169 Google News Lab funds First Draft, and has financially supported First Draft initiatives around the Frenchand UK elections, as well as training in Germany, Hong Kong and Korea.
170 Shahari, A. (Nov 11. 2017) Zuckerberg Denies Facebook’s Impact on the Election,
http://www.npr.org/sections/alltechconsidered/2016/11/11/501743684/zuckerberg-denies-fake-news-on-facebookhad-impact-on-the-election
171 Mosseri, A. (2016) News Feed FYI: Addressing Hoaxes and Fake News, Facebook Newsroom, December 15,
2016.
172 Caplan, R. “How Do You Deal with a Problem Like ‘fake News?’” Data & Society: Points, January 5, 2017.
https://points.datasociety.net/how-do-you-deal-with-a-problem-like-fake-news-80f9987988a9

59technologies can slow sharing, 173 and researchers, in experimental settings, have found thatwarning labels and pop-up boxes slow down the sharing of this content174 without access to
the results of this Facebook initiative we have no independent way of knowing whether the
program is slowing down the spread of polluted information on the platform. Despite
repeated calls for access to this data, most powerfully from the fact-checkers themselves, todate Facebook have refused to share the numbers. It’s therefore impossible to assess the
success of the project.
Experimental research by Leticia Bode in 2015 suggested that when a Facebook post that
includes mis-information is immediately contextualised in their ‘related stories’ feature
underneath, misperceptions are significantly reduced.175 In August 2017, Facebook
announced176 that they would be rolling out their related stories feature more widely to help
contextualize mis-information with fact-checked articles.
In January 2017, Facebook launched its ‘Facebook Journalism Project’ 177 and announced thatnews literacy would be a priority for the company. Beyond financially supporting non-profits
working in this space, they also rolled out a Public Service Announcement-type message at
the top of the New Feed in fourteen countries. This message linked to a post with 10 top tips
for spotting ‘false news’.178 These same tips were published as full page ads in newspapers inGermany, France and the U.K. They also committed a $14 million fund to help establish theNews Integrity Initiative based at the CUNY Journalism School in New York, which supports
the development of tools, research and media literacy programs globally.179
However, in a post titled ‘Facebook Must Either Innovate or Admit Defeat At The Hands Of
Fake News Hoaxsters’, Craig Silverman had some strong words for the platform:[If Facebook] “is truly committed to offering a quality Trending Topic (and News Feed)experience, its only option is to make massive strides in the detection and analysis of
the factual qualities of news articles. Developing what would likely be the world’s first
173 Bilton, R. (Feb.2, 2017) Reddit’s /r/worldnews community used a series of nudges to push users to fact-checksuspicious news, Nieman Lab, http://www.niemanlab.org/2017/02/reddits-rworldnews-community-used-a-seriesof-nudges-to-push-users-to-fact-check-suspicious-news/
174 Pennycook, G. et al. (2017) Unpublished research presented at the Harvard University event, Combating Fake
News: An agenda for Research and Action.
175
Bode, L. & Vraga, E (2015) In Related News, That Was Wrong: The Correction of Mis-information ThroughRelated Stories Functionality in Social Media, Journal of Communication, 65 (4): 619-638.
176 Su, S. (April 25, 2017) News Feed FYI: New Test With Related Articles,
https://newsroom.fb.com/news/2017/04/news-feed-fyi-new-test-with-related-articles/
177 Simon, F. (Jan. 11, 2017) Introducing: Facebook Journalism Project. Facebook Media Blog.
https://media.fb.com/2017/01/11/facebook-journalism-project/
178 Mullin, B. (April 6, 2016) Facebook rolls out global warning against fake news, Poynter
http://www.poynter.org/2017/facebook-rolls-out-global-warning-against-fake-news/454951/
179 Brown, C. (April 2, 2017) Introducing the News Integrity Initiative, Facebook Media Blog,
https://media.fb.com/2017/04/02/introducing-the-news-integrity-initiative/

60

algorithm to do this job with accuracy and consistency will require significant
engineering resources. But it’s what is necessary to actually stop Facebook from being
the world’s biggest platform for false and fake news without the use of editors. Right
now Facebook has no editors, a flawed algorithm, and a weak product.” 180
Facebook closed 30,000 automated accounts in France ahead of the election and ‘tens of
thousands’ in the U.K. ahead of their election. To date, these kinds of moves by Facebook
have timed with elections, but there are calls for Facebook to take an ongoing approach topolicing automation on the platform. 181
As discussed, the challenge of bots disseminating dis-information is a significant one. Ongoing
research by the Computational Propaganda Project 182 at the Oxford Internet Institutecontinues to identify the scale of the problem globally. In April 2017, Facebook’s Security
team published a paper on ‘Information Operations’, defining it as “actions taken byorganized actors (governments or non-state actors) to distort domestic or foreign politicalsentiment, most frequently to achieve a strategic and/or geopolitical outcome.” 183 It was thefirst-time Facebook had admitted the scale of the problem they face in terms of official,
tightly organised, networked agents using their platform to disseminate automated disinformation.
In September 2017, Facebook admitted that they had found evidence that ‘dark ads’ (ads thatare only visible to the intended audience, rather than publicly viewable on a page) had beenpurchased by a Russian organization and directed at US citizens. Facebook explained, “[T]heads and accounts appeared to focus on amplifying divisive social and political messages acrossthe ideological spectrum — touching on topics from LGBT matters to race issues toimmigration to gun rights.”184 A few days later, an investigation by the Daily Beast foundinauthentic accounts, seemingly located in Russia, had used the Facebook events function toorganize anti-immigration protests in the US.185
While Facebook is struggling on this topic, Twitter’s open APIs mean it is much easier for
academics and think tanks to visualise the bot networks that exist on their platform. Thus,
180 Silverman, C. (April, 30, 2017) Facebook Must Either Innovate Or Admit Defeat At The Hands Of Fake News

Hoaxsters, Buzzfeed, https://www.buzzfeed.com/craigsilverman/how-facebook-fell-into-a-fake-news-trap-of-itsown-making?utm_term=.uqGPlGAKp#.km8Vb7AmX
181
Hofileña, C. F. (Oct. 9, 2016) Fake accounts, manufactured reality on social media, Rappler,
https://www.rappler.com/newsbreak/investigative/148347-fake-accounts-manufactured-reality-social-media
182 http://comprop.oii.ox.ac.uk/
183 Weedon, J., W. Nuland & A. Stamos (April 27, 2017) Information Operations and Facebook, p. 4
https://fbnewsroomus.files.wordpress.com/2017/04/facebook-and-information-operations-v1.pdf
184 Stamos, A. (September 6, 2017) An Update On Information Operations On Facebook, Facebook Newsroom,https://newsroom.fb.com/news/2017/09/information-operations-update/
185
Collins, B. et al (September 11, 2017) Exclusive: Russia Used Facebook Events to Organize Anti-Immigrant
Rallies on U.S. Soil, The Daily Beast, http://www.thedailybeast.com/exclusive-russia-used-facebook-events-toorganize-anti-immigrant-rallies-on-us-soil

61

calls for action have been focused on Twitter. In a June 2017 blog post, Twitter explained its
efforts to fight against bots: “We’re working hard to detect spammy behaviors at source, suchas the mass distribution of Tweets or attempts to manipulate trending topics. We also reducethe visibility of potentially spammy Tweets or accounts while we investigate whether a policy
violation has occurred. When we do detect duplicative, or suspicious activity, we suspend
accounts. We also frequently take action against applications that abuse the public API toautomate activity on Twitter186, stopping potentially manipulative bots at the source.” 187
Ultimately, the question remains as to whether the social networks and technology
companies, as commercial entities, can ever spearhead serious solutions to the problem of
information disorder. As Martin Moore, Director of the Center for the Study of Media,
Communication and Power argued in his submission of evidence to the U.K. Parliamentary
Select Committee on Fake News:“Remedies solely or heavily based on technological fixes or market-driven corrections
will not, on their own, address these problems. Technology should be able to reduce
the spread of certain types of news (such as that which is shared without being readfirst), and to show where news is disputed. However, the long history of fake news,
the political, social and economic motivations for producing it, and the ease of selfpublishing online, mean that technology will only ever partly address the problem. It
also elicits dangers of its own with regard to the value-driven choices that engineers
will have to make when determining which news to promote and which to suppress.
Nor are market-driven corrections likely to solve, or even alleviate, the problem. The
technology platforms on which this news travels are reliant on advertising thatprioritises popular and engaging content that is shared widely. The content is notdistinguished by its trustworthiness, authority or public interest, since these are notcriteria that drive likes and shares.”
Emily Bell, director of the Tow Center for Digital Journalism argues, “The business of
publishing and monetizing information is never neutral; it is always deeply political. It shapes
opinion, informs markets, reinforces biases, creates understanding, and spreads confusion.
Facebook has said more than once that it does not want to be an arbiter of the truth, but it
also does not want to be the purveyor of lies. Journalists have known for a long time what
technology companies are just finding out: what you don’t publish is as brand-defining as
what you do.”188

186 Twitter, (April 6, 2016) Automation Rules, https://support.twitter.com/articles/76915
187 Crowell, C., (June 14, 2017) Our Approach to Bots and Automation,

https://blog.twitter.com/official/en_us/topics/company/2017/Our-Approach-Bots-Mis-information.html
188
Bell, E. (Dec. 15, 2015), Facebook drains the fake news swamp with new, experimental partnerships,
Columbia Journalism Review,https://www.cjr.org/tow_center/facebook_drains_fake_news_swamp_new_experimental_partnerships.php

62

In addition to these steps to shut down automated accounts by the social networks, theability to identify and discredit sources of dis-information in real time is an increasingly
necessary skill for newsrooms to master. We hope technology companies will support
newsrooms in being able to identify the agents of a post. If newsrooms were provided withthe tools to identify whether the agents of a piece of dis-information are official, organised
and automated, they would be able to quickly raise red flags for audiences.
When assessing moves by the technology companies over the past year, one of the mostfrustrating elements has been the failure to connect with the research, education, library, civilsociety, and policy communities at any substantive level. There are decades of research on
mis-information, how people ‘read’ and make sense of information, the factors that slow
down or exacerbate rumours. But the responses have often felt knee-jerk and atheoretical,
and at times public relations moves rather than serious attempts to tackle the complexity ofthe problem. On this topic, when the scale and seriousness require sophisticated responses,
the technology companies must work more closely with those who have research expertiseon this subject, as well as those working on the ground around the world, and experience
first-hand the real-world repercussions of information pollution.
Blacklisting, flagging and credibility scores
Creating lists of problematic sites was an early and popular suggestion, but as US academic
Melissa Zimdars found to her detriment 189, attempting to be the ‘arbiter of truth’ can cause
you to become incredibly unpopular. Notably, her list, now hosted at opensources.co, hasbeen used by a number of technologists building tools to help ‘flag’ problematic content
through browser extensions such as Check This190. One such tool from the French daily
newspaper Le Monde is based on a database of sites reviewed by Le Monde’s fact-checkers
and191allows readers to search a website’s URL to check whether it has published unreliable
content. The goal of most of these projects is to build a system that could be integrated with
Google and Facebook and used to down-rank certain content from ‘less credible’ sources sothat users are less likely to see it.
Recently, the social news agency Storyful teamed up with the advertising analytics firm Moat
and CUNY Journalism School to create the Open Brand Safety Framework, an attempt tocreate the master black list of ‘fake news’ sites that advertisers can avoid.192 The desire foradvertising companies to stay away from problematic content has become increasingly clear

189 Zimdars, M. (Nov 18, 2016) My ‘fake news list’ went viral. But made-up stories are only part of the problem,

https://www.washingtonpost.com/posteverything/wp/2016/11/18/my-fake-news-list-went-viral-but-made-upstories-are-only-part-of-the-problem/
190 https://chrome.google.com/webstore/detail/check-this-by-metacert/felmjclcjadopolhjmlbemfekjaojfbn?hl=en
191 http://www.lemonde.fr/verification/
192 Doctor, K., (May 2, 2017) Can a Master Blacklist Choke Off Fake News, Nieman Lab.
http://www.niemanlab.org/2017/05/newsonomics-can-a-master-blacklist-choke-off-fake-news-money-supply/63

over the past few months, as the success of the online activist group Sleeping Giants 193
demonstrates.194
Categorising content, while seemingly a well-meaning exercise for providing people withadditional context, can quickly backfire when people question the authority of those who
create the labels. Indeed, one can easily imagine algorithms programmed to identify, de-rankor take down certain types of content might produce unintended consequences. A
clampdown on dis-information should not become an excuse for suppressing dissenting or
minority views. An organization labelling something as ‘fake’ should provide full transparency
around how it makes its ‘blacklists.’
Credibility scores
In conversations about mis-information, there are regular comparisons drawn between
deceptive articles and email spam. Questions are often raised about why techniques similar to
those used to combat email spam can’t be used to identify and down-rank poor quality
content.
The Trust Project195, led by Sally Lehrman at Santa Clara University, has been working on a setof criteria that would help audiences know what content to trust. Standards include whether
an outlet has a corrections policy and whether a reporter has written on the topic previously.
The hope is that if newsrooms added this information as metadata to online articles,
Facebook and Google could ‘read’ these signals and place them higher algorithmically.
There are also two US-based projects working on credibility: the Technical Schema for
Credibility196, led by Meedan in collaboration with Hacks/Hackers, and the News Quality Score
Project, led by Frederic Filloux197. The projects are developing markers of credibility, to see if
there can be a programmatic way for social networks use their scores to influence algorithmic
rankings.
Stronger media
CUNY Journalism Professor Jeff Jarvis has said “our problem isn’t ‘fake news.’ Our problem is

193 https://www.facebook.com/slpnggiants/
194 Alba, D. (Dec. 15, 2016) Meet the Ad Companies Ditching Breitbart and Fake News, Wired, Available at:https://www.wired.com/2016/12/fake-news-will-go-away-tech-behind-ads-wont-pay/
195 http://thetrustproject.org/
196 Mina, A.X. (March 2. 2017) Building Technical Standards for Credibility, MisinfoCon on Medium.
https://misinfocon.com/building-technical-standards-for-credibility-59ef9ee4ab73
197 Filloux, F. (June 25, 2017) The News Quality Scoring Project, Medium, https://mondaynote.com/the-newsquality-scoring-project-surfacing-great-journalism-from-the-web-48401ded8b53

64trust.”198 As has been well documented, trust in mainstream media has been falling for
decades, as has (it must be noted) trust in other public institutions. 199 Ethan Zuckerman’s
recent essay on the topic describes the slow decline of trust:“Addressing the current state of mistrust in journalism will require addressing thebroader crisis of trust in institutions. Given the timeline of this crisis, which is
unfolding over decades, it is unlikely that digital technologies are the primary actor
responsible for the surprises of the past year. While digital technologies may help us
address issues, like a disappearing sense of common ground, the underlying issues of
mistrust likely require close examination of the changing nature of civics and public
attitudes to democracy.”200
It’s worth recognizing how trust in journalism varies geographically. Recent comparative
analysis by the Reuters Institute in its annual Digital News Report201 shows how news media
enjoy different levels of trust in different countries:

Figure 13: From the Digital News Report, 2017 by the Reuters Institute for the Study ofJournalism - Overall Trust in the News Media
There are many reasons for the decline of trust in media. Improving these numbers is notgoing to happen quickly, but initiatives to help build trust and credibility go hand in hand withany initiatives aiming to combat mis- and dis-information.
198

Jarvis, J. (2017) Our problem isn’t ‘fake news.’ Our problems are trust and manipulation.
https://medium.com/whither-news/our-problem-isnt-fake-news-our-problems-are-trust-and-manipulation5bfbcd716440\
199 Harrington, M (Jan. 16, 2017) Survey: People’s Trust Has Declined in Business, Media, Government, andNGOs, Harvard Business Review, https://hbr.org/2017/01/survey-peoples-trust-has-declined-in-business-mediagovernment-and-ngos
200 Zuckerman, E. (August 2017) Mistrust, Efficacy and the New Civics, A whitepaper for the Knight Foundation,
https://dspace.mit.edu/handle/1721.1/110987#files-area
201 Newman, N. (2017) Digital News Report, Reuters Institute for the Study of Journalism,
https://reutersinstitute.politics.ox.ac.uk/sites/default/files/Digital%20News%20Report%202017%20web_0.pdf

65Strategic Silence
As Data & Society outlined in their May 2017 report Media Manipulation and Dis-information
Online, “for manipulators, it doesn’t matter if the media is reporting on a story in order to
debunk or dismiss it; the important thing is getting it covered in the first place.” 202 Certainly,
during the weekend of the #MacronLeaks, Ryan Broderick of Buzzfeed reported thatmembers of 4Chan discussion boards were linking to stories debunking the information, andcelebrating them as a form of engagement.203 While reporting on these stories, and thepeople behind the stories, feels a natural response by journalists at this point in time, there isa real need for the industry to come together to discuss the impact of reporting on disinformation, and providing oxygen for rumours or fabricated content that otherwise would
stay in niche communities online. We would recommend cross-industry meetings whereby
senior editors could discuss whether there is a need to reach a shared agreement on when arumour or piece of content crosses a tipping point, and moves from niche online communities
to a wider audience. The French rules which prevented any discussion of election related
topics for the forty-eight hours before the polls closed, meant there was no discussion of theleaks by the mainstream media in France, something which raised eyebrows amongst US
journalists. The idea of strategic silence in the coverage of mal- and dis-information might sit
uncomfortably with some, but we would argue there is a need for these conversations to takeplace.
Identifying the sources of dis-information
In Paul & Matthews report on Russian propaganda for the RAND corporation, they argue that
one of the most effective ways of tackling the issue is to inoculate users, or to “forewarn
audiences of mis-information, or merely reach them first with the truth, rather thanretracting or refuting false ‘facts.’204
However, the current trend is fact-checking initiatives.205 Since 2016, we have seen thecreation of numerous fact-checking organizations, new teams206 and election-based initiatives
like CrossCheck,207 which worked to debunk rumours and claims around the French election.
The difficulty here is that if “fake news isn’t about facts, but about power, then independentfact-checking alone won’t fix it — particularly for readers who already distrust the
202 Marwick, A and R. Lewis (May 2017) Media Manipulation and Dis-information Online, Data & Society,https://datasociety.net/output/media-manipulation-and-disinfo-online/, p.39
203
Ryan Broderick, (broderick) “I covered a different 4chan Macron rumor last week. They don't care if it's nottrue. They want it debunked”. (May 5, 2017, 11.19am)
https://twitter.com/broderick/status/860423715842121728?lang=en
204 Paul and Matthews, (2016) p.9
205 Mantzarlis, A. (2016) There’s been an explosion of international fact-checkers, but they face big challenges,
Poynter, http://www.poynter.org/2016/theres-been-an-explosion-of-international-fact-checkers-but-they-face-bigchallenges/415468/
206 See the launch of the BBC’s Reality Check: Jackson, J. (Jan. 12, 2017) BBC sets up team to debunk fakenews, The Guardian, https://www.theguardian.com/media/2017/jan/12/bbc-sets-up-team-to-debunk-fake-news
207 https://firstdraftnews.com/project/crosscheck/66

organizations that are doing the fact-checking.”208 (Borel, 2017)
However, as Jeff Jarvis argues, there are other techniques which are not currently a naturalpart of reporting. For example, “journalism should cover the manipulators’ methods but nottheir messages…. We should not assume that all our tried-and-true tools — articles,
explainers, fact-checking — can counteract manipulators’ propaganda. We must experimentand learn what does and does not persuade people to favor facts and rationality.” (Jarvis,
2017)
A Belgium start-up, Saper Vedere is making a similar claim, based on its analysis of theeffectiveness of fact-checks during the French election 209. In its visualisation below, one cansee the audience for the rumour that Macron was funded by Saudi Arabia, as well as theaudience of its debunk. There is almost no overlap between these two groups.

Figure 14: Visualization of Twitter accounts discussion a rumour that Emmanuel Macron was being
funded by Saudi Arabia. The accounts on the left in the green box were discussing the debunk. The
account in red were discussing the rumour. The two communities hardly overlap. Credit: Alexandre
Alaphilippe and Nicolas Vanderbiest.

Instead, they argue that journalists need better tools to be able to identify the sources of disinformation in real-time: source-checking. When bot accounts who originated a rumour
appear to be based in a country other than the one connected with said rumour, it could
prove to be a faster way of encouraging skepticism in the audience than debunking the fact
itself.
208 Borel, B. (Jan. 4, 2017) Fact-checking Won’t Save Us from Fake News, FiveThirtyEight,

https://fivethirtyeight.com/features/fact-checking-wont-save-us-from-fake-news/
209 http://www.saper-vedere.eu/

67Education
In a large-scale exercise designed to evaluate students’ ability to evaluate information sources
online, researchers at Stanford University were surprised by the degree to which respondents
were unable to identify an advert from editorial content or question the partisan nature offacts presented to them.210 The call for more news literacy programs211 has been deafening
recently, and they are one solution on which almost everyone can agree.
danah boyd in a provocative piece titled ‘Did Media Literacy Backfire’ from January 2017
argued media literacy has actually taught students not to trust Wikipedia while failing to givethem sufficient critical research skills to know how to ascertain the credibility of any one pieceof information.212 boyd identified a significant problem: news literacy has been distorted into
distrust of the media and selective research that reaffirms beliefs.
The specifics of how such programs should be rolled out, in terms of the format, structure andcontent of program curriculum, have supported very vibrant discussions. In addition to more
traditional ideas around news literacy, such as how to differentiate between opinion and hardnews, there have been calls to include elements like the critical assessment of statistical andquantitative statements in the media 213, a deep understanding of algorithms and artificial
intelligence214 and greater emotional skepticism.215
There is also a need to educate people on the power of images to manipulate and persuade.
As discussed earlier, the way we understand visuals is fundamentally different to how wethink about text. While much of the ‘fake news’ debate to date has been about text-based
dis-information, the election monitoring projects First Draft has worked on in the US, UK,
France and Germany have shown how frequently dis-information appears in visual formats—whether doctored images, fabricated videos, misleading visualizations or ‘memes’ (striking
images with text superimposed over top). In an investigation carried out in the run up to the
French election, Buzzfeed discovered loose networks of US teenagers creating ‘meme shells’
210 Stanford History Education Group, (Nov. 22, 2016) Evaluation Information: The Cornerstone of Civic Online

Reasoning. https://sheg.stanford.edu/upload/V3LessonPlans/Executive%20Summary%2011.21.16.pdf
211 This document from 1999 titled ‘7 Great Debates in Media Literacy, is still incredibly relevant’
http://files.eric.ed.gov/fulltext/ED439454.pdf
212
boyd, d. (Jan. 5, 2017) Did Media Literacy Backfire, Data and Society: Points
https://points.datasociety.net/did-media-literacy-backfire-7418c084d88d
213 Written evidence submitted by the Royal Statistical Society to the UK Parliamentary Inquiry on Fake News,https://www.parliament.uk/business/committees/committees-a-z/commons-select/culture-media-and-sportcommittee/inquiries/parliament-2015/inquiry2/publications/
214 Written evidence submitted by the UCL Knowledge Lab to the UK Parliamentary Inquiry on Fake News
http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/culture-media-and-sportcommittee/fake-news/written/48571.html
215 On The Media, (2017) Rise and Fall of Fake News, WNYC, http://www.wnyc.org/story/rise-and-fall-fakenews/68

(generic images related to the candidates) that anyone could use to create memes for social
media.
A recent study at Stanford University observed 10 Ph.D. historians, 10 professional fact
checkers and 25 Stanford University undergraduates as they evaluated live websites andsearched for information on social and political issues. They found that “historians andstudents often fell victim to easily manipulated features of websites, such as official-looking
logos and domain names. They read vertically, staying within a website to evaluate its
reliability. In contrast, fact checkers read laterally, leaving a site after a quick scan andopening new browser tabs in order to judge the credibility of the original site. Compared tothe other groups, fact checkers arrived at more warranted conclusions in a fraction of thetime.”216
The question is how to make this type of ‘reading’ habitual amongst students.
Ultimately, there is an acceptance that any curriculum should not lecture students. Telling
students they are wrong is not a solution, and may even be counter-productive. As InformAll
and the CILIP Information Literacy Group testified to the UK Parliament, “[t]he essence of any
solution lies in stimulating curiosity and a spirit of enquiry, and crucially, finding effective
ways of triggering this curiosity, in the education system and beyond.” 217
One of the most impressive initiatives is The Digital Polarization Initiative218, which waslaunched by the American Association of State Colleges and Universities and is led by Mike
Caulfield. It is a national effort to build student civic, information and web literacy by having
students participating in a broad, cross-institutional project to fact-check, annotate andprovide context to the different news stories that show up in their Twitter and Facebook
feeds. As Caulfield explains: “The point is to get students to understand the mechanisms andbiases of Facebook and Twitter in ways that most digital literacy programs never touch. The
point is not to simply decode what’s out there, but to analyze what is missing from ourcurrent online environment, and, if possible supply it.”219
Programs that have focused on critical thinking, source evaluation and emotional
manipulation have seen success. In Ukraine, the nongovernmental organization IREX, trained
15,000 people on a program called Learn to Discern, which was designed to teach citizens
216 McGrew, S., T. Ortega, J. Breakstone & S. Wineburg, (Fall 2017) The Challenge That’s Bigger Than Fake

News: Teaching Students to Engage in Civic Online Reasoning. American Educator.
217 Written evidence submitted by InformAll and the CILIP Information Literacy Group to the UK Parliamentary
Inquiry on Fake News,
http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/culture-media-and-sportcommittee/fake-news/written/48215.html
218 http://www.aascu.org/AcademicAffairs/ADP/DigiPo/
219 Caulfield, M. (Dec 7. 2016) Announcing the Digital Polarization Initiative. Hapgood.
https://hapgood.us/2016/12/07/announcing-the-digital-polarization-initiative-an-open-pedagogy-joint/69

how to separate fact from fiction and recognize manipulation and hate speech. In theirevaluation of the project, they found an observed 24% increase in participants’ ability to
distinguish trustworthy news from false news, a 22% increase in those who cross-check theinformation in the news they consume, and a 26% increase in participants' confidence inanalyzing news.220
Established programs like the News Literacy Project221, which is focused on providing
materials and curricula to High School students; the Stonybrook Center for News Literacy222,
which offers skills training to University students; and a new online course being offered byHong Kong University223 are all also currently leading the thinking on best practices in thisarea.
It seems there is a need for a task force on the best approaches for teaching news literacy,
creative thinking about a standardized curriculum and rigorous testing of new techniques.
Suggested elements of any curriculum include: (i) traditional news literacy skills; (ii) forensic
social media verification skills; (iii) information about the power of algorithms to shape what
is presented to us; (iv) the possibilities but also the ethical implications offered by artificial
intelligence; (v) techniques for developing emotional scepticism to override our brain’s
tendency to be less critical of content that provokes an emotional response; and (vi) statisticalnumeracy.
Regulation
The First Amendment of the US Constitution means that, for all the discussion in the US about
the impact of fabricated and manipulated content, there is very little appetite in the US for
any type of regulatory intervention.224 In Europe however, the regulatory wheels have beenturning slowly, and we are starting to see legislation directed at information disorder.
Germany, for instance, recently passed the Network Enforcement Law, which concentrates
primarily on hate speech, and introduces a possible imposition of fines on the social networks
if they don’t take down hateful or defamatory content within twenty-four hours. The BBC
World Service survey measuring attitudes to information and the internet of eighteen

220 Susman-Peña, T. and Vogt, Katya (June 12, 2017) Ukrainians’ self-defense against information war: What welearned from Learn to Discern, IREX, https://www.irex.org/insight/ukrainians-self-defense-against-informationwar-what-we-learned-learn-discern
221 http://www.thenewsliteracyproject.org/
222 https://www.centerfornewsliteracy.org/
223 https://www.coursera.org/learn/news-literacy
224
Gillespie, T. (in press) Governance of and by platforms. In Burgess, J., Poell, T., & Marwick, A. (Eds), SAGE
handbook of social media. Retrieved from http://culturedigitally.org/wp-content/uploads/2016/06/GillespieGovernance-ofby-Platforms-PREPRINT.pdf

70countries, showed that only in two of the eighteen countries, China and the UK, did a majority
want their governments to regulate the internet.225
However, there’s little denying that the regulatory discourse in Europe has been loud sincelate December 2016, when Giovanni Pitruzzella, the chairman of the Italian Competition
Authority, told the Financial Times that EU countries should deal with "post-truth" politics bysetting up antitrust-like agencies devoted to spotting and removing fake news.226
And then, in January 2017, Andrus Ansip, the European Commission (EC) Vice President for
the Digital Single Market, warned that if Facebook and other tech companies didn’t taketougher stances on fake news, the EC might have to step in. Ansip told the Financial Times in
an interview, “I really believe in self-regulatory measures, but if some kind of clarifications areneeded then we will be ready for that.”227 However, on Twitter, he stressed228 that he was
not referring to a ‘ministry of truth.’
The European Commission has already pushed Facebook, Twitter, YouTube and Microsoft tosign up to a code of conduct229 that aims to tackle online hate speech and take down the
majority of potentially illegal content within 24 hours. Many fear this code of conduct might
become a blueprint for regulating fabricated content online.
In the United Kingdom, the Culture, Media and Sport Committee set up a Fake News Inquiry,
and evidence was submitted 230 by 79 experts and organizations. The inquiry was closed whenthe election was called, and it is unknown whether it will reconvene.
In the Czech Republic, officials are monitoring fake news directly. Ahead of the country’s
general election in October, the Czech government has set up a “specialised analytical andcommunications unit”231 within the Ministry of the Interior that, as part of its work to monitor
threats to internal security, will also target “dis-information campaigns.” According to the
Ministry, it will “not force the ‘truth’ on anyone, or censor media content.” Rather, as theunit’s Twitter page explains, it will assess whether the dis-information seriously affects

225 Cellan-Jones, R. (Sept 22, 2017) Fake news worries 'are growing' suggests BBC poll, BBC News,

http://www.bbc.com/news/technology-41319683
226 Politi, J. (Dec. 30, 2016) ‘Italy antitrust chief urges EU to help beat fake news’

https://www.ft.com/content/e7280576-cddc-11e6-864f-20dcb35cede2?mhq5j=e2
227 Bond, D. and Robinson, D. (Jan. 29, 2017), ‘European Commission fires warning at Facebook over fake news’
Financial Times, https://www.ft.com/content/85683e08-e4a9-11e6-9645-c9357a75844a?mhq5j=e2
228 https://twitter.com/Ansip_EU/status/826085369493995522
229 http://europa.eu/rapid/press-release_IP-16-1937_en.htm
230 https://www.parliament.uk/business/committees/committees-a-z/commons-select/culture-media-and-sportcommittee/inquiries/parliament-2015/inquiry2/publications/
231 http://www.mvcr.cz/cthh/clanek/centre-against-terrorism-and-hybrid-threats.aspx

71internal security, and, if so, it will respond by publicising available facts and data that disprove
the fake story.
Whatever happens in Europe will set an important global precedent. Already, Singapore’s
Law and Home Affairs Minister K. Shanmugam stated that laws to tackle the “scourge of fakenews” are expected to be introduced next year.232
Any attempt to create a regulatory framework will be problematic without appropriate
definitions for information disorder. When politicians or policymakers talk about ‘fake news’,
what are they targeting? Fabricated news sites created for profit? Twitter raids created byloose networks of bored US teens on 4Chan? Anti-European broadcasts propagated by Russia
Today?
As Jan Kleijssen, the Director of the Information Society and Action against Crime Departmentof the Council of Europe reminds us, “When we speak about freedom of expression today, we
often hear a ‘but’ - and then mention is made of ‘hate speech’ and ‘fake news’. At the Councilof Europe, we believe that we have to be very careful with that ‘but’ after freedom of
expression. We are talking about one of the most important foundations of democracy, oneof the most important foundations of democratic security.” The topics of mis-, mal- and disinformation are too important to start legislating and regulating around until we have ashared understanding of what we mean by these terms.

232 Yi, S.B. (June 19, 2017) ‘New legislation to combat fake news likely to be introduced next year: Shanmugam’,

Straits Times, http://www.straitstimes.com/singapore/new-legislation-to-combat-fake-news-next-year-shanmugam72

Figure 15: Cartoon by Cathy Wilcox, drawn for UNESCO for World Press Freedom Day 2017.
An easier regulatory move that we are likely to see soon is connected to online advertising on
Facebook. With the news that Russia was buying ‘dark’ posts on Facebook and targeting themat US citizens in the run up to the 2016 US election, there is a growing pressure for increasedtransparency around these types of advertisements.233 Without any oversight on what isbeing published and to whom, there can be no accountability. In most democracies paid-for
election related communication is held to certain standards before it can be broadcast or
published. In 2011, the Federal Communications Commission ruled that Facebook did nothave to require disclaimers on its paid-for posts, but we expect this to be revisited as theopportunities presented by this technology, to those trying to sow dis-information, become
clearer.
While Mark Zuckerberg announced on September 21, 2017 that Facebook will ensure thatanyone advertising on Facebook will have to disclose which page paid for an ad, and will also
ensure that you can visit an advertiser’s page and see the ads that they are currently running

233 Vaidhyanathan, S. (Sept 8. 2017) Facebook Wins, Democracy Loses. New York Times,

https://www.nytimes.com/2017/09/08/opinion/facebook-wins-democracy-loses.html

73

to any audience on Facebook. While this seems like a positive step, as a group of
distinguished academics wrote in response to the announcement via an open letter:
Transparency is a first step in the right direction. Digital political advertising
operates in a dynamic tension between data and humans, commerce andpolitics, power and participation. Some of these tensions can be resolved bytransparency, others not. The way forward is to engage with governments,
regulators, election monitoring bodies, civil society and academics to develop
public policies and guidelines for ensuring fairness, equality, and democratic
oversight in digital political campaigns.234

234

Helberger, N. et al. (Sept 22. 2017) Dear Mark: An Open Letter to Mark Zuckerberg in response to hisstatement on political advertising on Facebook, Available at:https://www.dropbox.com/s/7v3vpk9yw5sa18b/Dear%20Mark_final..pdf?dl=074

Part 4: Future trends
Messaging apps
As discussed, much of the recent debate around mis- and dis-information has focused on
their political impact, and this debate has been largely shaped by events during the US
election. As a result, much of the focus has been on the Facebook News Feed. But even acursory glance outside of the US demonstrates that the next frontier for mis- and disinformation is closed-messaging apps.235
According to the Digital News Report published by the Reuters Institute for the Study ofJournalism236, the use of the Facebook-owned WhatsApp as a news source rivals Facebook in
a number of markets, including Malaysia, Brazil and Spain. While WhatsApp is clearly themost dominant messaging app globally, the popularity of different apps in other countries isquite startling. For example, WeChat, the most popular messaging app in China, has 963
million users as of Q2 2017. 237
The obvious challenge of tackling rumours and fabricated content on these messaging apps isthat it’s impossible to know what is being shared. There are innovative projects attempting totackle rumours being shared on these apps. One example is the Thai News Agency’s ‘Sure andShare’ project, which encourages audience members to submit questions they have about
content, rumours or stories circulating on the messaging app LINE. The news agency thencreates engaging infographics or YouTube videos based on their fact-checking and shares
them on their LINE channel. Similar initiatives are emerging for WhatsApp in Colombia 238 andIndia239.
Augmented reality, artificial reality and voice recognition
As we continue to undertake research and work collaboratively on solutions, our biggest
challenge will be the speed at which technology is refining the creation of fabricated video
and audio.

235

Dias, N. (August 17, 2017) The Era of Whatsapp Propaganda is Upon Us, Foreign Policy,
http://foreignpolicy.com/2017/08/17/the-era-of-whatsapp-propaganda-is-upon-us/
236 Newman, N. (2017), p. 10.
237 Tencent. (n.d.). Number of monthly active WeChat users from 2nd quarter 2010 to 2nd quarter 2017 (inmillions). In Statista - The Statistics Portal. Retrieved September 15, 2017, fromhttps://www.statista.com/statistics/255778/number-of-active-wechat-messenger-accounts/
238 Serrano, C. (March 20, 2017) To slow the spread of false stories on WhatsApp, this Colombian news site isenlisting its own readers, Nieman Lab, http://www.niemanlab.org/2017/03/to-slow-the-spread-of-false-stories-onwhatsapp-this-colombian-news-site-is-enlisting-its-own-readers/
239 https://twitter.com/boomlive_in/status/861559074378452992

75

Research by Justus Thies and colleagues240 has demonstrated how technologists can change
facial expressions in live video. And, as Nick Bilton wrote in an article for Vanity Fair, “Thetechnology out of Stanford that can manipulate a real-time news clip doesn’t need an array ofhigh-end computers like those used by Pixar; it simply needs a news clip from YouTube and astandard Webcam on your laptop.”More recently, researchers at the University of Washington used artificial intelligence tocreate visually convincing videos of Barack Obama saying things he had said before, but in acompletely different context.241 The researchers fed a neural network seventeen hours offootage from the former president’s weekly addresses as ‘training data’. The resulting
algorithm was able to generate mouth shapes from Obama’s voice and overlay them onto
Obama’s face in a different “target” video.
Audio can be manipulated even more easily than video. Adobe has created Project VoCo,
which has been nicknamed ‘Photoshop for audio’. The product allows users to feed a 10-to-20
minute clip of of someone’s voice into the application and then dictate words in that person’s
exact voice. Another company called Lyrebird242 is working on voice generation. On its site, it
claims to “need as little as one minute of audio recording of a speaker to compute a unique
key defining her/his voice. This key will then generate anything from its corresponding voice.”
It also plans to create an API whereby other platforms could easily use those voices.
Finally, Mark Zuckerberg at Facebook’s Developer Conference, F8, in April 2017,
demonstrated new Augmented Reality technology that allows users to seamlessly ‘add’
features and filters to their images or videos. Zuckerberg used the example of adding moresteam to the image of his morning coffee. While this is a harmless example, the darker
versions of augmented reality are easy to imagine.

240 Thies, J. at al. (2016) Face2Face: Real-time Face Capture and Reenactment of RGB Videos, The IEEE

Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2387-2395
241 Suwajanakorn, S. et al. (July 2017) Synthesizing Obama: Learning Lip Sync from Audio, ACM Transactions
on Graphics, 36 (4). Article 95. http://grail.cs.washington.edu/projects/AudioToObama/siggraph17_obama.pdf
242 https://lyrebird.ai/demo76Part 5: Conclusions
This report has provided a conceptual framework for thinking about information disorder. We
hope that the definitions explained here will provide a structure for conversations bypolicymakers, legislators and researchers who are investigating the phenomenon. Only by
beginning with a shared understanding can we start constructively discussing solutions. We
also hope that our conceptual framework, which outlines the different elements and phasesof information disorder, will help bring nuance to debates about this issue.
First, we need to understand communication as something beyond just a transmission of
messages. People’s consumption of news and information is, first and foremost, a way toreaffirm their affinity with a larger dramatic narrative about the world and their place in it,
and transcends facts and figures.
Second, if we are serious about creating solutions, we need to consider the specificmotivations of different types of ‘Agents’, the characteristics of different types of ‘Messages’
and the factors impacting how people ‘Interpret’ those messages. We also need to recognise
how messages and the motivations about them can shift and transform as other agents reproduce and disseminate these messages.
We have also outlined research from different disciplines and methodological backgrounds. In
particular, we want to connect the excellent experimental work that has helped us tounderstand how people process information with the sociological and cultural theories thathighlight how and why people seek out information and use it to position themselves withincertain ‘tribes’.
To us, it is clear that any solutions will need to be based on a multi-disciplinary approach.
While some recent psychological research in the US context has highlighted the effects offact-checking initiatives in making people question information, it has also shown that these
initiatives can have little impact on people’s underlying beliefs—as in the case of Donald
Trump’s supporters. We need more research on the influence of emotions on the way
humans make sense of and use information in their lives.
As D’Ancona underlines, conspiracy theories are effective because they are based on
powerful narratives. They unconsciously tap into deep-seated fears. “Veracity will bedrowned out unless it is resonant.”243 There is research that shows that for false information
to be challenged effectively, our brains need it to be replaced with an alternative narrative244.
243 D’Ancona, (2017), p.131
244 Nyan, B. and J. Reifler (2015) ‘Displacing Mis-information about Events: An Experimental Test of Causal

Corrections’, Journal of Experimental Political Science, 2 (1) : 81-93

77

So to use rumours about Obama’s religious affiliation as an example, rather than stating‘Barack Obama is not a Muslim’ it is more effective to provide a story (preferably with apowerful narrative structure) of Obama going to his local Christian church with his family.
We need to fight rumours and conspiracy with engaging and powerful narratives thatleverage the same techniques as dis-information. As discussed in Part 1, effective strategies
for dis-information include: provoking an emotional response, repetition, a strong visual
aspect and a powerful narrative. If we remember the powerful, ritualistic aspects to
information seeking and consumption, the importance of integrating these elements into oursolutions is obvious.
While the explosion of fact-checking and debunking initiatives is admirable, there is an urgent
need to understand the most effective formats for sparking curiosity and skepticism inaudiences about the information they consume and the sources from which that information
comes. Simply pushing out more ‘factual information’ into the ecosystem, without sufficiently
understanding the emotional and ritualistic elements of communication, is potentially a wasteof time and resources.
We are seeing success with the use of nudge technology to remind people to check theveracity of information before re-sharing, as well as hopeful initiatives such as theInternational Center For Journalist’s TruthBuzz competition, which is encouraging people todesign fact-checking and debunking formats that are highly engaging and shareable. We arealso seeing games being created to help teach news literacy skills.245 Formats for reporting on
mis-information do seem matter in terms of connecting with audiences. A recent experiment
found that videos were considered “more interesting and understandable” than a comparable
print-based fact-checking story.246
And as the director of the International Fact-Checking Network, Alexios Mantzarlis concludes:“We need to find formats for people who are bored with reading long articles stuffed with

245 Schmidt, C. (Aug. 3, 2017) Games might be a good tool for fighting fake news. Here’s what three developers

have learned, Nieman Lab, http://www.niemanlab.org/2017/08/games-might-be-a-good-tool-for-fighting-fakenews-heres-what-three-developers-have-learned/
246 Young, D. et al. (2017) Fact-Checking Effectiveness as a Function of Format and Tone: Evaluating
FactCheck.org and FlackCheck.org, Journalism and Mass Communication Quarterly, 1-27.

78

hyperlinks. On this front I am glad to note some success on Snapchat 247 and bots248. But wehaven’t seen a breakaway podcast, and seen the struggles with TV.”249
The next area of research and testing should be devoted to understanding the performative
element of why people share mis-, mal- and dis-information. How can that be slowed down?
What cultural factors would make it shameful or embarrassing? In addition to teachingemotional scepticism as part of news literacy education programs, how do we teach people
how to call out their friends and family when they post fabricated or misleading content onsocial networks or in closed messaging apps? Brooke Borel, in a piece titled ‘How to Talk toYour Facebook Friends about Fake News,’250 provided tips based on academic literature and acase study of two old high school friends who managed to overcome their political differences
via a long exchange on Facebook. We need more research, and accessible resources derived
from this research, to help people navigate the challenges of peer-to-peer news literacy.
The technological developments outlined in the previous section demonstrate that, as we run
to catch up with the current phenomenon of polluted information streams, we are going tohave to run faster if we want to be well-placed to deal with these technologicaladvancements. In a short amount of time, audiences will have little trust in the information
they find online, dismissing any image, video or audio clip as potentially fabricated ormanipulated. The implications of this reality are truly terrifying, particularly as our societies
become increasingly polarized and divided. We must work together on solutions driven byresearch and experimentation to mitigate dis-information and significantly improveinformation literacy. Knee-jerk reactions based on poor definitional frameworks, or simplistic
calls to limit access to information, will only create more problems in the long run.
Information disorder cannot be solved overnight, but the first step is understanding thecomplexity of the issue. We hope this report has provided useful framing and context.

247 Mantzarlis, A (Feb. 2, 2016) Fact-checkers experiment with Snapchat, GIFs and other stuff millennials ♥.

Poynter.
http://www.poynter.org/2016/fact-checkers-experiment-with-snapchat-gifs-and-other-stuff-millennials%E2%99%A5/393992/
248 Mantzarlis, A (May 10, 2016) Did Clinton or Trump twist the facts? This messaging bot will tell you. Poynter.
http://www.poynter.org/2016/did-clinton-or-trump-twist-the-facts-this-messaging-bot-will-tell-you/409457/
249 Mantzarlis, A (June 7, 2016) There’s been an explosion of international fact-checkers, but they face big
challenges, Poynter. http://www.poynter.org/2016/theres-been-an-explosion-of-international-fact-checkers-butthey-face-big-challenges/415468/
250 Borel, B, (2017) Hpw to Talk to your Facebook Friends about Fake News, Open Notebook
http://www.theopennotebook.com/2017/02/21/how-to-talk-to-your-facebook-friends-about-fake-news/

79Part 6: Recommendations
What could technology companies do?
1. Create an international advisory council. We recommend the creation of an
independent, international council, made up of members from a variety of disciplinesthat can (1) guide technology companies as they deal with information disorder and(2) act as an honest broker between technology companies.
2. Provide researchers with the data related to initiatives aimed at improving thequality of information. While technology companies are understandably nervous
about sharing their data (whether that’s metrics related to how many people see afact-check tag, or the number of people who see a ‘disputed content’ flag and then do
not go on to share the content), independent researchers must have better access tothis data in order to properly address information disorder and evaluate theirattempts to enhance the integrity of public communication spaces. As such, platforms
should provide whatever data they can—and certainly more than they are currently
providing.
3. Provide transparent criteria for any algorithmic changes that down-rank content.
Algorithmic tweaks or the introduction of machine learning techniques can lead tounintended consequences, whereby certain types of content is de-ranked or removed.
There needs to be transparency around these changes so the impact can be
independently measured and assessed. Without this transparency, there will be claimsof bias and censorship from different content producers.
4. Work collaboratively. Platforms have worked together to fight terrorism and childabuse. Slowly, collaboration is also beginning to happen around information disorder,
and we encourage such collaboration, particularly when it involves sharing
information about attempts to amplify content.
5. Highlight contextual details and build visual indicators. We recommend that social
networks and search engines automatically surface contextual information andmetadata that would help users ascertain the truth of a piece of content (for example
automatically showing when a website was registered or running a reverse image
search to see whether an image is old). The blue verification tick is an example of ahelpful visual indicator that exists across platforms. We argue that technology
companies should collaborate to build a consistent set of visual indicators for thesecontextual details. This visual language should be developed in collaboration withcognitive psychologists to ensure efficacy.
6. Eliminate financial incentives. Technology companies as well as advertising networks
more generally must devise ways to prevent purveyors of dis-information from gaining
financially.
7. Crack down on computational amplification. Take stronger and quicker action against
automated accounts used to boost content.

80

8. Adequately moderate non-English content. Social networks need to invest intechnology and staff to monitor mis-, dis- and mal-information in all languages.
9. Pay attention to audio/visual forms of mis- and dis-information. The problematic
term ‘fake news’ has led to an unwarranted fixation on text-based mis- and disinformation. However, our research suggests that fabricated, manipulated or falselycontextualized visuals are more pervasive than textual falsehoods. We also expect
fabricated audio to become an increasing problem. Technology companies must
address these formats as well as text.
10. Provide metadata to trusted partners. The practice of stripping metadata fromimages and video, (for example location information, capture date and timestamps),
although protective of privacy and conservative of data, often complicates verification.
Thus, we recommend that trusted partners be provided increased access to such
metadata.
11. Build fact-checking and verification tools. We recommend that technology companies
build tools to support the public in fact-checking and verifying rumors and visualcontent, especially on mobile phones.
12. Build ‘authenticity engines’. As audio-visual fabrications become more sophisticated,
we need the search engines to build out ‘authenticity’ engines and water-marking
technologies to provide mechanisms for original material to be surfaced and trusted.
13. Work on solutions specifically aimed at minimising the impact of filter bubbles:a) Let users customize feed and search algorithms. Users should be given the
chance to consciously change the algorithms that populate their social feeds
and search results. For example, they should be able to choose to see diverse
political content or a greater amount of international content in their social
feeds.
b) Diversify exposure to different people and views. Using the existingalgorithmic technology on the social networks that provides suggestions forpages, accounts, or topics to follow, these should be designed to provide
exposure to different types of content and people. There should be a clear
indication that this is being surfaced deliberately, and while the views orcontent might be uncomfortable or challenging, it is necessary to have anawareness of different perspectives.
c) Allow users to consume information privately. To minimize performative
influences on information consumption, we recommend that technology
companies provide more options for users to consume content privately,
instead of publicizing everything they ‘like’ or ‘follow.
d) Change the terminology used by the social networks. Three common conceptsof the social platforms unconsciously affect how we avoid different views andremain in our echo chambers. ‘To follow’, for most people subconsciously
implies a kind of agreement, so it emotionally creates a resistance against81

exposure to diverse opinion. ‘Friend’ also connotes a type of bond you
wouldn't want to have with those you strongly disagree with but are curious
about. So is the case with ‘like’, when you want to start reading a certain
publication on Facebook. We should instead institute neutral labels such as
connecting to someone, subscribing to a publication, bookmarking a story, etc.

What could national governments do?
1. Commission research to map information disorder. National governments shouldcommission research studies to examine information disorder within their respective
countries, using the conceptual map provided in this report. What types of
information disorder are most common? Which platforms are the primary vehicles fordissemination? What research has been carried out that examines audience responses
to this type of content in specific countries? The methodology should be consistentacross these research studies exercises, so that different countries can be accurately
compared.
2. Regulate ad networks. While the platforms are taking steps to prevent fabricated‘news’ sites from making money, other networks are stepping in to fill the gap. Statesshould draft regulations to prevent any advertising from appearing on these sites.
3. Require transparency around Facebook ads. There is currently no oversight in termsof who purchases ads on Facebook, what ads they purchase and which users aretargeted. National governments should demand transparency about these ads so thatad purchasers and Facebook can be held accountable.
4. Support public service media organisations and local news outlets. The financial
strains placed on news organisations in recent years has led to ‘news deserts’ incertain areas. If we are serious about reducing the impact of information disorder,
supporting quality journalism initiatives at the local, regional and national level needs
to be a priority.
5. Roll out advanced cyber-security training. Many government institutions use bespoke
computer systems that are incredibly easy to hack, enabling the theft of data and thegeneration of mal-information. Training should be available at all levels of government
to ensure everyone understands digital security best practices and to prevent
attempts at hacking and phishing.
6. Enforce minimum levels of public service news on to the platforms. Encourage
platforms to work with independent public media organisations to integrate quality
news and analysis into users’ feeds.

What could media organisations do?
1. Collaborate. It makes little sense to have journalists at different news organisations
fact-checking the same claims or debunking the same visual content. When it comesto debunking mis- or dis-information, there should be no ‘scoop’ or ‘exclusive’. Thus,
82

2.
3.

4.

5.

6.

7.

8.

we argue that newsrooms and fact-checking organisations should collaborate toprevent duplications of effort and free journalists to focus on other investigations.
Agree policies on strategic silence. News organisations should work on best practices
for avoiding being manipulated by those who want to amplify mal- or dis-information.
Ensure strong ethical standards across all media. News organizations have been
known to sensationalize headlines on Facebook in ways that wouldn’t be accepted on
their own websites. News organizations should enforce the same content standards,
irrespective of where their content is placed.
Debunk sources as well as content. News organisations are getting better at factchecking and debunking rumours and visual content, but they must also learn to track
the sources behind a piece of content in real time. When content is being pushed out
by bot networks, or loose organised groups of people with an agenda, news
organisations should identifying this as quickly as possible. This will require journalists
to have computer programming expertise.
Produce more segments and features about critical information consumption. The
news media should produce more segments and features which teach audiences how
to be critical of content they consume. When they write debunks, they should explain
to the audience how the process of verification was undertaken.
Tell stories about the scale and threat posed by information disorder. News andmedia organisations have a responsibility to educate audiences about the scale ofinformation pollution worldwide, and the implications society faces because of it, in
terms of undermining trust in institutions, threatening democratic principles,
inflaming divisions based on nationalism, religion, ethnicity, race, class, sexuality or
gender.
Focus on improving the quality of headlines. User behaviour shows the patterns bywhich people skim headlines via social networks without clicking through to the wholearticle. It therefore places greater responsibility on news outlets to write headlines
with care. Research251 using natural language processing techniques are starting toautomatically assess whether headlines are overstating the evidence available in the
text of the article. This might prevent some of the more irresponsible headlines fromappearing.
Don’t disseminate fabricated content. News organisations need to improve standards
around publishing and broadcasting information and content sourced from the socialweb. There is also a responsibility to ensure appropriate use of headlines, visuals,
captions and statistics in news output. Clickbait headlines, the misleading use ofstatistics, unattributed quotes are all adding to the polluted information ecosystem.

251 Chesney, S., M. Liakata, M. Poesio and M. Purver (2017) Incongruent Headlines: Yet Another Way to

Mislead Your Readers, http://www.eecs.qmul.ac.uk/~mpurver/papers/chesney-et-al17nlpj.pdf

83What could civil society do?
1. Educate the public about the threat of information disorder. There is a need toeducate people about the persuasive techniques that are used by those spreading disand mal-information, as well as a need to educate people about the risks of
information disorder to society, i.e., sowing distrust in official sources and dividing
political parties, religions, races and classes.
2. Act as honest brokers. Non-profits and independent groups can act as honest brokers,
bringing together different players in the fight against information disorder, including
technology companies, newsrooms, research institutes, policy-makers, politicians andgovernments.

What could education ministries do?
1. Work internationally to create a standardized news literacy curriculum. Such acurriculum should be for all ages, based on best practices, and focus on adaptable
research skills, critical assessment of information sources, the influence of emotion on
critical thinking and the inner workings and implications of algorithms and artificial
intelligence.
2. Work with libraries. Libraries are one of the few institutions where trust has notdeclined, and for people no longer in full time education, they are a critical resource
for teaching the skills required for navigating the digital ecosystem. We must ensurecommunities can access both online and offline news and digital literacy materials via
their local libraries.
3. Update journalism school curricula. Ensure journalism schools teach computational
monitoring and forensic verification techniques for finding and authenticating content
circulating on the social web, as well as best practices for reporting on information
disorder.

What could Grant-Making Foundations do?
1. Provide support for testing solutions. In this rush for solutions, it is tempting tosupport initiatives that ‘seem’ appropriate. We need to ensure there is sufficientmoney to support the testing of any solutions. For example, with news literacy
projects, we need to ensure money is being spent to assess what types of materials
and teaching methodology are having the most impact. It is vital that academics areconnecting with practitioners working in many different industries as solutions aredesigned and tested. Rather than small grants to multiple stakeholders, we need
fewer, bigger grants for ambitious multi-partner, international research groups andinitiatives.
2. Support technological solutions. While the technology companies should be required
to build out a number of solutions themselves, providing funding for smaller startups
to design, test and innovate in this space is crucial. Many solutions need to be rolled84

out across the social platforms and search engines. These should not be developed as
proprietary technology.
3. Support programs teaching people critical research and information skills. We must
provide financial support for journalistic initiatives which attempt to help audiences
navigate their information ecosystems, such as public-service media, local news media
and educators teaching fact-checking and verification skills.

85

Appendix: European Fact-checking and Debunking
Initiatives
According to a study published in 2016 by Lucas Graves and Federica Cherubini, there are 34
permanent fact checking operations that exist across 20 European countries. There are twodifferent types: those attached to news organisations (around 40%) and those that operate as
nonprofits (around 60%). 252
Much of the following information is drawn from the fact-checking database created by Duke
University’s Reporter’s Lab.
Austria: Fakt ist Fakt, an independent fact checking organisation. They examine the truths ofpublic figures, especially politicians.
Bosnia: Istinomjer, a project of Bosnia’s Zašto ne? (Why Not?), a peace-building group,
founded in 2001. Its goal is to make political and public discourse in Bosnia & Herzegovina“more relevant to the promotion of political accountability as the fundamental principle ofdemocracy.” Zašto ne? has been financially supported by the National Endowment for
Democracy (NED) since its inception.
Croatia: Faktograf rates political claims on a scale from “Fact” to “Not even the F of Fact” andkeeps a databse that supports researchers’ efforts in collecting information on the degree towhich promises made by public officials are fulfilled.. Their rating system emulates the one
popularized by PolitiFact’s Truth-O-Meter and was adopted with modifications by the mostexternal fact-checkers worldwide. Zašto ne?, which is behind the Bosnian Istinomjer, did theprogramming and designing for Faktograf’s website with support from the National
Endowment for Democracy and TechSoup.
Czech Republic: Demagog was created in February 2012 and is linked to the Slovak version of
the project, Demagog.sk. It originated as a voluntary and independent initiative of Matej
Hruška and Ondrej Lunter, students of Masaryk University in Brno.
Denmark:❏ Detektor is a fact-checking show produced by Denmark’s public broadcaster, DR.
❏ TjekDet is the fact-checking vertical of Mandag Morgen, a Danish weekly business andpolitical magazine.
Finland: Faktabaari is a Finnish site launched in 2014 by an NGO called the Open Society
Association. It is managed by a voluntary staff of professional journalists, EU experts andtechnical staff with the help of a broader network of topical experts.
France:❏ Libération’s Désintox, which was launched in 2008, fact-checks politicians and debunks
outright rumors and fake stories.
❏ Le Vrai du Faux, from the radio and television network franceinfo, is a news site that252 Graves, L.& Cherubini, F. (2016). The rise of fact-checking sites in Europe. Reuters Institute for the Study ofJournalism. http://reutersinstitute.politics.ox.ac.uk/publication/rise-fact-checking-sites-europe

86sifts through mis-information circulating on websites and social networks. It also has a
programme that airs daily on franceinfo.
❏ Les Observateurs, an online hub and television programme from the multilingual
network France24, has a collaborative site in four languages (French, English, Arabic
and Persian) and a TV show on France 24. They cover international current affairs byusing eyewitness accounts from people who were at the heart of events. Eyewitnesses
send photos and videos, which Les Observateurs’ team of professional journalists in
Paris verify and contextualize.
❏ Les Décodeurs, Le Monde’s dedicated fact checking unit, verifies declarations,assertions and rumours of all kinds. It has a ten-point charter, which shapes theirwork. They also built Decodex, a browser extension that helps fight false information
by informing readers when they see an article that comes from a site that frequently
posts fabricated or misleading content.
❏ LuiPresident.fr is a financially and politically independent website created in 2012 by
three students from the Ecole Supérieure de Journalisme de Lille (ESJ-Lille) to verify
Francois Hollande's campaign commitments during his term in office. In 2017,
LuiPresident was transformed into a project of the ESJ-Lille to follow EmmanuelMacron's promises during the five-year period of his presidency 2017-2022. It is run by
students under the supervision of professional journalists.
❏ CrossCheck253 was a collaborative news verification project led by FirstDraft andfunded by Google News. It involved 37 different partners, including newsrooms andtechnology companies.
Georgia: FactCheck Georgia is a political news-and-information project established by
Georgia’s Reforms Associates (GRASS). Modelled on successful international political newsand-information watchdog services, it aims to rate the factual accuracy of statements madeby Members of Parliament (MPs), the Prime Minister, the President and the Government
economic team. The service is offered in a fully bilingual Georgian-English format. FactCheckis implemented with the support of the Embassy of the Kingdom of the Netherlands in Tbilisi,
the German Marshall Fund, the European Endowment for Democracy and the US Embassy inGeorgia.
Germany:❏ Fakt oder Fake is a vertical on Zeit Online.
❏ Faktenfinder is a collaboration between regional members of the German publicbroadcasting consortium ARD, who contribute multimedia fact checks and explainers
to investigate widely spread claims and rumours from across the country and world.
The project is an initiative of tagesschau24, the digital TV news channel produced for
ARD by one of its regional partners, Norddeutscher Rundfunk (NDR) in Hamburg.
❏ Correctiv is an independent fact-checking site. It was launched to focus on political
253 https://crosscheck.firstdraftnews.com/france-en/87claims and viral mis-information with financial support from the Open Society
Foundation. It was Facebook’s German partner for its third party fact-checking
initiative. It also partnered with First Draft to monitor dis-information around the 2017
German Federal election.
Ireland: theJournal.ie, a commercial digital news outlet, started a fact-checking project during
the county’s February 2016 general election and continued after the result. It often focuses
on claims flagged by its readers.
Italy:❏ Pagella Politica is a website that fact-checks claims by politicians. It also produces factchecks for Virus, a public affairs programme on RAI.
❏ Agi Fact-Checking, a service of the Italian news service Agi, distributes reports by
Pagella Politica.
Kosovo: Kryptometer is a fact-checking vertical on Kallxo.com and produced for the video
program "Jeta në Kosovë”. Both initiatives are projects of the Balkans Investigative Reporting
Network (Rrjeti Ballkanik i Gazetarisë Hulumtuese) and Internews Kosova, a nongovernmental organization that supports regional media projects and training. Kallxo is an
online accountability platform for Kosovo citizens that was funded by the United Nations
Development Programme.
Latvia: Melu Detektors is hosted by LSM.lv, the digital news portal for Latvijas Sabiedriskie
Mediji (Latvian Public Media), which includes TV and radio channels. The project operates on
its own using a combination of government funding and advertising. It began as a partnershipwith Re:Baltica, a non-profit investigative news outlet supported by the Open Society Institutein Latvia, and launched with support from the U.S. Baltic Foundation and the U.S. State
Department.
Lithuania: Patikrinta is a branch of the established Lithuanian news website ‘15 min’ that
focuses on statements by Lithuanian politicians, often based on recommendations fromreaders.
Macedonia: Vistinomer is a fact-checking website run by the Macedonian NGO
Metamorphosis. It is connected to Why not? from Bosnia and Herzegovina and LINnet from
Serbia.
Norway: Faktisk is a new fact-checking collaboration between the four biggest news
organizations in Norway.
Poland:❏ Demogog is a project of the Demogog Association, the first professional fact-checking
organization in Poland.
❏ OKO.press fact-checks statements made by Polish public figures. It was founded in2016 and is supported entirely by individual donations.
Portugal: Fact Checks do Observador is a reporting project of El Observador, a free, adsupported online newspaper based in Lisbon.
Romania: Factual is run by volunteer contributors and a team at Funky Citizens, an88accountability and budget transparency organization. It is funded by voluntary contributions
and a grant from the Alumni Grants Programme of the Open Society Institute.
Serbia: Istinomer was established in 2009 by the Center for Research, Transparency andAccountability.
Slovakia: Demagog was founded in 2010 by a pair of political science students at Masaryk
University in Brno, and quickly spread to sister sites in the Czech Republic and Poland.
Spain❏ El Objetivo con Ana Pastor is a highly rated weekly public affairs programme on the
Spanish television network La Sexta, which goes out to between 1.5 and 2 million
viewers each Sunday.
❏ Maldito Bulo is an online-only fact-checking initiative linked to El Objetivo.
❏ La Chistera is a blog published by the data journalism unit at El Confidencial, acommercial digital news service based in Madrid and operated by Titania Compañía
Editorial SL.
Sweden: Viralgranskaren is the fact-checking project of the Swedish newspaper Metro.
Switzerland:❏ Swissinfo.ch is the fact-checking initiative of the Swiss Broadcasting Corporation.
Reports are posted in multiple languages and appear most frequently during thecountry's voting periods.
❏ Tages Anzeiger Faktenchecks is a project of the Swiss German-language daily
newspaper Tages Anzeiger. The frequency of its fact-checks increases during periodsof public campaigning and political debate.
Turkey:❏ Do ruluk Payı (or ‘share of truth’) is a fact-checking initiative established by the
Dialogue for Common Future Association. It is funded by the National Endowment for
Democracy.
❏ Teyit is a verification and debunking service that monitors news reports and social
media for mis- and dis-information. The initiative is a non-profit, social enterprise
based in Ankara and is supported by the European Endowment for Democracy.
Ukraine:❏ StopFake was founded by students and faculty of the Kyiv Mohyla School of
Journalism. The site is now in 11 different languages.
❏ Slovo i Dilo was created by a non-governmental organization called the People's
Control System to track the political promises of Ukrainian officials at the national andlocal level.
❏ VoxCheck is a branch of VoxUkraine, which does research-based policy analysis and isfunded in part by the National Endowment for Democracy. It fact-checks Ukrainianpoliticians and has utilized crowd-funding for much of its support.
❏ FactCheck Ukraine is an independent fact-checking initiative that examines claims by

89Ukrainian politicians and public figures is funded by individual donations and nongovernmental organizations.
United Kingdom:❏ Full Fact is the UK’s largest independent, non-partisan fact-checking organization. As
well as publishing fact-checks, the organisation actively pushes for corrections where
necessary and works with government departments and research institutions toimprove the quality and communication of information at source. It is supported byindividuals, charitable trusts and foundation support. They recently received moneyfrom the Omidyar Network to build on their work on automated fact-checking. They
partnered with First Draft to monitor mis-information during the 2017 UK snap
election.
❏ FactCheck is a fact-checking feature from Channel 4 News, which appears on theirwebsite.
❏ Reality Check is the BBC’s dedicated fact-checking project. It was introduced in 2015 tocover the Brexit referendum and then re-started in the autumn of 2016. Fact-checkers
for the BBC’s Reality Check appear on high-reach outlets, including the BBC News
Channel, BBC World Television, Radio 5 Live, Radio 4 and the BBC World Service.
❏ The Ferret is an investigative news organisation based in Scotland. In the Spring of
2017, they launched Fact Service, Scotland’s first non-partisan fact-checking service.
They check statements from politicians, pundits and prominent public figures. Fact
checks are also undertaken on some viral claims, hoaxes and memes.
❏ FactCheckNI is an independent fact-checking organisation. They provide tools,
information and advice to citizens so they can undertake their own fact-checks on
information they hear from politicians and and the media.

90References
Alba, D. (2016) Meet the Ad Companies Ditching Breitbart and Fake News, Wired, December,
15, 2016. Available at: https://www.wired.com/2016/12/fake-news-will-go-away-tech-behindads-wont-pay/
Allcott, H., & Gentzkow, M. (2017). Social Media and Fake News in the 2016 Election.
National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w23089
Political Behavior
Bakamo (2017a) The Role and Impact of Non-Traditional Publishers in the 2017 French
Presidential Election. Available at: https://www.bakamosocial.com/frenchelection/
Bakamo (2017b) Patterns of Dis-information in the 2017 French Presidential Election.
Available at: Available at: https://www.bakamosocial.com/frenchelection/
Barber, B. (1998) ‘Which Technology and Which Democracy?’, Talk given at Democracy andDigital Media Conference, http://web.mit.edu/m-i-t/articles/barber.html
Bell, E. (2016) Facebook drains the fake news swamp with new, experimental partnerships,
Columbia Journalism Review, December 15, 2016. Available at:https://www.cjr.org/tow_center/facebook_drains_fake_news_swamp_new_experimental_partn
erships.php
Benedictus, L., “Invasion of the troll armies: from Russian Trump supporters to Turkish statestooges,” The Guardian. https://www.theguardian.com/media/2016/nov/06/troll-armies-socialmedia-trump-russian
Benkler Y. et al (March 3, 2017) Study: Breitbart-led right-wing media ecosystem alteredbroader media agenda. Columbia Journalism Review. https://www.cjr.org/analysis/breitbartmedia-trump-harvard-study.php
Beran, Dale. “4chan: The Skeleton Key to the Rise of Trump.” Medium, February 14, 2017.
https://medium.com/@DaleBeran/4chan-the-skeleton-key-to-the-rise-of-trump-624e7cb798cb
Bernstein, J. (April 5, 2017) Never Mind The Russians, Meet The Bot King Who Helps Trump
Win Twitter, Buzzfeed News, https://www.buzzfeed.com/josephbernstein/from-utah-with-love/
Bessi, A. (2015) Science vs Conspiracy: Collective Narratives in the Age of Mis-information.
PLoSOne, 10, (2)Bharat, K. (April 27, 2017) How to Detect Fake News in Real-Time, NewsCo on Medium,https://shift.newco.co/how-to-detect-fake-news-in-real-time-9fdae0197bfd

91

Bilton, N. (Jan. 26, 2017) Fake News is About to Get Scarier than You’d Ever Dreamed.
Vanity Fair. http://www.vanityfair.com/news/2017/01/fake-news-technology
Bilton, R. (Feb.2, 2017) Reddit’s /r/worldnews community used a series of nudges to pushusers to fact-check suspicious news, Nieman Lab, http://www.niemanlab.org/2017/02/redditsrworldnews-community-used-a-series-of-nudges-to-push-users-to-fact-check-suspicious-news/
Birdsell, D. S., & Groarke, L. (1996). Toward a Theory of Visual Argument. Argumentation
and Advocacy, 33(1), 1-10.
Born, K. (2017) The future of truth: Can philanthropy help mitigate mis-information? Hewlett
Foundation. Available at: http://www.hewlett.org/future-truth-can-philanthropy-help-mitigatemis-information/
Bordia, P. (2013). Rumor Clustering, Consensus, and Polarization: Dynamic Social Impact and
Self-Organization of Hearsay. Journal of Experimental Social Psychology, 49(3), 378-399.
boyd, d.. (2017) “Did Media Literacy Backfire?” Data & Society: Points, January 5, 2017.
https://points.datasociety.net/did-media-literacy-backfire-7418c084d88d
boyd, d. (March 27,2017) “Google and Facebook can’t just make fake news disappear”,
Backchannel, https://medium.com/backchannel/google-and-facebook-cant-just-make-fakenews-disappear-48f4b4e5fbe8
Bond, D. and Robinson, D. (Jan. 29, 2017), ‘European Commission fires warning at Facebook
over fake news’ Financial Times, https://www.ft.com/content/85683e08-e4a9-11e6-9645c9357a75844a?mhq5j=e2
Borel, B. (2017) Fact Checking Won’t Save Us From Fake News, FiveThirtyEight, January 4,
2017. Available at: https://fivethirtyeight.com/features/fact-checking-wont-save-us-from-fakenews/
Borel, B, (2017) Hpw to Talk to your Facebook Friends about Fake News, Open Notebook
http://www.theopennotebook.com/2017/02/21/how-to-talk-to-your-facebook-friends-aboutfake-news/
Boyd, L. & Vraga, E (2015) In Related News, That Was Wrong: The Correction of Misinformation Through Related Stories Functionality in Social Media, Journal ofCommunication, 65 (4): 619-638.
Bradshaw, S. and P. Howard (August 2017) Troops, Trolls and Troublemakers: A Global
Inventory of Organized Social Media Manipulation92http://comprop.oii.ox.ac.uk/wp-content/uploads/sites/89/2017/07/Troops-Trolls-andTroublemakers.pdf
Broderick, R. (2017) Trump Supporters Online Are Pretending To Be French To Manipulate
France’s Election, Buzzfeed News, January 24, 2017. Available at:https://www.buzzfeed.com/ryanhatesthis/inside-the-private-chat-rooms-trump-supporters-areusing-to?utm_term=.aaMXl5lvgG#.crwErDrpYj
Brown, C. (April 2, 2017) Introducing the News Integrity Initiative, Facebook Media Blog,
https://media.fb.com/2017/04/02/introducing-the-news-integrity-initiative/
Bucay, Y., V. Elliott, J. Kamin, A. Park, America’s Growing News Deserts, Columbia
Journalism Review, https://www.cjr.org/local_news/american-news-deserts-donuts-local.php
Caplan, Robyn. “How Do You Deal with a Problem Like ‘Fake News?’” Data & Society:
Points, January 5, 2017. https://points.datasociety.net/how-do-you-deal-with-a-problem-likefake-news-80f9987988a9
Carey, J. (1989) Communication as Culture: Essays on Media and Society, London:
Routledge.
Caulfield, M. (Dec 7. 2016) Announcing the Digital Polarization Initiative. Hapgood.
https://hapgood.us/2016/12/07/announcing-the-digital-polarization-initiative-an-openpedagogy-joint/
Cellan-Jones, R. (Sept 22, 2017) Fake news worries 'are growing' suggests BBC poll, BBC
News, http://www.bbc.com/news/technology-41319683
Chan, M.S., C. R.Jones, K.H. Jamieson, D. Albarracín (2017) Debunking: A Meta-Analysis of
the Psychological Efficacy of Messages Countering Mis-information, Psychological Science,
1-16.
Chesney, S., M. Liakata, M. Poesio and M. Purver (2017) Incongruent Headlines: Yet Another
Way to Mislead Your Readers, http://www.eecs.qmul.ac.uk/~mpurver/papers/chesney-etal17nlpj.pdf
Chu, Z., S. Gianvecchio, H. Wang, S. Jajodia, (Nov/Dec 2012) Detecting Automation of
Twitter Accounts: Are You a Human, Bot, or Cyborg? IEEE Transactions on Dependable andSecure Computing, Vol. 9, No. 6
Collins, B., K. Poulsen & S. Ackerman (September 11, 2017) Exclusive: Russia Used
Facebook Events to Organize Anti-Immigrant Rallies on U.S. Soil, The Daily Beast,
http://www.thedailybeast.com/exclusive-russia-used-facebook-events-to-organize-antiimmigrant-rallies-on-us-soil
93

Constine, J. (2017) Facebook shows Related Articles and Fact Checkers Before you Open
Links, TechCrunch, https://techcrunch.com/2017/04/25/facebook-shows-related-articles-andfact-checkers-before-you-open-links/
Crowell, C., (June 14, 2017) Our Approach to Bots and Automation, Twitter Blog.
https://blog.twitter.com/official/en_us/topics/company/2017/Our-Approach-Bots-Misinformation.html
d'Ancona, M. (2017). Post-Truth: The New War on Truth and How to Fight Back. London:
Ebury Press.
de Benedictis-Kessner, J., Baum, M.A., Berinsky, A.J., & Yamamoto, T. “Persuasion in Hard
Places: Accounting for Selective Exposure When Estimating the Persuasive Effects of Partisan
Media.” Unpublished Manuscript, Harvard University and MIT.
De Feyter, S. (2015). ‘They are like Crocodiles under Water’: Rumour in a Slum Upgrading
Project in Nairobi, Kenya. Journal of Eastern African Studies, 9(2), 289-306
De Seta, G., (2017) “Trolling, and Other Problematic Social Media Practices,” in The SAGE
Handbook of Social Media, ed. Jean Burgess, Alice E. Marwick, and Thomas Poell. Thousand
Oaks, CA: SAGE Publications.
Derakhshan H. (July 14, 2015) The Web We Have to Save. Matter. Available at:https://medium.com/matter/the-web-we-have-to-save-2eb1fe15a426
Derakhshan H. (Nov 29, 2016) Social Media Is Killing Discourse Because It’s Too Much Like
TV, MIT Technology Review. https://www.technologyreview.com/s/602981/social-media-iskilling-discourse-because-its-too-much-like-tv/
Dias, N. (June 22, 2017), Reporting on a new age of digital astroturfing, First Draft News,https://firstdraftnews.com/digital-astroturfing/Dias, N. (August 17, 2017) The Era of Whatsapp Propaganda is Upon Us, Foreign Policy,
http://foreignpolicy.com/2017/08/17/the-era-of-whatsapp-propaganda-is-upon-us/
DiFonzo, N., Beckstead, J. W., Stupak, N., & Walders, K. (2016). Validity judgments ofrumors heard multiple times: the shape of the truth effect. Social Influence, 11(1), 22-39.
DiFonzo, N., Bourgeois, M. J., Suls, J., Homan, C., Stupak, N., Brooks, B. P., ... &Doctor, K., (May 2, 2017) Can a Master Blacklist Choke Off Fake News, Nieman Lab.
http://www.niemanlab.org/2017/05/newsonomics-can-a-master-blacklist-choke-off-fake-newsmoney-supply/

94

Rachel Donadio, (May 8, 2017) Why the Macron Hacking Attack Landed with a Thud inFrance, The New York Times, https://www.nytimes.com/2017/05/08/world/europe/macronhacking-attack-france.html
Dutton, B. (May 2017) Fake News, Echo chambers and Filter Bubbles are an Exaggerated
Threat. Here's Why. World Economic Forum Blog.
https://www.weforum.org/agenda/2017/05/fake-news-echo-chambers-and-filter-bubbles-arean-exaggerated-threat-heres-why
Dutton, William H., et al. (2017) "Search and Politics: A Cross-National Survey." QuelloCenter Working Paper No. 2944191 Available at:https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2944191
Eddy, M. and M. Scott (June 30 2017), Delete Hate Speech or Pay Up, Germany Tells Social
Media Companies, New York Times, https://www.nytimes.com/2017/06/30/business/germanyfacebook-google-twitter.html
El-Bermawy, M. (Nov. 18, 2016) Your Filter Bubble is Destroying Democracy, Wired.
https://www.wired.com/2016/11/filter-bubble-destroying-democracy/
Eliseev, A (July 20, 2017) The Gupta scandal: how a British PR firm came unstuck in South
Africa, The New Statesman,
http://www.newstatesman.com/culture/observations/2017/07/gupta-scandal-how-british-prfirm-came-unstuck-south-africa
EU East StratCom Task Force, Means, Goals and Consequences of the Pro-Kremlin Disinformation Campaign, Italian Institute for International Political Studies (ISPI), January 19,
2017. Available at: https://euvsdisinfo.eu/commentary-means-goals-and-consequences-of-thepro-kremlin-dis-information-campaign/
Evaluating Information: The Cornerstone of Civic Online Reasoning. Stanford History
Education Group. 2016. Available at:https://sheg.stanford.edu/upload/V3LessonPlans/Executive%20Summary%2011.21.16.pdf
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016). The Rise of Social Bots.
Communications of the ACM, 59(7), 96-104.
Filloux, F. (August 21, 2017) More than 600 global brands still feed the fake news ecosystem,
The Monday Note on Medium, https://mondaynote.com/more-than-600-global-brands-stillfeed-the-fake-news-ecosystem-d1ddfbd80458

95

Filloux, F. (August 6, 2017) You can’t sell news for what it costs to make, The Walkley
Magazine on Medium, https://medium.com/the-walkley-magazine/you-cant-sell-news-forwhat-it-costs-to-make-7a4def964ffa
Filloux, F. (June 25, 2017) The News Quality Scoring Project, Medium,https://mondaynote.com/the-news-quality-scoring-project-surfacing-great-journalism-from-theweb-48401ded8b53
Flynn, D.J. & Nyhan, Brendan & Reifler, Jason. (2017). ‘The Nature and Origins ofMisperceptions: Understanding False and Unsupported Beliefs About Politics’, Political
Psychology. 38: 127-150
Gillespie, T. (in press) Governance of and by platforms. In Burgess, J., Poell, T., & Marwick,
A. (Eds), SAGE Handbook of Social Media. Retrieved from http://culturedigitally.org/wpcontent/uploads/2016/06/Gillespie-Governance-ofby-Platforms-PREPRINT.pdf
Goffman, E, (1956) The Presentation of Self in Everyday Life. Random House.
Goodman, E. (2017) Write up from the Westminster Media Forum Keynote Seminar: Fakenews - scope, public trust and options for policy, LSE,
http://blogs.lse.ac.uk/mediapolicyproject/2017/08/10/the-evolving-conversation-around-fakenews-and-potential-solutions/
Granovetter, M.S. (1973) The Strength of Weak Ties, Journal of Sociology, 78(6):1360-1380
Graves, L.& Cherubini, F. (2016). The Rise of Fact-Checking Sites in Europe. Available at:
http://reutersinstitute.politics.ox.ac.uk/publication/rise-fact-checking-sites-europe
Greenhill, K. M. (forthcoming). Whispers of War, Mongers of Fear: Extra-factual Sources ofThreat Conception and Proliferation
Greenhill, K. M., & Oppenheim B. (forthcoming). Rumor Has It: The Adoption of Unverified
Information in Conflict Zones. International Studies Quarterly.
Gu, L., Kropotov, V. & Yarochkin, F. (2017) The Fake News Machine: How Propagandists
Abuse the Internet and Manipulate the Public. Oxford University. Available at:https://documents.trendmicro.com/assets/white_papers/wp-fake-news-machine-howpropagandists-abuse-the-internet.pdf
Habermas, J. (1962) (1989 translation, The Structural Transformation of the Public Sphere: An
Inquiry into a Category of Bourgeois Society. MIT Press.
Haigh, M., Haigh, T., & Kozak, N. I. (2017). Stopping Fake News: The work practices of peerto-peer counter propaganda. Journalism Studies, 1-26.
96

Hall, S. (1973). Encoding and Decoding in the Television Discourse. Birmingham: Centre forContemporary Cultural Studies
Hansen, F. S. (August 2017) Russian Hybrid Warfare: A Study of Dis-information, Danish
Institute for International study. http://pure.diis.dk/ws/files/950041/DIIS_RP_2017_6_web.pdf
Harrington, M (Jan. 16, 2017) Survey: People’s Trust Has Declined in Business, Media,
Government, and NGOs, Harvard Business Review, https://hbr.org/2017/01/survey-peoplestrust-has-declined-in-business-media-government-and-ngos
Helberger, N. et al. (Sept 22. 2017) Dear Mark: An Open Letter to Mark Zuckerberg in
Response to his Statement on Political Advertising on Facebook, Available at:https://www.dropbox.com/s/7v3vpk9yw5sa18b/Dear%20Mark_final..pdf?dl=0
Hendrix, J. and Carroll, D. (2017) Confronting a Nightmare for Democracy. Available at:https://medium.com/@profcarroll/confronting-a-nightmare-for-democracy-5333181ca675
Higgins, A., McIntire, M., & Dance, G. J. (2016). Inside a Fake News Sausage Factory: ‘This
is all about income’. New York Times, November 25, 2016. Available at:https://www.nytimes.com/2016/11/25/world/europe/fake-news-donald-trump-hillary-clintongeorgia.html
Hofileña, C. F. (Oct. 9, 2016) Fake accounts, manufactured reality on social media, Rappler,
https://www.rappler.com/newsbreak/investigative/148347-fake-accounts-manufactured-realitysocial-media
Iyengar S. and S. J. Westwood (2015) Fear and Loathing across Party Lines: New Evidence onGroup Polarization, American Journal of Political ScienceVol. 59, No. 3 (July 2015), pp. 690-707
Jack, C. (2017) Lexicon of Lies, Data & Society,https://datasociety.net/pubs/oh/DataAndSociety_LexiconofLies.pdf
Jackson, J. (Jan. 12, 2017) BBC sets up team to debunk fake news, The Guardian,
https://www.theguardian.com/media/2017/jan/12/bbc-sets-up-team-to-debunk-fake-news
Jankowicz, N. (Sept. 25, 2017) The Only Way to Defend Against Russia’s Information War,The New York Times, https://mobile.nytimes.com/2017/09/25/opinion/the-only-way-to-defendagainst-russias-information-war.html
Jarvis, J. (2017) Our problem isn’t ‘fake news.’ Our problems are trust and manipulation.
Whither News. Available at: https://medium.com/whither-news/our-problem-isnt-fake-newsour-problems-are-trust-and-manipulation-5bfbcd716440\
97

Jolley, D., & Douglas, K. M. (2014) “The Social Consequences of Conspiracism: Exposure toConspiracy Theories Decreases the Intention to Engage in Politics and to Reduce One’s
Carbon Footprint.” British Journal of Psychology, 105 (1): 35–56.
Kahan, D. et al (2013) Motivated Numeracy and Enlightened Self-Government, Behavioural
Public Policy, 1, 54-86
Kahan, D. (2011) What is Motivated Reasoning and How Does It Work? Science and Religion
Today, http://www.scienceandreligiontoday.com/2011/05/04/what-is-motivated-reasoning-andhow-does-it-work/
Karlova, N. A., & Fisher, K. E. (2013). Plz RT: a Social Diffusion model of Mis-information
and Dis-information for Understanding Human Information Behaviour. Information Research,
18(1), 1-17.
King, G, J. Pan & M. Roberts, (May 2016) How the Chinese Government Fabricates Social
Media Posts for Strategic Distraction, not Engaged Argument, Harvard University,
http://gking.harvard.edu/files/gking/files/50c.pdf?m=1463587807
Kington, T. (March 30, 2010) Twitter hoaxer comes clean and says: I did it to expose weak
media, The Guardian, https://www.theguardian.com/technology/2012/mar/30/twitter-hoaxertommaso-de-benedetti
Knight Commission on the Information Needs of Communities in a Democracy, (2009)
Informing Communities: Sustaining Democracy in the Digital Age, The Aspen Institute,
https://production.aspeninstitute.org/publications/informing-communities-sustainingdemocracy-digital-age/
Krasodomski-Jones, A. (2016) Talking to Ourselves? Political Debate Online and the Echo
Chamber Effect. Demos. Available at: https://www.demos.co.uk/project/talking-to-ourselves/
Lakoff, G. (2010) “Why "Rational Reason" Doesn't Work in Contemporary Politics”, Truth
Out. http://www.truth-out.org/buzzflash/commentary/george-lakoff-why-rational-reasondoesnt-work-in-contemporary-politics/8893-george-lakoff-why-rational-reason-doesnt-workin-contemporary-politics
Lakoff, George (1996) Moral Politics: What Conservatives Know That Liberals Don't.
University of Chicago Press.
Lakoff, George (1997). Metaphors We Live By. University of Chicago Press
Lazer, D., Baum. M, Grinberg, N., Friedland, L., Joseph, K., Hobbs, W. and Mattsson, C.

98

(2017) Combating Fake News: An Agenda for Research and Action. Harvard. Available at:https://shorensteincenter.org/combating-fake-news-agenda-for-research/
Lev-on, A. and Manin, Bernard, (2009) Happy Accidents: Deliberation and Online Exposure to
Opposing Views, Online Deliberation: Design, Research, and Practice (edited by Todd Davies
and Seeta Peña Gangadharan)
Lewandowsky, S., Ecker, U. K., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misinformation and its correction continued influence and successful debiasing. Psychological
Science in the Public Interest, 13(3), 106-131.
Lewandowsky, S., Stritzke, W. G., Freund, A. M., Oberauer, K., & Krueger, J. I. (2013). Misinformation, dis-information, and violent conflict: From Iraq and the “War on Terror” to future
threats to peace. American Psychologist, 68(7), 487.
Lin, X., Spence, P. R., & Lachlan, K. A. (2016). Social media and credibility indicators: The
Effect of Influence Cues. Computers in Human Behavior, 63, 264-271.
Lindgren, A., J. Corbett & J. Hodson, (Jan. 23, 2017) Canada’s Local News Poverty, Policy
Options, http://policyoptions.irpp.org/magazines/january-2017/canadas-local-news-poverty/
Maffesoli, M. (1996) The Time of the Tribes, London: Sage.
Marwick, A. and Lewis, R (2017) Media Manipulation and Dis-information Online. Data andSociety. Available at: https://datasociety.net/output/media-manipulation-and-disinfo-online/
Mantzarlis, A (Feb. 2, 2016) Fact-checkers experiment with Snapchat, GIFs and other stuff
millennials ♥. Poynter.http://www.poynter.org/2016/fact-checkers-experiment-with-snapchatgifs-and-other-stuff-millennials-%E2%99%A5/393992/
Mantzarlis, A (May 10, 2016) Did Clinton or Trump twist the facts? This messaging bot willtell you. Poynter. http://www.poynter.org/2016/did-clinton-or-trump-twist-the-facts-thismessaging-bot-will-tell-you/409457/
Mantzarlis, A (June 7, 2016) There’s been an explosion of international fact-checkers, but they
face big challenges, Poynter. http://www.poynter.org/2016/theres-been-an-explosion-ofinternational-fact-checkers-but-they-face-big-challenges/415468/
Mantzarlis, A. (July 12, 2017), European policy-makers are not done with Facebook, Google
and fake news just yet, Poynter, https://www.poynter.org/2017/european-policy-makers-arenot-done-with-facebook-google-and-fake-news-just-yet/465809/

99

McGregor, J. (May 31, 2017) Facebook Wades Into Another Election, Forbes.
https://www.forbes.com/sites/jaymcgregor/2017/05/31/facebook-wades-into-anotherelection/#37265336635c
McGrew, S., T. Ortega, J. Breakstone & S. Wineburg, (Fall 2017) The Challenge That’s Bigger
Than Fake News: Teaching Students to Engage in Civic Online Reasoning. American
Educator.
Meeker, M (2017) Internet Trends, 2017. Available at:
http://dq756f9pzlyr3.cloudfront.net/file/Internet+Trends+2017+Report.pdf
Messing, S., & Westwood, S. J. (2014). Selective exposure in the age of social media:
Endorsements trump partisan source affiliation when selecting news online. Communication
Research, 41(8), 1042-1063.
Metzger, M. J., & Flanagin, A. J. (2013). Credibility and trust of information in online
environments: The use of cognitive heuristics. Journal of Pragmatics, 59, 210-220.
Metzger et al. (2010) Social and Heuristic Approaches to Credibility Evaluation Online,
Journal of Communication, 60 (3):413-439
Mina, A.X. (March 2. 2017) Building Technical Standards for Credibility, MisinfoCon on
Medium. https://misinfocon.com/building-technical-standards-for-credibility-59ef9ee4ab73
Moschella, M. and R. Watts, (June 19, 2017) What we Learned Fact-Checking the UK
Election, First Draf Newst, https://firstdraftnews.com/joint-venture-learnings/
Mosseri, A. (2016) News Feed FYI: Addressing Hoaxes and Fake News, Facebook Newsroom,
December 15, 2016.
Mullin, B. (April 6, 2016) Facebook rolls out global warning against fake news, Poynter
http://www.poynter.org/2017/facebook-rolls-out-global-warning-against-fake-news/454951/
Musgrave, S. (August 9, 2017) I Get Called a Russian Bot 50 Times a Day, Politico,
http://www.politico.com/magazine/story/2017/08/09/twitter-trump-train-maga-echo-chamber215470
NATO Strategic Communications Centre of Excellence (September 2017) Robotrolling,
http://www.stratcomcoe.org/robotrolling-20171
Newman, N. (2017) Digital News Report, Reuters Institute for the Study of Journalism,
https://reutersinstitute.politics.ox.ac.uk/sites/default/files/Digital%20News%20Report%202017%20web_0.pdf

100

Nielsen, J. (2003). "IM, Not IP (Information Pollution)". ACM Queue. 1 (8): 75–76.
Nyhan, B. and J. Reifler (2015) ‘Displacing Mis-information about Events: An Experimental
Test of Causal Corrections’, Journal of Experimental Political Science, 2 (1) : 81-93
Nyhan, B., and J. Reifler (2010) “When Corrections Fail: The persistence of politicalmisperceptions.” Political Behavior, 32 (2): 303–330.
Oyeyemi, S. O., Gabarron, E., & Wynn, R. (2014). Ebola, Twitter, and Mis-information: aDangerous Combination?. BMJ, 349.
Pariser, E. 2011. The Filter Bubble: What the Internet Is Hiding from You. London: Penguin.
Pariser, E (2016) & many others, Design Solutions for Fake News, [November 2016 ongoing].
Available at:https://docs.google.com/document/d/1OPghC4ra6QLhaHhW8QvPJRMKGEXT7KaZtG_7s5UQrw/edit
Paul, C. & Matthews, M. (2016) The Russian ‘Firehose of Falsehood’ Propaganda Model:
Why It Might Work and Options to Counter It, Rand Corporation, July 2016. Available at:
http://www.rand.org/content/dam/rand/pubs/perspectives/PE100/PE198/RAND_PE198.pdf
Pennycook, G. et al (July 5, 2017) Prior Exposure Increases Perceived Accuracy of Fake News,
Available at SSRN: https://ssrn.com/abstract=2958246
Picone, I. (2015) Impression Management in Social Media, Published Online: 11 FEB 2015.
Available at:
http://onlinelibrary.wiley.com/doi/10.1002/9781118767771.wbiedcs071/abstract
Politi, J. (Dec. 30, 2016) ‘Italy antitrust chief urges EU to help beat fake news’. Financial
Times,
https://www.ft.com/content/e7280576-cddc-11e6-864f-20dcb35cede2?mhq5j=e2
Pondsford, D. (March, 31, 2017) The decline of local journalism is a far greater threat to media
plurality than Rupert Murdoch, The Press Gazette, http://www.pressgazette.co.uk/the-declineof-local-journalism-is-a-far-greater-threat-to-media-plurality-than-rupert-murdoch/
Posetti, P. (July 2017) This is why Online Harassment Still Needs Attention, MediaShift
http://mediashift.org/2017/07/online-harassment-still-needs-attention/
Postman, N. (1985). Amusing Ourselves to Death: Public Discourse in the Age of Television.
New York: Viking.

101

Potter, M. C. (2014). Detecting and remembering briefly presented pictures. In K. Kveraga &M. Bar (Eds.), Scene Vision Cambridge, MA: MIT Press, pp. 177-197.
Prior, M. (2002). Any good news in soft news? The impact of soft news preference on politicalknowledge. Political Communication, 20(2), pp.149-171.
Qiu, X., D. F. M. Oliveira, A.S. Shirazi, A. Flammini and F. Menczer (2017) Limited
Individual Attention and Online Virality of Low-Quality Information, Nature Human
Behaviour, 1
Quattrociocchi, W. (14 Jan, 2016) How does mis-information spread online? World Economic
Forum Blog, https://www.weforum.org/agenda/2016/01/q-a-walter-quattrociocchi-digitalwildfires/
Ratkiewicz et al., (2011) Detecting and Tracking Political Abuse in Social Media, Proceedingsof the Fifth International AAAI Conference on Weblogs and Social Media,
https://www.aaai.org/ocs/index.php/ICWSM/ICWSM11/paper/view/2850
Renner, N. (January 30, 2017) Memes trump articles on Breitbart’s Facebook page, Columbia
Journalism Review. https://www.cjr.org/tow_center/memes-trump-articles-on-breitbartsfacebook-page.php
Resnick, B. (May 6, 2017) 7 Psychological Concepts that Explain the Trump Era of Politics,
Vox, https://www.vox.com/science-and-health/2017/3/20/14915076/7-psychological-conceptsexplain-trump-politics
Resnick, B. (July 10, 2017) “Trump supporters know Trump lies. They just don’t care.” Vox,
https://www.vox.com/2017/7/10/15928438/fact-checks-political-psychology
Rutenberg, J. (Sept. 13, 2017) RT, Sputnik and Russia’s New Theory of War, New York Times,
https://www.nytimes.com/2017/09/13/magazine/rt-sputnik-and-russias-new-theory-ofwar.html?
Salganik, M. et al. (2006) Experimental Study of Inequality and Unpredictability in an
Artificial Cultural Market, Science, Vol. 311, pp.854-856
Schmidt, C. (Aug. 3, 2017) Games might be a good tool for fighting fake news. Here’s what
three developers have learned, Nieman Lab, http://www.niemanlab.org/2017/08/games-mightbe-a-good-tool-for-fighting-fake-news-heres-what-three-developers-have-learned/
Schwartz, J. (May 24, 2016) Most Popular Messaging App in Every Country, Market
Intelligence Blog, https://www.similarweb.com/blog/worldwide-messaging-apps

102

Shaffer, K, Carey, CE & Starling, B. (2017) Democracy Hacked. Available at:https://medium.com/data-for-democracy/democracy-hacked-a46c04d9e6d1
Shahari, A. (Nov 11. 2017) Zuckerberg Denies Facebook’s Impact on the Election, NPR,
http://www.npr.org/sections/alltechconsidered/2016/11/11/501743684/zuckerberg-denies-fakenews-on-facebook-had-impact-on-the-election
Shao, C., G.L. Ciampaglia, O. Varol, A. Flammini and F. Menczer, (July 24, 2017) The
Spread of Fake News by Social Bots, https://arxiv.org/pdf/1707.07592.pdf
Shao, C. et al., (2016) Hoaxy: A Platform for Tracking Online Mis-information, Proceedingsof the 25th International Conference Companion on World Wide Web, pp. 745-750
Shukla, S. (August 28, 2017) Blocking Ads from Pages that Repeatedly Share False News.
Facebook Newsroom. https://newsroom.fb.com/news/2017/08/blocking-ads-from-pages-thatrepeatedly-share-false-news/
Silverman, C. (2015), Lies, Damn Lies and Viral Content, Tow Center for Digital Journalism.
Available at: http://towcenter.org/wpcontent/uploads/2015/02/LiesDamnLies_Silverman_TowCenter.pdf
Silverman, C. (2016a) Facebook Must Either Innovate or Admit Defeat At The Hands Of Fake
News Hoaxsters, Buzzfeed News, August 30, 2016. Available at:https://www.buzzfeed.com/craigsilverman/how-facebook-fell-into-a-fake-news-trap-of-itsown-making?utm_term=.tpdEQ2QJnN#.poVnBOBx2E
Silverman, C. (2016b) This Analysis Shows How Viral Fake Election News Stories
Outperformed Real News on Facebook, Buzzfeed News, November 16, 2016. Available at:https://www.buzzfeed.com/craigsilverman/viral-fake-election-news-outperformed-real-newson-facebook?utm_term=.aaMXl5lvgG#.wtVW3y3jYn
Silverman et al. (August 8, 2017) Inside the Partisan Fight for your NewsFeed, Buzzfeed News,https://www.buzzfeed.com/craigsilverman/inside-the-partisan-fight-for-your-news-feed/
Silverman, C. et al., (2016) Hyperpartisan Facebook Pages Are Publishing False And
Misleading Information At An Alarming Rate, Buzzfeed News, October 20, 2016. Available at:https://www.buzzfeed.com/craigsilverman/partisan-fb-pagesanalysis?utm_term=.lgVgaAa8Mo#.xmk3BQBoEp
Silverman, C. & Singer-Vine, J. (2016a). “Most Americans Who See Fake News Believe It,
New Survey Says.” BuzzFeed News, December 6. Available at:https://www.buzzfeed.com/craigsilverman/fake-newssurvey?utm_term=.uov6wAwgJK#.jvLRNKN136

103

Silverman, C. & Singer-Vine, J. (2016b). “The True Story Behind The Biggest Fake News Hit
Of The Election.”, Buzzfeed News, December 16. Available at:https://www.buzzfeed.com/craigsilverman/the-strangest-fake-newsempire?utm_term=.yrzPyEpLXq#.nqXD9N7opO
Silverman, C. & Alexander, L. (2016) “How Teens In the Balkans Are Duping Trump
Supporters With Fake News.” Buzzfeed News, November 3, 2016. Available at:https://www.buzzfeed.com/craigsilverman/how-macedonia-became-a-global-hub-for-protrump-misinfo?utm_term=.ftzDj7japR#.ex0dMBM37O
Smith, L. C., Lucas, K. J., & Latkin, C. (1999). Rumor and gossip: Social discourse on HIV
and AIDS. Anthropology & Medicine, 6 (1), 121-131.
Sontag S. (1977) On Photography, New York: Farrar Straus and Giroux.
Spencer, S. (Jan. 25, 2017) How we fought bad ads, sites and scammers in 2016, Google Blog,
https://www.blog.google/topics/ads/how-we-fought-bad-ads-sites-and-scammers-2016
Stamos, A. (September 6, 2017) An Update on Information Operations On Facebook,
Facebook Newsroom, https://newsroom.fb.com/news/2017/09/information-operations-update/
Stanley, S. (May 16, 2017) Mis-information and hate speech in Myanmar, First Draft News,https://firstdraftnews.com/mis-information-myanmar/
Stelzenmüller, C. (June 28, 2017) Testimony to the US Senate Committee on Intelligence: The
Impact of Russian interference on Germany’s 2017 Elections, Available at:https://www.brookings.edu/testimonies/the-impact-of-russian-interference-on-germanys-2017elections/
Stelter, B. (Feb. 11, 2016) Apple CEO Tim Cook calls for "massive campaign" against fakenews, CNN, http://money.cnn.com/2017/02/11/media/fake-news-apple-ceo-timcook/index.html
Stray, J. (Feb 27, 2017), Defense Against the Dark Arts: Networked Propaganda and CounterPropaganda, Tow Center, Medium. https://medium.com/tow-center/defense-against-the-darkarts-networked-propaganda-and-counter-propaganda-deb7145aa76a
Subramanian, S. (2017) Inside the Macedonian Fake News Complex, Wired, February 15,
2017. Wired. Available at: https://www.wired.com/2017/02/veles-macedonia-fake-news
Sunstein, C. R., Bobadilla-Suarez, S., Lazzaro, S. C., & Sharot, T. (2016). How People Update
Beliefs about Climate Change: Good News and Bad News (SSRN Scholarly Paper No. ID
2821919). Rochester, NY: Social Science Research Network.

104Sunstein, Cass R., and Adrian Vermeule. (2009) “Conspiracy Theories: Causes and Cures”.
Journal of Political Philosophy. 17 (2): 202–227.
Sullivan, M (Jan 6, 2017,) It’s Time To Retire the Tainted Term Fake News, The Washington
Post, https://www.washingtonpost.com/lifestyle/style/its-time-to-retire-the-tainted-term-fakenews/2017/01/06/a5a7516c-d375-11e6-945a-76f69a399dd5_story.html
Susman-Peña, T. and Vogt, Katya (June 12, 2017) Ukrainians’ self-defense against information
war: What we learned from Learn to Discern, IREX, https://www.irex.org/insight/ukrainiansself-defense-against-information-war-what-we-learned-learn-discern
Suwajanakorn, S. et al. (July 2017) Synthesizing Obama: Learning Lip Sync from Audio, ACM
Transactions on Graphics, 36 (4). Article 95.
http://grail.cs.washington.edu/projects/AudioToObama/siggraph17_obama.pdf
Sydell, L. (2017) (Nov 23, 2016) We Tracked Down A Fake-News Creator In The Suburbs.
Here's What We Learned, NPR,
http://www.npr.org/sections/alltechconsidered/2016/11/23/503146770/npr-finds-the-head-of-acovert-fake-news-operation-in-the-suburbs
Taber, C. S., & Lodge, M. (2006). Motivated skepticism in the evaluation of political beliefs.
American Journal of Political Science, 50(3), 755-769.
Tambini, D. (2017) How advertising fuels fake news. LSE Media Policy Project Blog,
available at: http://blogs.lse.ac.uk/mediapolicyproject/2017/02/24/how-advertising-fuels-fakenews/
Tandoc, Jr., E. C, Lim, Z. W., and Ling, R. (Aug. 2017) Defining ‘Fake News’: A Typology of
Scholarly Definitions, Digital Journalism, 5 (7): 1-17
Thies, J. at al. (2016) Face2Face: Real-time Face Capture and Reenactment of RGB Videos,The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 23872395
Thompson, A. (Dec. 8, 2016) Parallel Narratives, Clinton and Trump Supporters Really Don’t
Listen to One Another, Vice. https://news.vice.com/story/journalists-and-trump-voters-live-inseparate-online-bubbles-mit-analysis-shows
Thompson, D. (Nov. 3, 2016) The Print Apocalypse and How to Survive It, The Atlantic.
https://www.theatlantic.com/business/archive/2016/11/the-print-apocalypse-and-how-tosurvive-it/506429/
Uberti, D. (December 15, 2016) The Real History of Fake News, Columbia Journalism

105

Review, https://www.cjr.org/special_report/fake_news_history.php
Venkatraman, A., Mukhija, D., Kumar, N., & Nagpal, S. J. S. (2016). Zika virus misinformation on the internet. Travel medicine and Infectious Disease, 14(4), 421-422.
Vaidhyanathan, S. (Sept 8. 2017) Facebook Wins, Democracy Loses. New York Times,
https://www.nytimes.com/2017/09/08/opinion/facebook-wins-democracy-loses.html
Van Damme, I. & K. Smets, 2014, p. 310. The power of emotion versus the power ofsuggestion: Memory for emotional events in the mis-information paradigm. Emotion. 14 (2):310-320
Weedon, J., W. Nuland & A. Stamos (April 27, 2017) Information Operations and Facebook,
Facebook Newsroom. https://fbnewsroomus.files.wordpress.com/2017/04/facebook-andinformation-operations-v1.pdf
Wardle, C. (February 16, 2017) Fake News. It’s Complicated. First Draft News. Available at:https://firstdraftnews.com/fake-news-complicated/
Wardle, C. (2016) Timeline: Key Moments in the Fake News Debate, First Draft News, (Sept30 – Dec 1, 2016). Available at: https://medium.com/1st-draft/key-moments-in-the-fake-newsdebate-bce5fb6547d4
Woolley, S and P. Howard (2017) ‘Social Media, Revolution and the Political Bot’, inRoutledge Handbook or Media, Conflict and Security, (edited by Piers Robinson, Philip Seib,
Romy Frohlich), London: Routledge.
Woolley, S & P. Howard, (2017) Computational Propaganda Worldwide: An Executive
Summary, http://comprop.oii.ox.ac.uk/wp-content/uploads/sites/89/2017/06/CasestudiesExecutiveSummary.pdf
Yale University’s Information Society Project’s ‘Fighting Fake News’ Workshop
https://law.yale.edu/system/files/area/center/isp/documents/fighting_fake_news__workshop_report.pdf
Yi, S.B. (June 19, 2017) New legislation to combat fake news likely to be introduced next
year: Shanmugam, Straits Times, http://www.straitstimes.com/singapore/new-legislation-tocombat-fake-news-next-year-shanmugam
Young, D. G., K.H. Jamieson, S. Poulsen and A. Goldring (2017) Fact-Checking Effectivenessas a Function of Format and Tone: Evaluating FactCheck.org and FlackCheck.org, Journalism
And Mass Communication Quarterly, 1-27

106

Yurieff, K. (April 5, 2017) eBay founder commits $100 million to combat 'fake news', CNN,
http://money.cnn.com/2017/04/05/technology/pierre-omidyar-donation-fake-news/index.html
Zaller, J. (1992) The Nature and Origins of Mass Opinion, Cambridge University Press.
Zimdars, M. (Nov 18, 2016) My ‘fake news list’ went viral. But made-up stories are only partof the problem, The Washington Post,
https://www.washingtonpost.com/posteverything/wp/2016/11/18/my-fake-news-list-wentviral-but-made-up-stories-are-only-part-of-the-problem/
Zuckerman, E. (January 30, 2017) Stop Saying ‘Fake News’ It’s Not Helping,
Ethanzuckerman.com, Available at: http://www.ethanzuckerman.com/blog/2017/01/30/stopsaying-fake-news-its-not-helping/
Zuckerman, E. (August 2017) Mistrust, Efficacy and the New Civics, A Whitepaper for the
Knight Foundation, https://dspace.mit.edu/handle/1721.1/110987#files-area

107

This report provides a new framework for policy-makers,
legislators, researchers, technologists and practitioners
working on the theoretical and practical challenges related
to mis-, dis- and mal-information — the three elements of
information disorder. While the historical impact of rumours
and fabricated content have been well documented,
the complexity and scale of information pollution in our
digitally-connected, increasingly polarised world presents
an unprecedented challenge. There is an immediate need towork collaboratively on workable solutions and this reportprovides a framework for the different stakeholders involved
in research, policy discussions, and technical innovations
connected to this phenomenon of information disorder.

PREMS 162317

This report has been commissioned by the Council of Europe and produced in co-operation
with First Draft and the Shorenstein Center on Media, Politics and Public Policy.

www.coe.int/freedomofexpressionwww.coe.int

The Council of Europe is the continent’s leading humanrights organisation. It comprises 47 member states, 28 ofwhich are members of the European Union. All Council ofEurope member states have signed up to the European
Convention on Human Rights, a treaty designed toprotect human rights, democracy and the rule of law.
The European Court of Human Rights oversees theimplementation of the Convention in the member states.

ENG