POLITIEK EN SOCIALE MEDIA MANIPULATIE
THE POLITICS OF SOCIAL MEDIA MANIPULATION

Edited by Richard Rogers and Sabine Niederer

The Politics of Social Media Manipulation
Edited by Richard Rogers and Sabine Niederer

The Politics of Social Media Manipulation
Editors: Richard Rogers and Sabine Niederer
Design: Federica Bardelli, Gabriele Colombo, and Carlo De Gaetano
Digital Methods Initiative, Media Studies, University of Amsterdam
© The authors, 2019.

This research was commissioned by the Dutch Ministry of the Interior and Kingdom Relations
(BZK). The authors are responsible for all findings and conclusions presented in the report.
The content does not necessarily reflect the views of the Minister of the Interior and Kingdom
Relations.
Dit onderzoek is uitgevoerd in opdracht van het Ministerie van Binnenlandse Zaken en
Koninkrĳksrelaties. De verantwoordelĳkheid voor de inhoud van het onderzoek berust bĳ de
auteurs. De inhoud vormt niet per definitie een weergave van het standpunt van de Minister van
Binnenlandse Zaken en Koninkrĳksrelaties.

1

Table of contents
Nederlandse samenvatting: Politiek en sociale media manipulatie

3

1. The politics of social media manipulation
Richard Rogers and Sabine Niederer

14

2. Junk news on Facebook during the 2019 Dutch elections
Stijn Peeters and Richard Rogers

43

3. Junk news in search engines: Exploring Google’s sensibility towards hyperpartisan
sources during the Dutch elections
Guillén Torres and Richard Rogers

63

4. Twitter, junk news and troll-like users during the Dutch provincial and European
elections
Sabine Niederer and Maarten Groen

81

5. The presence of fakeness in the Dutch political Instagram: Fake content, sources and
followers
100
Gabriele Colombo and Carlo De Gaetano
6. Dutch junk news on Reddit and 4chan/pol/
Sal Hagen and Emilija Jokubauskaite

116

7. Fake news and the Dutch YouTube political debate space
Marc Tuters

152

8. Conclusions: Mainstream under fire
Richard Rogers and Sabine Niederer

167

2

Nederlandse samenvatting:
POLITIEK EN SOCIALE MEDIA MANIPULATIE

Politiek en sociale media manipulatie

Samenvatting
Van desinformatie en nepnieuws naar het bredere begrip junknieuws
Desinformatie en nepnieuws zijn hedendaagse fenomenen met een rijke historie. Desinformatie,
of de opzettelijke introductie van foutieve informatie met het doel schade te berokkenen, kent
verschillende recentelijke en historische voorbeelden van met name buitenlandse inmenging in
nationale mediasystemen met als doel om verdeeldheid te zaaien. In de jaren 80 verspreidde de
Russische campagne ‘Operation Infektion’ het gerucht dat het HIV-virus zou voortkomen uit
lab-experimenten met biochemische wapens in de Verenigde Staten. Het verhaal werd opgepikt
door gerenommeerde nieuwsmedia en zelfs in het avondjournaal uitgezonden.
Tegenwoordig omvatten nationale medialandschappen ook globale sociale mediaplatforms zoals
Twitter en Facebook. In 2018 werd bekend dat een trollenfabriek in Sint-Petersburg met de
naam ‘Internet Research Agency’ actief desinformatie verspreidde over de MH17 ramp op
sociale media. Waar een dataset met de twitterberichten (‘tweets’) die vanuit deze trollenfabriek
werden verzonden destijds werd vrijgegeven om onderzoek te doen naar mogelijke inmenging in
de Amerikaanse verkiezingen, bleken de trollen juist het actiefst te zijn geweest op de dag van en
de twee dagen na het neerhalen van MH17, overigens hoofdzakelijk in het Russisch en slechts
marginaal in het Nederlands (Noorda en van de Ven 2019).
Nepnieuws, ofwel gefingeerde verhalen in de gedaante van een nieuwsbericht, zien we door de
jaren heen telkens verschijnen bij de introductie van nieuwe mediatechnologieën in de
journalistiek, zoals bij de ‘broadsheet’ kranten van eeuwen geleden tot de meer recentelijk in de
‘blogosphere’. De term nepnieuws kennen we ook als aantijging, waarbij nieuwsorganisaties
wordt verweten nep -fake news- of zelfs leugenachtig te zijn- ‘der Lügenpresse’. Dit gebruik van de
term nepnieuws kent een duistere geschiedenis die in de literatuur vooral wordt geassocieerd met
autoritaire regimes of met populistische aanvallen op zogenaamde ‘elite media’.
Vandaag de dag spelen sociale media platforms een belangrijke rol bij zowel desinformatie als bij
nepnieuws in beide betekenissen. Zoals besproken in theoretisch en empirisch onderzoek naar
dit onderwerp, hebben sociale media de inmenging van buitenlandse desinformatie-operaties in
nationale mediaslandschappen mogelijk gemaakt. Daarnaast hebben deze platforms de
wijdverbreide circulatie van gefingeerde content gefaciliteerd, en bovendien ruimte geboden aan
tendentieuze nieuwscommentatoren -veelal met een omvangrijk publiek- die de mainstream
media als nepnieuws bestempelen.
Wereldwijd worden desinformatie en nepnieuws vooral gezamenlijk bestudeerd, maar het is
relevant om deze fenomenen ook apart te bezien. Zoals in eerder onderzoek beschreven is in
Nederland vrijwel geen bewijs voor buitenlandse desinformatie te vinden en zijn voorbeelden
van Nederlandse spelers die Russische desinformatie-tactieken toepassen ook maar zelden
gedocumenteerd. In tegenstelling tot de situatie in de VS, waar het meeste onderzoek naar
verricht is, is er in Nederland ook geen sprake van de opkomst van namaak-nieuwsorganisaties
3

of ‘fake’ actiegroepen die doen alsof zij sociale groepen of doelen representeren. Sterker nog,
wanneer men nauwe definities hanteert van desinformatie en nepnieuws is er nauwelijks zoiets te
vinden in Nederland.
Maar definities van nepnieuws en met name het aanpalende ‘junknieuws’ worden vaak juist
verruimd, en omvatten dan naast desinformatie ook complottheorie, clickbait, alsmede
extremistische, sensationalistische, tendentieuze en politiek sterk gekleurde (‘hyperpartisan’)
bronnen en verhalen. Daarnaast kennen sociale media het probleem van de ‘artificiële
amplificatie’, waarbij accounts en content door middel van nep-volgers en -likes populairder
lijken dan ze in werkelijkheid zijn en daarmee symbolische macht verwerven. Toen het beroemde
artikel van Buzzfeed News in oktober 2016 vaststelde dat nepnieuws het in de VS in de aanloop
naar de presidentiële verkiezingen beter deed dan mainstream nieuws op Facebook, omvatte de
definitie waarop de studie gebaseerd was ook clickbait en politiek sterk gekleurde bronnen.
Wanneer we met die ruimere blik naar de situatie in Nederland kijken, treffen we wel degelijk
junknieuws aan.
Eerdere studies hebben geconcludeerd dat Nederlanders grootverbruikers zijn van clickbait en
zogenaamde junk content. Daarnaast is er sprake van een aantal zeer populaire tendentieuze en
nieuws-achtige bronnen, vooral aan de rechterzijde van het politieke spectrum. Tenslotte zijn er
al incidenten gerapporteerd met nepvolgers en -fans van Nederlandse politici en muzikanten. Uit
onderzoek blijkt dat clickbait meer gelezen wordt dan mainstream nieuws, maar er is dan ook
meer van in absolute aantallen. Nieuws aangeboden door commerciële en publieke partijen
wordt nog altijd in grotere getalen genoemd, geliket en gedeeld dan tendentieuze bronnen,
hoewel bronnen uit die laatste categorie zeker rondom verkiezingen veel circuleren. Het
kunstmatig opvijzelen van online impact kan op korte termijn iemands imago een boost geven
maar leidt tot mini-schandalen wanneer nepvolgers worden ontmaskerd door nieuwe online
detectietools en datajournalistiek.
Of junknieuwsbronnen hun lezers ook overtuigen wordt in toenemende mate onderzocht. Waar
een eerste lichting onderzoekers zich vooral richtte op de productie en verspreiding van
desinformatie, richt een tweede golf zich vooral op de effecten ervan. Gegeven de ‘verhardende
houding’ van het publiek rijst de vraag of de invloed van desinformatie en junknieuws ooit groter
dan minimaal zal zijn.
In dat opzicht is de opkomst van extreme content (waaronder ‘extreme clickbait’) die wordt
gecirculeerd op sociale media er een van bijzondere aandacht en zorg, met oproepen tot
regulering van deze platforms tot gevolg. Een ander aandachtspunt is het mainstreamen van
twijfel en wantrouwen jegens publieke instellingen en de media, dat gelijk opgaat met de
opkomst van ‘alternatieve feiten’ en de infrastructuren die daaraan ten grondslag liggen. Dit
wordt ook de ‘post-truth conditie’ genoemd, waar feiten in twijfel worden getrokken en gefactchecked en tevens naast de mainstream nieuwsmedia een concurrerende media ecologie
wordt opgebouwd. Neemt de autoriteit van mainstream nieuws en kennisinstellingen af nu die
concurrerende media ecologie in toenemende mate wordt gedeeld en geliket? Uit Amerikaans
onderzoek is gebleken dat in de VS vooral oudere nieuwsconsumenten een gewillig publiek
vormen voor dergelijke bronnen en daar ook nog eens relatief veel gebruik van maken.
4

Het consumeren en delen van nepnieuws is ook het aandachtsgebied van initiatieven ter
bevordering van mediawijsheid, middels quizzen, serious games en publieke campagnes. Door
verhoogde bewustwording, vooral rond de verkiezingen, kan een al te grote impact van
desinformatie en dubieuze content worden afgewend, zo is de gedachte. Vrijwillige en
professionele fact-check initiatieven worden in de literatuur ook besproken, net als het (al dan
niet automatisch) labelen van sociale media content die op juistheid gecontroleerd zou moeten
worden. De vraag die hierbij vaker wordt gesteld is of het volume van junk content op deze
platforms niet veel groter is dan de beschikbare capaciteit om het te reviewen. En daaraan
gerelateerd is het de vraag wie de reviewers controleert.
Tenslotte worden er in de literatuur volop zorgen geuit over de beperkte toegankelijkheid tot de
data van sociale media, ook bekend als de ‘locked platforms’ discussie. Toekomstig onderzoek
zou zich kunnen richten op de mate waarin desinformatie en nepnieuws in al hun
verschijningsvormen online nog altijd floreren. De vraag is dan of er voldoende capaciteit is om
deze te monitoren zodat het algemeen gebruik van die bronnen kan worden gemeten, de
overtuigingskracht kan worden vastgesteld en de bredere maatschappelijke implicaties kunnen
worden gewogen.
Bevindingen in het kort
Deze studie bestaat uit een reeks empirische casussen omtrent het engagement met
nep/junknieuws (in de rest van deze samenvatting kortweg aangeduid als junknieuws), evenals
politiek sterk gekleurde (‘hyperpartisan’) en tendentieuze bronnen, in Nederlandse politieke
arena’s op sociale media tijdens twee verkiezingen in 2019. Deze arena’s zijn afgebakend middels
zoekopdrachten naar de namen van de politici en politieke partijen, en sociale kwesties. Sommige
van deze kwesties zijn gerelateerd aan de verkiezingen (zoals EU) en andere zijn controversiëler
van aard (zoals Zwarte Piet). Hier zijn de bevindingen samengevat, conclusies per platform kort
beschreven, en tenslotte in perspectief geplaatst met een bespreking van de beleidsaanbevelingen.
Vraag: In hoeverre resoneren desinformatie en zogenaamd junknieuws in de online politieke
arena’s op sociale media, voor en tijdens de Provinciale Statenverkiezingen en Europese
Parlementsverkiezingen van 2019 in Nederland?
1) Wij hebben geen buitenlandse desinformatiecampagne of nep-actiegroepen gevonden,
rond de Provinciale Staten- en Europese parlementsverkiezingen van 2019.
2) Mainstream nieuws wordt op alle platforms meer gecirculeerd en verkrijgt meer
gebruiksinteractie dan junknieuws, maar dit geldt niet voor alle specifieke kwesties in alle
periodes (8 februari – 25 maart 2019, 26 april - 24 mei 2019 of langere perioden).
3) Zwarte Piet en MH17 hebben aanzienlijke hoeveelheden tendentieus en nep/junknieuws
en zijn doorlopend actief (niet aan een periode gebonden).
4) Met betrekking tot sociale mediamanipulatie, zijn trol-achtige gebruikers actief omtrent
meerdere politieke kwesties (op Twitter). We vermoeden ook artificiële versterking
(nepvolgers op Instagram).

5

5) Er is een opkomende tendentieuze en polariserende mediasfeer, die concurreert met de
mainstream, in de zin dat ze niet alleen door trol-achtige sociale media gebruikers maar
ook door reguliere sociale media gebruikers worden verspreid.
6) Facebook heeft relatief de grootste hoeveelheid junknieuws vergeleken met de andere
platforms, gevolgd door Twitter. YouTube is een belangrijke rechtse debat-arena voor
‘nepnieuws’ als maatschappelijke kwestie.
7) De Nederlandse 4chan en Reddit refereren aanzienlijk minder aan junknieuws dan aan
mainstream nieuws. 4chan is een incubator voor uiterst rechtse activiteit in Nederland,
middels het verwijzen naar polariserende YouTube video’s.
Beknopte conclusies per platform
Facebook: een vruchtbare bodem voor junknieuws
De studie van Facebook is inhoudelijk gebaseerd op (data-)journalisiek van Buzzfeed News en
NRC Handelsblad, maar waar Buzzfeed opzettelijke desinformatie heeft aangetroffen in hun
studie in de Verenigde Staten komt in onze data gericht op Nederland geen desinformatie voor.
Het onderzoek bevestigt wel de reputatie van Facebook als vruchtbare bodem voor junknieuws
in vergelijking met andere platforms en toont aan dat ondanks de initiatieven die het platform
heeft genomen om dergelijke content te weren en verwijderen, deze nog steeds op het platform
circuleert. Sterker nog, waar de NRC concludeerde dat ‘hoogstens tien procent” (Kist &
Zantingh 2017) van de nieuwsconsumptie politiek sterk gekleurde en tendentieuze content
betrof, zien wij in onze analyse (die iets meer dan een jaar later plaatsvond) dat dit aandeel is
gestegen tot 25%.
Alhoewel een deel van het verschil in bevindingen kan worden toegeschreven aan verschillen in
de criteria voor het categoriseren van de content, is te concluderen dat zelfs als junknieuws in de
minderheid is, het zeker niet marginaal is. Junknieuws neemt toe in het Nederlandse
medialandschap en wordt soms net zoveel gedeeld en geliket als mainstream nieuws. Het
verdient aanbeveling om dergelijke interactie (ook wel aangeduid met de term ‘engagement’) met
junknieuws nader te analyseren om te zien wat dit in de praktijk betekent. Interactie met een
bepaald artikel -bijvoorbeeld door het artikel te delen- betekent niet per definitie dat die
gebruiker de inhoud van het artikel steunt. Nu we zien dat junknieuws een belangrijke factor is
op Facebook, is het van belang de motivatie van het deelgedrag beter te doorgronden om te
kunnen vaststellen welke rol het speelt in het politieke debat.
Terwijl Facebook een stevige onderstroom kent van junknieuws waarbinnen sterk politiek
gekleurd nieuws en een aantal complotwebsites circuleren, is er geen reden om aan te nemen dat
junknieuws een grote rol heeft gespeeld in het nieuws over de verkiezingen in Nederland in 2019,
zoals dit tijdens de aanloop van de Amerikaanse presidentsverkiezingen in 2016 in de Verenigde
Staten wel het geval was. Onze studie toont aan dat er geen onmiddellijke aanleiding is voor
zorgen over de rol van desinformatie bij Nederlandse verkiezingen, maar dat junknieuws wel een
groeiende factor is die kritisch moet worden gevolgd.
Google Web Search: zoekopdrachten in spreektaal resulteren in junknieuws
De Google zoekmachine-studie draait om het lokaliseren van nep/junknieuws in de eerste
6

twintig Google.nl zoekresultaten voor zoekopdrachten die te maken hebben met Nederlandse
politieke partijen en hun belangrijkste kwesties tijdens de provinciale staten- en Europese
parlementsverkiezingen van 2019. De zoekopdrachten zijn geformuleerd door de namen van de
politieke partijen te combineren met sleutelwoorden voor bepaalde kwesties. Hierbij is gekozen
om zowel ‘officiële taal’ te verzamelen uit de partijprogramma’s, als ‘spreektaal’ uit de
commentaren op de Facebookpagina’s van de verschillende partijen rond de Provinciale
Statenverkiezingen, om deze ook onderling met elkaar te kunnen vergelijken. Onze resultaten
tonen aan dat de junknieuws websites die aanwezig zijn in de Google.nl zoekresultaten voor
politieke zoekopdrachten vrijwel uitsluitend politiek sterk gekleurde (hyperpartisan) bronnen
betreffen, en niet zozeer bronnen die desinformatie, complottheorie of clickbait verspreiden. Er
zijn geen nepactiegroepen of buitenlandse desinformatiebronnen gevonden.
Zoekopdrachten naar kwesties in combinatie met de namen van centrum-rechtse en rechtse
partijen resulteren in meerdere mate in junknieuwsbronnen dan zoekopdrachten waarin
centrum-links of linkse partijen genoemd worden. Op google.nl is er een aanzienlijke kans dat
een zoekopdracht met naam van een rechtse partij en hun kwesties leidt tot junknieuws in de top
zoekresultaten. Een andere bevinding is dat de aanwezigheid van junknieuws in de
zoekresultaten niet stabiel is, maar varieert door de tijd heen. Voor de Provinciale
Statenverkiezingen zien we meer junknieuws in de topresultaten tijdens de dagen voorafgaand
aan de verkiezingen, om tijdens de verkiezingen en direct daarna van de radar te verdwijnen
(zoals duidelijk te zien rond de kwestie migratie). Bij de EU-verkiezingen is het effect precies
tegenovergesteld, en zijn junk nieuwsbronnen vooral prominent aanwezig in de topzoekresultaten op de dag na de verkiezingen.
Wanneer we de resultaten van de twee soorten zoekopdrachten (resp. gebaseerd op officiële taal
uit partijprogramma’s en spreektaal uit Facebook commentaren) met elkaar vergelijken, zien we
dat een zoekopdracht die is geformuleerd in spreektaal leidt tot een hoger percentage junknieuws
in de resultaten, bijvoorbeeld waar het gaat om migratie. In andere woorden: spreektaal leidt tot
gekleurder nieuws in de resultaten dan ‘officiële’ partijprogramma-taal.
Twitter: junknieuws en trol-activiteit rond polariserende kwesties
In de Twitter-studie hebben we gekeken naar de aanwezigheid van junknieuws, tendentieus
nieuws en trol-achtige activiteit tijdens de campagneperiodes van de Provinciale
Statenverkiezingen en Europese Parlementsverkiezingen in Nederland in 2019. Tijdens de
Provinciale Statenverkiezingen is trol-achtige activiteit aangetroffen rond zowel politieke thema’s
(zoals de verkiezingen zelf en de partijleiders) als polariserende kwesties zoals MH17, Zwarte
Piet en de aanslag in Utrecht. Trol-achtige activiteit verwijst naar een aantal indicatoren in
twittergedrag, waarvan de belangrijkste een opvallend hoog aantal tweets in een heel korte
periode is. Uit de analyse blijkt dat veertien trol-achtige twittergebruikers rond al deze thema’s
actief waren, en maar liefst 29 waren actief rond vier van de vijf thema’s. Vier van deze trolachtige twitterprofielen waren nog altijd (of wederom) actief tijdens de Europese verkiezingen.
Deze trol-achtige gebruikersprofielen verspreidden vooral sterk politiek gekleurde en
tendentieuze bronnen, gevolgd door complotwebsites.

7

Rond polariserende kwesties is in beide campagneperiodes activiteit waargenomen waarin
junknieuws een aanzienlijke rol speelt. Tijdens de Provinciale Staten campagneperiode waren de
kwesties Zwarte Piet en MH17 actief, ook al vindt het Sinterklaasfeest in december plaats en was
er in die periode geen MH17 herdenking. Tendentieuze en politiek sterk gekleurde bronnen
worden door trol-achtige gebruikers het meest verspreid, gevolgd door complottheorie-bronnen
die vooral rond tragedies (MH17 en Utrecht) circuleren. De pro-Russische site novini.nl, die op
verhaal-niveau laveert tussen sterk politiek gekleurd, complottheorie en desinformatie, is door
trol-achtige gebruikers rond alle kwesties gedeeld, in het bijzonder rond MH17 en slechts
marginaal rond de politieke leiders. Tijdens de Europese Verkiezingsperiode was op
verschillende momenten junknieuws zelfs prominenter aanwezig dan mainstreamnieuws rond de
polariserende onderwerpen Zwarte Piet en MH17. Belangrijk hierbij aan te merken is dat dit
vooral is veroorzaakt door een afname van mainstream media-aandacht voor deze onderwerpen
in deze periodes, terwijl junknieuws-aandacht actief blijft en daar de polarisatie voedt.
Gebaseerd op deze resultaten lijkt de Nederlandse Twittersfeer geen algemeen desinformatieprobleem te hebben, al zijn er trol-achtige gebruikers actief die bepaalde stemmen en bronnen
kracht bijzetten. Hoewel we geen professionele of grootschalige trollencampagne hebben
aangetroffen, is er wel trol-achtige activiteit gesignaleerd die zich uitstrekte over meerdere
kwesties. Deze trol-achtige activiteit is afkomstig van verschillende soorten gebruiksprofielen,
zowel in de vorm van geautomatiseerde ‘bots’ als van schijnbaar semi-geautomatiseerde profielen
die een combinatie tonen van automatisch retweeten en het posten van originele content. Rond
polariserende thema’s blijft de activiteit in junknieuws en tendentieuze nieuwsbronnen stabiel
(hoe marginaal ook) in beide onderzoeksperioden, hetgeen suggereert dat deze kwesties in
junknieuws het gehele jaar leven en niet alleen in het verwachte seizoen (november/december
voor Zwarte Piet en juli voor MH17).
Instagram: rechtse mediasfeer en tekenen van artificiële amplificatie
De Instagramstudie onderzoekt vanuit drie invalshoeken de aanwezigheid (en afwezigheid) van
desinformatie: op post-niveau, kijkend naar de circulatie van junknieuws in politieke arena’s, op
bron-niveau door de volgers te vergelijken van Nederlandse politieke partijen en politiek leiders
met die van junknieuwsbronnen, en op activiteit-niveau, middels de detectie van artificiële
amplificatietactieken zoals nepvolgers van politieke partijen en leiders. In het algemeen troffen
wij op Instagram een ‘gezonde’ politieke arena aan met slechts een klein aantal voorbeelden van
junknieuws en artificiële amplificatie. De overgrote meerderheid van de meest ‘gelikete’ content
in de Nederlandse politieke arena betreft geen junknieuws, al zien we rond rechtse politici en een
aantal kwesties wel een kleine hoeveelheid politiek sterk gekleurde en tendentieuze content. Voor
het publiek van de politieke partijen en leiders was ook op Instagram in beide periodes
mainstreamnieuws prominenter aanwezig dan junknieuws. Kijkend naar gebruikers van het
platform die actief zijn geweest rond politieke partijen en leiders, zien wij een schijnbaar
authentiek publiek met vrijwel geen tekenen van artificiële activiteit.
Binnen een in het algemeen relatief gezonde politieke arena over vrijwel het gehele politieke
spectrum, is er alleen bij uiterst rechts enige verdachte activiteit waargenomen. Zo zien we rond
politieke entiteiten aan de rechterzijde van het politieke spectrum meer circulatie van
junknieuwsbronnen. Daarnaast zien we alleen bij deze entiteiten enige signalen van artificiële
8

activiteit. Ook wanneer we kijken naar volgers zien we dat de politieke partijen en politici aan de
rechterzijde van het politieke spectrum een uniek volgers-signatuur hebben. Zij zijn ook kijkend
naar co-volgers (dit zijn volgers die zowel een nieuwsbron als een politieke partij of leider
volgen) relatief nauwer verbonden met junknieuws en politiek sterk gekleurde nieuwsbronnen
dan partijen en politici elders op het politieke spectrum. De artificiële activiteit die we hier
aantreffen is in lijn met het schandaal in 2015, wanneer na verwijdering van nepvolgers op
Twitter bleek dat ook in de Nederlandse politiek van dergelijke artificiële amplificatietechnieken
gebruik was gemaakt.
Reddit en 4chan: YouTube video’s als nieuwsbron dragen bij aan polarisatie
Hoewel Reddit en 4chan vaak worden beschreven als alternatieve zones op het web, laten de
resultaten van onze studie van deze platforms geen groot aandeel van alternatieve
nieuwsnetwerken zien die desinformatie verspreiden binnen deze platforms in de Nederlandse
context. Naast een paar voorbeelden waarin de pro-Russische site novini.nl een rol speelt en
verdachte activiteiten vanuit één Reddit account, zien we hier geen gecoördineerde campagnes
van kwaadwillende gebruikers die links posten naar desinformatie. Er is zeker junk content te
vinden, maar vergeleken met de algehele activiteit blijft het aandeel daarvan marginaal. Met name
Reddit lijkt goed weerbaar tegen de circulatie van desinformatie.
Wat wel uit het onderzoek naar voren komt zijn de verschillende typen junknieuws van politiek
sterk gekleurde aard, die met name op 4chan/pol/ zijn aangetroffen. Dit zijn hoofdzakelijk links
naar tendentieuze en politiek sterk gekleurde bronnen als The Post Online en De Dagelijkse
Standaard, maar ook complottheorie op NineForNews. Deze rechtse bias was te verwachten op
4chan/pol/ dat bekend staat als uiterst-rechtse hub, maar voor Reddit lag het minder voor de
hand omdat we daar een politiek diverse set sub-Reddits als startpunt hebben genomen. Hoewel
het problematisch is om nieuwsbronnen als “nep” te bestempelen, treffen we op die bronnen
wel degelijk polariserende content aan. De artikelen die de meeste gebruiksinteractie vertonen
gaan dan ook vaker over onderwerpen als migratie en de Islam dan over geopolitieke events als
Russische inmenging of MH17.
Desalniettemin blijft het vooral populair om te linken naar gevestigde nieuwsbronnen als
NOS.nl, op zowel 4chan/pol/ als Reddit. Dit gaat tegen de verwachtingen in, gezien het “fringe”
karakter dat vaak wordt toegeschreven aan deze platforms en hun anonieme gebruikers waarbij
ook het verwijzen naar marginale of alternatieve kennisbronnen kan worden verwacht. Onze
bevindingen weerleggen deze aannames echter en laten zien dat mainstream bronnen minstens
enige autoriteit genieten op deze online platforms. Belangrijk hierbij op te merken is dat wij niet
hebben onderzocht hoe de mainstream websites zijn besproken. Zo kunnen we verwachten dat
NOS.nl op Reddit wordt behandeld als een betrouwbare bron, terwijl deze op 4chan/pol/
wellicht enkel wordt geridiculiseerd of zelfs bestempeld als “nepnieuws”.
Tenslotte is het belangrijk om reguliere nieuwsbronnen niet te beschouwen als de enige bron van
nieuwsvoorziening op het web. Zo hebben we kunnen vaststellen dat YouTube is uitgegroeid tot
een bijzonder grote speler in nieuwsconsumptie en -circulatie, met name op 4chan/pol/. Op
Reddit vormen “News & Politics” video’s op YouTube de op een na belangrijkste bron, terwijl
ze op 4chan/pol/ waarschijnlijk alle andere nieuwsbronnen ver achter zich laten. Uit een kleine
9

exploratieve studie van de YouTube kanalen die kunnen worden aangetroffen op beide
platforms blijkt dat de Nederlandse Reddit vooral naar gevestigde bronnen verwijst, zoals
PowNed, Zondag met Lubach en NOS, terwijl op 4chan/pol/ juist aan alternatieve,
buitenlandse en politiek sterk gekleurde informatiekanalen wordt gerefereerd zoals aan Mike
Cernovich en RT. Als deze trend zich doorzet zou niet-Nederlandse YouTube content weleens
een polariserende of zelfs radicaliserende werking kunnen hebben op Nederlandse gebruikers
van dit platform.
Beleidsthema’s: nepnieuws als morele paniek en de bedreiging van de mainstream
In de academische literatuur houdt een kleine maar groeiende groep wetenschappers zich bezig
met de vraag hoe nepnieuws kan worden beschouwd als een morele paniek (Morozov, 2017;
Hirst, 2017). Die term verwijst naar momenten in de geschiedenis wanneer “weldenkende
mensen” (in de klassieker van Stanley Cohen gedefinieerd als “editors, bishops and politicians”)
een maatschappelijke conditie waarnemen die een plotseling verval van maatschappelijke
standaarden en waarden veroorzaakt (1972). Wanneer nepnieuws door die lens van een morele
paniek wordt beschouwd, rijst de zorg dat de traditionele journalistiek als steunpilaar van de
democratie thans wordt uitgehold door sociale media en daar vervangen door clickbait van lage
kwaliteit en politiek sterk gekleurde commentaren, die op zo’n wijze worden verpakt dat de
consumptie ervan doet denken aan die van ongezond junkfood (vandaar de term: junknieuws).
De algehele gezondheid van de media als sociaal weefsel staat op het spel, omdat burgers die
sociale media gebruiken als bron van politieke informatie worden benadeeld in de mogelijkheden
om een oordeel te vormen over sociale kwesties en politiek in algemenere zin (Carlson, 2018).
Een tweede subset in de literatuur beschrijft hoe de media-aandacht voor nep- of junknieuws, en
in het bijzonder de relatie daarvan met de groei van een uiterst-rechtse media-ecologie, hier juist
‘zuurstof’ aan geeft (Phillips, 2018). Philips roept op om te stoppen met aandacht geven aan het
maar al te fascinerende subculturele milieu waarin extreemrechts zichzelf cultiveert, en juist
aandacht te schenken aan de slachtoffers van deze praktijken. Er zijn voorbeelden van politici die
extremistische of verdeeldheid zaaiende content circuleren, juist omdat ze het sterk verwerpen en
openlijk willen bekritiseren, en daarmee toch die materialen ook zuurstof geven. Door het te
delen dragen zij onbedoeld bij aan de circulatie van het extreme materiaal en wellicht zelfs aan de
normalisering ervan. In lijn van deze literatuur, bevatten de beleidsaanbevelingen aandacht voor
het identificeren van en reageren op de bedreiging van de mainstream, zowel vanuit sociale
media platforms als de journalistieke praktijk, online content creatie en politiek leiderschap.
Uit onze studie blijkt dat er bepaalde platforms en onderwerpen zijn waar de bedreiging van de
mainstream acuter lijkt dan bij andere platforms en onderwerpen het geval is. Hoewel het geen
platform is waarop Nederlandse junknieuwsbronnen worden verspreid, is de Nederlandse 4chan
een incubator van extremistisch sentiment, vooral waar het gaat om antisemitisme en antiimmigratie. De polariserende content is daarbij veelal afkomstig van YouTube. Andere platforms
zijn om andere redenen problematisch. Nederlandse politieke arena’s op Facebook en Twitter,
die je kunt afbakenen met zoekopdrachten naar politici, politieke partijen en maatschappelijke
kwesties, hebben de grootste hoeveelheden junknieuws, maar het engagement met die bronnen is
op die platforms nog altijd lager dan de algemene consumptie van mainstream nieuws. Binnen
het junknieuws zijn politiek sterk gekleurde bronnen populairder dan desinformatie,
10

complottheorie of clickbait. Voor polariserende onderwerpen zoals klimaatverandering, MH17
en Zwarte Piet behalen verhalen van dergelijke junknieuwsbronnen soms meer engagement dan
verhalen uit de mainstream pers, omdat de aandacht in die junknieuwsbronnen stabiel blijft in
tegenstelling tot het mainstreamnieuws.
Op Twitter bijvoorbeeld, tijdens de Europese Parlementsverkiezingscampagne, deed een proNexit verhaal in De Dagelijkse Standaard, over het uit de EU treden van Nederland, het beter
dan een artikel over datzelfde onderwerp in NRC Handelsblad. Ook zijn er sporen van een
gepolariseerde media ecologie te vinden. Op YouTube is een alternatieve (rechtse) mediasfeer
ontstaan, waar extreme YouTubers (‘micro-celebrities’) de scepter zwaaien. Instagram heeft ook
een rechtse mediasfeer, die in de analyse gedetecteerd kon worden door gedeelde volgers van
rechtse politici en sterk politiek gekleurde mediabronnen. Deze zijn vooral te typeren als ‘alt lite’,
dat wil zeggen anti-establishment en anti-politieke correctheid, met content die ook als counterjihadist en anti-Islam gezien kan worden. Er zijn geen linkse equivalenten hiervan gevonden.
Om onderzoek te kunnen doen naar polarisatie, opruiing rond maatschappelijke kwesties en de
publicatie, verspreiding of consumptie van extremistisch materiaal, is toegang nodig tot data van
sociale mediaplatforms. Dergelijke platformdata moet beschikbaar worden gemaakt aan
onderzoekers en waakhond-organisaties, zowel middels APIs als in publieke
onderzoeksarchieven, waar ook verwijderde materialen beschikbaar in zouden moeten zijn
evenals metadata omtrent de verwijdering.
Aanbevelingen
In tegenstelling tot de situatie in andere landen tijdens de Europese parlementsverkiezingen, is in
onze studie geen buitenlandse desinformatie gevonden maar wel zogenaamd junknieuws, met
name rond bepaalde kwesties, zoals Zwarte Piet, MH17, het klimaat en de Europese Unie (Peel,
2019). Ook bij het onderwerp nepnieuws, hier bestudeerd als kwestie, was dit het geval. Hoewel
er een flinke hoeveelheid junknieuws werd aangetroffen, deed deze het in het algemeen niet beter
dan mainstream nieuws. De grootste hoeveelheden junknieuws circuleerden niet rond algemene
thema’s zoals politieke partijen en politieke leiders, maar juist rond specifieke en polariserende
kwesties. Activiteit op sociale media rond deze kwesties vertoonde in sommige gevallen een
duidelijke piek tijdens de verkiezingen of bleef constant gedurende de gehele periode. Hiermee is
de vraag niet langer alleen of er junknieuws aanwezig is rond de verkiezingen, maar ook wanneer
junknieuws zich manifesteert en voor hoe lang.
De onderstaande beleidsaanbevelingen zijn dan ook toegespitst op deze meer specifieke kijk op
het fenomeen junknieuws. In plaats van een algemene aanpak doen wij aanbevelingen die zich
richten zich op het herkennen en monitoren van polarisering van het medialandschap, een breed
maatschappelijk debat omtrent junknieuws toespitsen op polariserende kwesties in plaats van
desinformatie in algemene zin, trainingen ontwikkelen voor beroepsgroepen die zich
bezighouden met het produceren van content, en het mogelijk maken van onderzoek en
mediamonitoring met (thans ontoegankelijke) sociale mediadata. Met deze aanbevelingen richten
wij ons tot beleidsmakers, Ngo’s, waakhonden, onderzoekers, journalisten, mediaproducenten en
partijen in het maatschappelijk middenveld, die zich bezighouden met de bestrijding of
bestudering van desinformatie. Daarnaast zien wij een belangrijke rol voor de sociale media
11

platforms zelf bij het ontsluiten van hun data ten behoeve van onderzoek, monitoring en
archivering.
1)
Mediamonitoring van polarisatie, ofwel het monitoren van de groei en het mainstreamen
van polariserende media met extreme content.
Sociale mediaplatforms zetten software, gebruikers en content-reviewers in om extreme content
te herkennen. Door platforms wordt steeds meer van dergelijke content verwijderd. Maar
historisch bezien hebben sociale mediabedrijven wisselend aandacht besteed aan content
moderatie en is ook de definitie van extreme content niet stabiel gebleken. Daarmee blijft het
wenselijk om onafhankelijke monitoring te organiseren. Hier zien wij een rol weggelegd voor
academisch onderzoekers, non-gouvernementele organisaties, overheidsdiensten op het gebied
van extremisme en polarisatie, en media-waakhonden. Een training voor media monitoring zou
door partijen in het maatschappelijk middenveld kunnen worden opgezet.
2)
Mediawijsheid-training voor professionele content makers -van journalisten tot digitale
media producenten- zowel op het gebied van online bronkritiek als het omgaan met
polariserende content online.
Nederland kent mediawijsheid programma’s, bijvoorbeeld gericht op senioren of op
schoolkinderen (en via hen, hun ouders). Deze aanbeveling richt zich echter expliciet op het
ontwikkelen van een mediawijsheid-training voor professionele content-makers zoals
journalisten en redacteurs. Dit zou kunnen worden opgezet binnen bestaande of nieuwe
mediawijsheid-programma’s, met betrokkenheid van inhoudelijk experts op het gebied van
journalistiek, digitale mediaproductie en digitale geletterdheid, en gericht op een combinatie van
bronkritiek en het omgaan met polariserende content online (zie ook aanbeveling 3). Een
dergelijk programma is ook relevant voor docenten in hoger en wetenschappelijk onderwijs,
beleidsmakers, en het maatschappelijk middenveld.
3)

Geen zuurstof geven aan extreemrechtse content.

In onze studie zien wij hoe tendentieuze nieuwsmedia ruimschoots circuleren tijdens
verkiezingen en daarbuiten. De artikelen van deze bronnen worden gedeeld en geliket door trolachtige gebruikers maar ook door reguliere nieuwsconsumenten, zoals te zien op Twitter tijdens
de Provinciale Statenverkiezingen. In mindere mate delen en liken trol-achtige en reguliere
nieuwsconsumenten ook discriminerende, antisemitische, misogynische en xenofobische
content.
Het is van belang om tendentieuze en extreemrechtse media niet aan elkaar gelijk te stellen, ook
niet wanneer zij dezelfde standpunten presenteren. Dat tendentieuze media in opkomst zijn en
mainstreamen betekent niet dat evenveel gewicht moet worden toegekend aan extreemrechtse
media. De aanbeveling is om geen zuurstof (als in: aandacht) te geven aan extreemrechtse media
– en daarmee hun content dus niet te delen, liken, reageren, becommentariëren, retweeten, of
(video-)debatteren. Iedere vorm van het delen van extreemrechtse content betekent het
vergroten van diens zichtbaarheid, en levert wellicht zelfs een bijdrage aan het normaliseren
12

daarvan. Deze aanbeveling gaat uiteraard op voor publieke omroepen, maar ook voor
commerciële mediaorganisaties en tendentieuze nieuwsmedia. In plaats van journalistieke
aandacht te besteden aan extreemrechtse subculturen zelf, kan men de blik beter vestigen op de
slachtoffers daarvan (Philips, 2018).
4)
Bij polariserende kwesties zoals Zwarte Piet strekt het de aanbeveling om het
maatschappelijk gesprek breed in te richten met ruimte voor burgerparticipatie.
Het onderzoek suggereert dat polariserende thema’s zoals Zwarte Piet het gehele jaar actief zijn
en niet alleen in het verwachte seizoen. Wanneer we herkennen dat er toenemende polarisatie
plaatsvindt in de maatschappij, biedt dit ook aanleiding om te verkennen hoe
gemeenschappelijke grond kan worden gevonden. Nederland kent rond maatschappelijke en
ethische vraagstukken de traditie van de Brede Maatschappelijke Discussie. Daarnaast zijn er
eigentijdse vormen van burgerparticipatie die in het gesprek rond dergelijke polariserende
thema’s en culturele contestatie kunnen worden ingezet. Hier zien wij een rol weggelegd voor
bottom-up initiatieven en bestaande organisaties die zich bijvoorbeeld richten op het faciliteren
en op gang brengen van een maatschappelijk gesprek en het ontwikkelen of toepassen van
vernieuwende vormen van burgerparticipatie.
5)
Belangenbehartiging van onderzoekers en waakhonden voor het verkrijgen van toegang
tot sociale mediadata en de creatie van onderzoeksarchieven voor content, inclusief inmiddels
verwijderde materialen.
Sociale mediabedrijven dienen toegang te bieden tot hun data ten behoeve van onderzoek,
monitoring en archivering. De kwestie van ‘gesloten platforms’ draait om de mate waarin sociale
mediabedrijven hun platformdata verminderd toegankelijk maken voor onderzoekers,
journalisten en non-gouvernementele organisaties. Als antwoord op zorgen van de overheid
omtrent ‘dark posts’ (advertenties die alleen in iemands nieuwsfeed te zien zijn) en politieke
advertenties zonder duidelijke afzender heeft Facebook recentelijk een politieke advertentie tool
en API gelanceerd. Maar tegelijkertijd heeft Facebook onderzoekers de toegang gedeeltelijk of
geheel ontzegd tot services zoals de Pages API en Graph Search, die veel worden gebruikt voor
onderzoek. Wij zien een belangrijke rol voor de sociale mediabedrijven zelf bij het ontsluiten van
hun data ten behoeve van onderzoek, monitoring en archivering. Overheidsinstellingen kunnen
hier sturend in optreden.

13

THE POLITICS OF SOCIAL MEDIA MANIPULATION

Richard Rogers and Sabine Niederer

The Politics of Social Media Manipulation
Richard Rogers and Sabine Niederer
Introduction: Influence campaigning in political spaces online and the question of
persuasion
In reviewing the scholarship surrounding so-called fake news, one would out of necessity make a
distinction between the dominant work on the art of influence campaigning and computational
propaganda online and the consequences to date for its consumers, but also the few findings,
often journalistic, in the relatively understudied case of the Dutch political space online, both on
the web as well as in social media. Much work has been undertaken on the influence of Russian
(and Russian-style) influence campaigning in the US, and the presence or absence thereof during
elections in Germany, France, Sweden and elsewhere. With respect to the Netherlands, the case
studies have been reserved to the early Russian influence campaigning around the downing of
the MH17 Malaysian airliner (beginning in 2014) and the suicide bombings in the Brussels
airport and metro (2016), discovered through the use of Twitter data sets of Russian trolls, or
influence campaigners. Other work has been performed on the existence of home-grown troll
networks, operating in a predominantly right-wing media sphere, that are at times ‘pro-Russian’
but do not seem to have had foreign input.
Crucially, in the studies and journalistic treatments to date it is increasingly remarked that there
has been a shift in Russian disinformation campaigning from inflaming conflict with the West to
stirring it within the West. It is also argued that disinformation could be said to be ‘Russifying’,
i.e., the borrowing of so-called Russian techniques by domestic actors. The campaigning,
whether foreign or domestic, does more than create narratives that divide; it also employs
computational means to inflate and amplify them through bot work, fake following, astroturfing,
the creation of front groups and other artificial publicity tactics.
It is also argued that more attention ought to be paid to the rise of extreme and divisive media
on social media platforms, where the point is often made that great emphasis is being placed on
foreign disinformation when by comparison it performs poorly in European news spheres. The
growth of ‘hyperpartisan’ news and commentary also may be viewed as an alternative fact or
knowledge infrastructure, contributing to discussions of a post-truth condition and the
contention that established institutions are under threat.
It is of equal importance to examine the critique on persuasion, or the extent to which the
influence campaigning strategies, fakery and hyperpartisan sources have discernible impacts on
their consumers, especially the voters. They appear to be minimal. Indeed, there is a small, but
growing literature critiquing transfer models, also known as hypodermic needle or magic bullet
theories which themselves could be considered part and parcel of the fake news hype and
fascinations with so-called psyops activities such as in the Cambridge Analytica case.1 Transfer
1 The Cambridge Analytica case or scandal refers to the illegitimate use of over 80 million Facebook users’

information to develop micro-targeted advertising (Cadwalladr and Graham-Harrison, 2018). It prompted US
Congressional and UK Parliamentary investigations, and also led to Facebook’s tightening its data access for
academics and public scrutiny more generally (Bruns et al., 2018).

14

models do not take into account the active filtering of media users or phatic sharing, it is argued,
whereby one circulates dubious media more to connect with others or for amusement than to
pass along substantive information. Such models also would discount hardened attitude, and
studies finding that campaigns generally have minimal effects.
As for the measures to be taken, the literature both describes and occasionally questions factchecking and media literacy efforts because of the assumption that corrected information would
assuage knowledge deficits, for attitudes often remain the same. Nonetheless, among the policy
recommendations most frequently put forward are bolstering media literacy initiatives, together
with flagging questionable content, manually and automatically, for further scrutiny. Social media
platforms are facing regulation and are asked to address extreme content and create public
archives.
One aspect of the literature review relevant to empirical work concerns the methods employed
to demarcate political space online for the subsequent study of the scope and impact of fake
news, junk news and computational propaganda – to use some of the terms for the larger
phenomenon under study. Under consideration here are largely mixed (quanti-quali) techniques
and digital methods from media studies and data journalism. These provide distinctive political
space demarcation strategies for the web as well as social media per platform as well as
approaches for cross-platform analysis. They query engines and platforms, measure significant
political stories (in terms of engagement) and determine construals of dubiousness through news
criticism, categorising significantly engaged-with stories into genres such as disinformation,
conspiracy, clickbait, hyperpartisan and (automated) amplification. While often practiced on a
story level, the determination of dubiousness also may be made through source criticism,
according to the extent to which publishers’ output repeatedly accords with junk news
definitions, discussed in the next section. It is also worth studying how signal-based or
algorithmic determinations of fakeness comport with qualitative methods that are based on
source (provenance) criticism.
Fake news, junk news and computational propaganda
Historically, fake news could be thought of as linked to particular novel publishing practices both
“when old media were new” but also nowadays through social media platforms (Marvin, 1988;
Gitelman, 2006). The term ‘canard’, meaning unfounded rumour or story, refers to the contents
printed in the French broadsheets of the eighteenth century; ‘scandal sheets’ are the British term
for the same era of publishing (Darnton, 2010). In the U.S., in particular, ‘fake news’ as a term
recently experienced a revival and travelled internationally, in the numerous senses in which it
has been deployed historically: “news satire, news parody, fabrication, manipulation, advertising,
and propaganda” (Tandoc et al., 2018). As directed towards contemporary social media
platforms, the charge of fake news and similar terms often has been uttered as a lament after the
introduction of new media technologies, where there are concomitant calls for new journalistic
standards, as witnessed with the competing tabloids and their sensationalist, yellow journalism in
the late 1890s and into World War I as well as the radio and newswire in the 1920s (Opper, 1894;
Lippmann, 1922; McQueen, 2018).

15

With the rise of corporate public relations, the blurring of the distinction between the editorial
and the advertisement sent over the wire or into the airwaves prompted the use of the moniker,
“news fakers” (McKernon, 1925; Lazer et al., 2018). Similarly, the contents of the early, unedited
web, populated by self-publishers, and later the blogosphere, were often described as “too fresh
to be true”, given the speed of news production and the potential for those looking for a scoop
to convert unsubstantiated rumor into news (Hall, 2001; Rogers, 2005). More recently, the
notion would be routinely deployed by satirical news sources such as Saturday Night Live! in the
US (Day and Thompson, 2012); in fact, The Daily Show, the progressive comedy news program,
described itself proudly as a “fake news program” (Newman, 2010; Harsin, 2018). Parody, it
should be recalled, was behind the origination of the most circulated fake news story during the
US presidential campaigns of 2015-2016, “Pope Francis Shocks World, Endorses Donald Trump
for President” (Allcott and Gentzkow, 2017). While many definitions concentrate on a story’s
falseness, fake news have the ‘trappings of news’ through the use of the news apparatus,
including the style, formats and containers employed (Laquintano and Vee, 2017; Grinberg et al.,
2019). Indeed, narrower definitions of fake news take as their point of departure how the sources
“falsely claim to be news organizations” (Tucker et al., 2018).
Fake news also has been deployed politically as a barb against the free press when publishing
inconvenient truths, or speaking ‘truth to power’ (Cary, 1955; Darnton, 2017). Since the midnineteenth century, labelling the news media generally and disagreeable reporting specifically as
product of der Lügenpresse or the lying press is a discrediting ploy or even communication strategy,
still practiced today by right-wing social movements as Pegida in Germany, chanting at street
rallies Lügenpresse, halt die Fresse (lying press, shut your mouth) (Beiler and Kiesler, 2018). It was
the German Unwort des Jahres (notorious word of the year) in 2014, in the competition organised
by TU Darmstadt. Fake news is also a label, used in highly conservative political circles in the
US, for particular news sources, notably CNN, MSNBC, The New York Times, and The Washington
Post; the designation is similar, albeit perhaps more extreme, to past portrayals of the agendasetting ‘elite media’ in contrast to conservative upstarts as Fox News (Marwick, 2018; Tripodi,
2018; Peck, 2019). In this respect, one could call the current situation just the latest fake news
scare, or even moral panic (Brennen, 2017; Morozov, 2017).
When discussing the fake news phenomenon in relation to social media and other online
sources, researchers at the computational propaganda project at the Oxford Internet Institute
(OII) often offer the umbrella term ‘junk news’, defined as “extremist, sensationalist,
conspiratorial, masked commentary” (Howard et al., 2017). Other catch-alls include
“problematic information”, “information disorders” and “false news” (Jack, 2017; Wardle and
Derakhshan, 2017). Apart from sensationalist, conspiratorial and masked – features that have
been a part of fake news ontologies for centuries – the OII definition emphasises another
element, extremist, which cuts to the heart of contemporary concern for the phenomenon when
studied not only as a practice of media and public opinion manipulation but also a trigger for
societal unrest.
With respect to the growing anxiety over fake news as harbinger of unrest, one may refer to the
distinctions made between a variety of information disorders, as well as the coinage of new
terminology that captures excitable, Internet-related media and speech (Wardle, 2018). First,
16

disinformation and misinformation are both false, but the latter is unintentionally so, whilst the
former is fashioned for the purposes of intentional disruption and causing harm. A third term,
‘mal-information’ (a neologism), seemingly borrowed from malware or malicious software
categorisations, has been introduced to describe harassment, toxic content and hate speech
online (Wardle and Derakhshan, 2017). These are the tools for the so-called ‘weaponization’ of
social media platforms to foment discord through seeding the news and public opinion with
divisive content. Indeed, ‘extreme speech’ is a term that has been offered as a nuancing of the
hate speech discourse as it is applied to online toxicity. It is meant to capture a form of charged
language and cultural conflict that stops short of hate, and has emerged with social media,
defined as “vitriolic exchange on Internet-enabled media” (Pohjonen and Udupa, 2017). Its rise
has prompted social media companies as Facebook, Twitter and Alphabet (owners of YouTube)
to expand their content reviewer pools as well as widen their internal mandates to identify and
remove more than violence, pornography and hate (Gillespie, 2018). Google also installed a
feedback system for its web search to report inappropriate autosuggestions, after reports of
queries for the ‘holocaust’ autocompleting with ‘is a hoax’ (Solon and Levin, 2016; Hern, 2017).
As with new media technologies of old, social media platforms currently are said to enable the
‘supercharging’ or the acceleration of the spread of fake news (Bounegru et al., 2018). Two terms
have been used to capture the web and subsequently social media as accelerationist media:
clickbait and computational propaganda. Clickbait connotes titillating and sensational content
and is formulaic in its presentation, often containing numbered lists (sometimes referred to as a
‘listicle’) as well as an ‘information gap’ that sparks curiosity, e.g., ‘twenty things you should not
do when visiting Japan’. Facebook, in seeking to identify and downgrade clickbait in its news
feed, defines it as “a posted link with a headline that encourages people to click to see more,
without telling them much information about what they will see” (O’Donovan, 2018). Generally
social media companies seek to operationalise substantive definitions into computational signals.
Thus, to Facebook, brief attention (or short ‘time-on-site’) is a signal of clickbait, for readers,
having been lured in to the “junk food of content consumption”, are subsequently dissatisfied
with the low-quality content, and leave the page quickly (DeAmicis, 2014). Clickbait, often
innocuous, has been combined with divisive content, however (Burger and Schenk, 2019).
‘Extreme clickbait’ was a part of the the story behind the allegedly apolitical Macedonian teens
based in Veles, who used “spammy techniques” in optimising pro-Trump sites to make money,
in the run-up to the US presidential elections of 2016 (Silverman and Alexander, 2016). Followup reporting has sought to debunk that narrative, finding that the clickbait campaign was
orchestrated by political operatives (Wendling, 2018; Silverman et al., 2018).
Computational propaganda, the second term, refers to the “the assemblage of social media,
autonomous agents and algorithms tasked with the manipulation of opinion” (Neudert, 2017).
The breadth of the definition is intended to capture the bots that amplify content, the advertising
platforms that enable micro-targeting and personalisation of influence messaging, and the click
farms that inflate the follower counts and engagement scores, granting posts higher “vanity
metrics” and thus greater symbolic power through fake support (Rogers, 2018a). For
computational propaganda, bots increase the spread or reach of the posts and inflate their metric
counts (Woolley and Howard, 2016). “Low-credibility content” is spread disproportionately by
‘social bots,’ which refer to bots or autonomous agents tasked with influencing discussion and
17

public opinion; such a finding has led to calls for curtailing their use (Shoa et al., 2018). As a part
of the ‘assemblage’ of actors and software practicing computational propaganda, the work of
software-assisted, political operatives has come under scrutiny, especially in the run-up to
elections. Sock puppets, assuming the false identity of a grassroots organiser or a concerned
individual, create and circulate political content, organise events and mobilise audiences, making
interventions in the physical world through hashtags, internet memes and Facebook events
(Mina, 2019). ‘Front groups’ or even faux ‘hashtag publics’ also mobilise followings and organise
demonstrations (see Table one); one notorious case concerned an anti-Islam protest and
counter-protest in Houston, Texas, in 2016, where both groups were mobilised by Russian
campaigners operating under the names of the Blacktivists and the Heart of Texas, respectively
(Shane, 2018).
A related term for fake content insertion for political ends is astroturfing. It is the artificial
seeding of newspapers and other content providers with political (or corporate) advertising
disguised as genuine citizen concern. Such content is a different category than sponsored
political content, where there are regulations that mandate labelling it as ‘paid for by’ a particular
candidate or campaign (Vaidhyanathan, 2017). Nonetheless there have been calls to have
‘masked’ political content unmasked and marked as sponsored, however much in the recent case
of a pro-Brexit group, Britain’s Future, investigative journalists were long not able to unearth the
funding source, despite the transparency of its being labelled.
Particular forms of native social media advertising have prompted the calls for further public
scrutiny of political ads, and also perhaps an expansion of the definition of such. ‘Dark posts’
(aka ‘promoted posts’) on Facebook refer to micro-targeted advertisements, without a referral
page anchoring the content for further investigation (Bump, 2017). Used by political operatives,
including foreign influence campaigners, in the US in 2014-2017 and beyond, such campaigning
tactics assemble ‘keyword publics’ algorithmically by querying the Facebook advertising platform
for words such as ‘second amendment’ or other pro-gun terminology and sending
advertisements to the news feeds of the tens or hundreds of thousands of those users
determined to have such an interest (Angwin et al., 2017). These publics are targeted not so
much because they are persuadable voters but rather to have them circulate and amplify
messaging.
2016 Fake rallies planned and promoted
Date

Fake rally

Location

25 June

March for Trump

New York

9 July

Support Hillary. Save American Muslims

Washington, D.C.

23 July

Down with Hillary

New York

20 Aug.

Florida goes Trump

Several Florida cities

18

2 Oct.

Miners for Trump

Several Pennsylvania cities

12 Nov.

Show your support for President-Elect Donald
Trump

New York

12 Nov.

Trump is NOT my president

New York

19 Nov.

Charlotte against Trump

Charlotte, N.C.

Table one: Overview of 2016 Fake rallies planned and promoted, as listed in the US indictment
of 13 Russian nationals concerning foreign election interference. Source: Parlapiano and Lee,
2018.
Apart from particular social media advertising products such as dark posts, other formats have
been identified as energizing publics with divisive messages. ‘Image macros’ are photos with two
lines of text, one opening and one closing line, that are a popular format for political messaging
on Facebook and have been among the most shared and otherwise most engaged-with content
on the platform (Renner, 2017). Indeed, in the data analysis of the most shared posts of the
‘fake’ (or astroturfing) activist group pages set up by the Russian Internet Research Agency
(Blacktivists, United Muslims of America, Being Patriotic, Heart of Texas, Secured Borders and
LGBT United), the image macros and other meme material scored particularly well (Chen, 2015;
Albright, 2017; Timberg, 2017).
Russian influence campaigning, Russification and the ‘hyperpartisan’ style
‘Dark globalization’ is a term put forward by the historian Timothy Snyder to refer to how
knowledge of western societal problems provides opportunities to influence campaigners from
abroad, or Russia in particular (2018). In the US Snyder refers to the complex of race,
gerrymandering and the electoral college, and the capacity to target voters in specific
geographical areas (such as counties in ‘swing states’) with racialist political messaging that
amplify or provide ‘oxygen’ to viewpoints. There have been detailed analyses of the Russian
influence campaign of 2014-2017 commissioned by the US Congress, both of which benefited
from data provided by Facebook, Twitter and Alphabet (Google) that previously had not been
made available for research (Howard et al., 2018; New Knowledge, 2018). They are a part of a
litany of literature that has appeared since the commissioning by governments to study the
‘tactics’ of the influence campaigners as well as the contemporary art of propaganda and the
development of counter-narratives more generally. These studies also have led to
recommendations concerning how to combat the effects.
The New Knowledge study (and its coverage) emphasise the collective cognitive dissonance that
effective propaganda achieves, introducing (and popularising) language from intelligence and
counterintelligence work. Among the goals of the propagandists is to create “a wilderness of
mirrors”, originally a phrase from a T.S. Elliot poem but mobilised by the intelligence
community (Holzman, 2008). It refers to an environment where truth (and its establishment) are
no longer self-evident (Groll, 2018).

19

To achieve that goal, New Knowledge argues, one particular tactic is the creation of a similarly
termed “media mirage,” or “interlinked information ecosystems designed to immerse and
surround targeted audiences” (2018: 42). They are enveloped in an information “cacophony,”
where stories from the press are repurposed, and given another author (‘blacktivists’),
interpretation and tone. Here is one example, taken from an original newspaper story about how
an “11-Year-Old Texas Boy Invents Device to Prevent Hot Car Deaths” (Dahlgren and Arkin,
2017). It was reworked as follows: “[T]hese are stories of Black children the media don’t want
you to see”; “White people invent tools for killing, this Black child is inventing a tool for saving
lives” (New Knowledge, 2018: 62). The divisiveness and the effectiveness ascribed to the sample
post derives not only from the transformation of the feel-good news story into a contrived ingroup and out-group divide based on race. Note, too, the format used; the second sentence is a
two-liner, to be cast into an image macro or meme, the popular format for sharing and further
circulation of grievance, outrage as well as mockery. The story also brings together categories of
fake news. It is both clickbait as well as extreme content (be it hyperpartisan or junk), as it invites
the consumer to read more about the grievance. It is also packaged to be shared.
The purpose of such campaigning is to sow discord and enmity, but it is only one of a variety of
tactics where the overall goal is to remove a sense of a collective and shared experience of the
world, as analysts have phrased it, and reify group formation (Gessen, 2018). Apart from the
creation of a media mirage, the other tactics listed are as follows: “targeting, asset development,
cross-platform brand building, memetics, inflecting a common message for different audiences,
narrative repetition and dispersal, repurposing and re-titling pages and brands, manipulating
journalism, amplify conspiratorial narratives, sow literal division, and dismiss and redirect” (New
Knowledge, 2018: 2). With respect to social media, as discussed above, targeting could refer to
the audience segmentation available in platforms for advertising purposes, and memetics to the
use of both the image macro to formulate a punchy message as well as to build the meme as an
additive content container for narrative reinforcement.
It is worthwhile to mention that the expert studies are snapshots, but these as well as subsequent
reporting have pointed to the ‘ongoing efforts’ of the influence campaigners, and its global
spread. While social media companies – since the Cambridge Analytica and fake news scandals –
have become more active in identifying and suspending accounts of known Russian and other
state-sponsored trolls (e.g., Iran), similarly named accounts are active and can be traced to known
networks of political operatives (New Knowledge, 2018; FireEye, 2018). New accounts are
continually made (Vaidhyanathan, 2018); the Chief Technology Officer at Facebook speaks of
“blocking more than one million fake accounts every day, sometimes just when they are created”
(O’Brien, 2019). The percentage of influence campaigner accounts in that large number is not
given.
Recently, there has been growing concern not only about the ongoing efforts of Russian
influence campaigners but also the uptake by other groups (or “domestic actors”) of the socalled ‘Russian playbook’ (Frenkel et al., 2019). Journalistic coverage was prompted by the
announcement by Twitter that prior to the US Congressional elections of 2018 it removed
accounts of Americans posing as members of state Republican parties (Harvey and Roth, 2018).
Facebook also announced that hyperpartisan pages on both sides of the political spectrum in the
20

US would be removed. Discussions of the ‘Russification’ of online political campaigning also
historicised disinformation, pointing to the classic examples, such as the claim that the HIV virus
was the leaked product of a US bioweapons lab; it was planted in news outlets beginning in 1983
by Soviet dezinformatsiya campaigners in ‘Operation Infektion’ and ultimately spread four years
later to US TV news (Boghardt, 2009; Ellick and Westbrook, 2018). Comparing the time span of
such a news spread to the dynamics of reach in the hybrid media system nowadays is how one
may describe how the ‘platform press’ has supercharged fake news (Chadwick, 2013; Bell and
Owen, 2017).
In a well-cited article in the New York Times, Facebook, as a leading example of the ‘platform
press’, was described as a “totally insane, unintentionally gigantic, hyperpartisan political-media
machine” (Herrman, 2016). The author spends some time describing the manner in which
Facebook mixes posts in its news feed from both family members and faint acquaintances, but
also discusses the presence of upstart media organizations and self-styled advocacy groups that
only exist online, many only in social media. Most are described as ‘hyperpartisan’. These sources
populating the platform with content are defined as “openly ideological web operations”. They
also are successful, not just because more extreme and sensational content spreads faster than
more sobering truth (Vosoughi et al., 2018). It is also because they employ employ formats that
engage large numbers of users and learn from their engagement and reach. ‘Operating’ in a
continuous feedback loop of metrics data, posts are optimised to perform well in social media.
The performance measures are based on the virality of posts, and those that work well are
emulated. There are particular formats as well as styles that drive engagement. Memes and
clickbait such as listicles, cliffhanger headlines and human interest stories are among the formats
used, as mentioned above. The hyperpartisan style has a variety of substantive features, not all of
which are equally applied, but many appear to be working well. Often anti-establishment as well
as positioned as against or in competition with the truth-seeking and fact-finding of the
mainstream media, the media operations post stories that are alternatives. These alternatives may
be interpretations, facts and editorial commentary on events. They become media layers on the
news. The presentation is often edgy, both in terms of being knowledgeably on trend but also
sharp in tone. The posts are regular, and as such are part of the permanent updating culture,
providing a competing ‘feed’ about what is happening in the world and in media.
The post-truth condition
There is a series of contemporary utterances that have contributed to public discourse about a
post-truth condition. One is the satirical notion of ‘truthiness’ (Colbert Report, 2005).
Developed as political news commentary and comedy, it refers to having the appearance of being
true, but without evidentiary basis. Another – ‘alternative facts’ – is of more recent, political
coinage. It refers to the insistence by a member of the Trump administration that the number of
attendees at the presidential inauguration in 2016 was higher than reported and measured by
crowd science (Still, 2017). The clarification of the meaning behind ‘alternative facts’ is more to
the point: “additional facts, alternative interpretation” (Nuzzi, 2017). Compared to truthiness,
here facticity does not derive from appearance or perhaps authority but rather from other factmaking.

21

In response to what is sometimes considered first-order objectivity battles, or disputes over
matters of fact (Margolis, 1995; Latour, 2008), newspaper reporting with such headlines as “Here
Are the Real [Facts]” as well as the work by fact-checking bureaus and initiatives are contesting
fact claims with increasing urgency (Fandos, 2017). These are public debates about facts, inputs
into which include fact-checking, a common practice of journalists and university research
groups seeking to confirm the basis behind particular statements by politicians and others
(Graves, 2016). Recently, scholarship on the effectiveness of fact-checking has developed in at
least two directions: the extent to which fact-checking corrects the record as well as factual
beliefs, and whether it changes attitudes (Barrera et al., 2017). Both are part of the decades-long
discussion and critique of the ‘information deficit’ and ‘diffusion’ models, which challenge ideas
that providing correctives clears up controversies (Wynne, 1991; McNeil, 2013).
In the fake news scholarly discourse, it has been found that there are distinct audiences for
‘alternative facts’ and ‘fact-checked facts’ (Bounegru et al., 2018). Whilst there may be a
correction to the record, the original audience may not have been exposed to it. Fact-checked
stories also have similar circulation patterns to alternative facts; they are forwarded to likeminded audiences (Shin and Thorson, 2017). Though it does not tell the entire story about
exposure, both the original as well as the fact-checking publications are outlets with distinctive
audiences or subscriber bases, with fact-checking newsletters often with smaller, specialty
circulations, though their visibility may increase as they are built into the Facebook interface. In
the other strand of work, it is asked, does exposure to fact-checked facts change factual beliefs as
well as attitudes? Here one set of findings is in keeping with the critiques of the effectiveness of
fact-checking and the information deficit model more generally, for respondents saw their factual
accuracy improve, but their attitudes remain unchanged (Nyhan et al., 2019). Fact-checking,
however, could be understood as a documenting process that corrects the record by capturing a
dubious story and committing it, and its debunking or exposure, to searchable databases and
other media.
The post-truth condition, though, has been described as a competition with respect to not firstorder but second-order objectivity. In such a circumstance there is a rise of competing regimes
of truth (Fuller, 2018). Expertise becomes ‘sectarian’ (Turner, 2001). The idea of the media
mirage (evoked to describe effective disinformation campaigns) does not in itself create a
competing truth regime or infrastructure. Rather, it introduces noise into an infrastructure. But
when propagandists, or in a different reading of the contemporary situation, a right-wing media
ecology, create an alternative news and information infrastructure, those efforts fit with
descriptions of the post-truth condition (Benkler et al., 2017; Sängerlaub et al., 2017).
In other words, post-truth is a term that should not be construed as signifying hoodwinked (or
radicalised) consumers, or the ‘wholesale cheapening’ of fact-making (Sismondo, 2017). Rather,
in asking whether “we can have our facts back”, the debate concerns whether (or when) publics
can agree on the ‘facticity infrastructure’ or even the modernist project of knowledge institutions
(Marres, 2018). As a case in point, there are ideologically distinctive alternatives to Wikipedia
(such as Infogalactic, Metapedia and Conservapedia), producing encyclopaedias challenging not
only what is known or settled fact, but also the sources rooting it (Fitts, 2017).

22

Elections, disinformation, and the Dutch case
Three recurring topics are often discussed in the news and (commissioned) research on
disinformation and fake news in the Dutch context. First of all, of particular concern are Russian
trolls and their spreading of disinformation in the Netherlands. Secondly, there are the (nonRussian) fake accounts and fake fans that that inflate the popularity of a campaign or a
prominent figure, granting them greater symbolic power. And thirdly, publications are addressing
its discernibility and possible countermeasures. How to recognise it and combat it? Each of these
discussions is often set against the backdrop of a changing news media landscape, whereby
mainstream news is increasingly competing with more tendentious and hyperpartisan outlets, and
digitization is leading to user-driven and algorithm-driven personalization. That may narrow the
horizon of news that users encounter and perhaps increase fringe consumption, though in
empirical studies such has not been found (Wieringa et al., 2017). Comparisons of the Dutch
situation are also drawn with the US.
While digitization may be changing how people consume news, a study of online news behavior,
disinformation, and personalization of the news by Rathenau Institute stresses that in the
Netherlands, the traditional news media still hold a firm and stable position in the media
landscape (van Keulen et al., 2018). The study also finds that there is not (yet) widespread
algorithmic personalization in Dutch media sites. And, in stark contrast to the current situation
in the US, Dutch news consumers tend to use a variety of sources and have trust in the
traditional news media (and less so in social media). Lastly, the report underlines that the
Netherlands does not have such a particularly polarised media landscape as the US.
Overall, there is a strikingly moderate tone of voice in the literature, both in news reporting and
research reports. Since 2016, several studies have looked at disinformation practices in the Dutch
political landscape, and each of them has concluded that neither is there any large-scale
disinformation activity in the Dutch media nor does disinformation have a significant impact on
Dutch citizens. However, in the Summer of 2017, Wilfred Rietdijk, a Dutch general and national
security advisor, announced in an interview with Dutch newspaper de Volkskrant that the
Netherlands could no longer deal with the digital threat (van Zijl and Modderkolk, 2017). A
“landslide of fake news”, as the subsequent tabloid headline read, would lead the country into
chaos and division (Jonker, 2017). Including breaches and intrusions in his threat assessment
(thereby widening the scope beyond disinformation), Rietdijk explained how Dutch companies
are “in the line of fire” from “thousands of hackers from Russia, China, and countries such as
Iran and even Sudan” (van Zijl and Modderkolk, 2017). The general is not the first to warn of
Russian interference in the Dutch online space, though case studies were lacking, at least in the
public domain.
Russian trolling and its perceived insignificance in the Netherlands
When the Minister of Internal Affairs, Kajsa Ollongren, warned the Dutch government of
Russian disinformation in the Netherlands, she initially was criticised for not having compelling
examples (Pleijter, 2017; Kist and Wassens, 2018a). Two journalistic studies that have looked
into Russian tweets have found activity in the Dutch online realm, however. A study by NRC
Handelsblad mined 200,000 tweets from Russian Internet Research Agency (IRA) accounts and
found disinformation campaigning beginning in 2014 and another spate in 2016. The weekly
23

magazine De Groene Amsterdammer combined the NRC Handelsblad data with larger collections of
Russian troll accounts, made available on the American public opinion analysis website,
FiveThirtyEight as well as the lists published by American Congress (van der Noordaa and van de
Ven, 2018a). Both studies found a peak in trolling activity after the downing of MH17 in July of
2014. The NRC Handelsblad study finds that Russian trolls posted 57,500 tweets, most of which
were in Russian and aimed to influence public opinion in Russia and Ukraine, and only four of
the tweets were in Dutch (Kist and Wassens, 2018b). The study by De Groene Amsterdammer
confirms that most tweets on MH17 were in Russian but finds more mentions of Dutch
“conspiracy theorists and activists”, indicating a shift from challenging Western narratives (for
Russian-speaking audiences) to seeking to stir conflict within the West.
A second event revealed more coordinated Russian troll activity in the Dutch language Twitter
space (in Belgium and the Netherlands), and a further example of striving to foment unrest,
albeit unsuccessfully (according to engagement measures) (van der Noordaa and van de Ven,
2018b). It concerned the spreading of anti-Islam content directly following the terrorist attacks in
the Brussels airport and metro in March 2016, and in the two years after the attacks. This antiIslam ‘campaign’ involved about 950 tweets in the Dutch language that were circulated by some
150 IRA-related accounts. These tweets were rarely retweeted, however. In the event, Russian
trolls are more successful in the Netherlands with the circulation of English-language content.
While these tweets are not related to Dutch issues and focus on for instance the US elections,
they have been shared widely by over 6,000 Dutch Twitter users with a total of 9.5 million
followers (Kist and Wassens, 2018a).
Perhaps counterintuitively, there was only minimal Russian interference with the Ukraine
referendum in the Netherlands in April of 2016 (NOS, 2017). There was the Russian video
capturing fake Ukrainian far-right militia members threatening terrorist attacks in the
Netherlands and burning a Dutch flag, but it was readily recognised as propaganda (Bellingcat,
2016). Otherwise, only a handful of tweets propagating a ‘No’ vote was found in the larger set of
tweets under study (van der Noorda and van de Ven, 2018a).
The NRC Handelsblad concludes its work on the Twitter data set by noting that it is possible
there is larger scale Russian activity in the Netherlands; it should be studied beyond just Twitter
to include other platforms with known troll activity, such as Facebook, Instagram, YouTube and
Reddit. Indeed, especially after Trump’s victory in the US presidential elections of 2016, many
news outlets pointed towards Facebook. As discussed in some detail below, a study by BuzzFeed
News compiled the most engaged-with posts in the nine months prior to the elections and found
that fake news during that time was circulating more than mainstream news. Journalists from the
NRC Handelsblad replicated the study’s general method for the Netherlands, but with a narrower
definition of fake news. They determined that the one hundred most-shared political news
articles from January and February of 2017, in the run-up to the Dutch general elections, did not
contain fake news (Kist and Zantingh, 2017). Certain articles could be considered misleading or
biased, they thought, for they exaggerated news facts or took them out of context. The themes
that were most resonant during the campaign period in the Netherlands were immigration, Islam
and Geert Wilders.

24

Dutch fake followers and trolls
Until November of 2017 much of the reporting has insisted that the Netherlands – and the
Dutch elections in particular – have been largely unaffected by disinformation or fake news.
Much of the news coverage that speaks of it concerns ‘fake followers’. For instance, in 2015,
there was a small scandal about Geert Wilders concerning a dubious increase in his followers on
Twitter. Indeed, when Twitter addressed the issue of fake followers and follower count inflation
through a mass removal of suspect accounts in 2018, Wilders as well as other Dutch politicians
(including from the political party Denk) saw their metrics decline (NOS, 2018). In perhaps the
most well-known case, the Dutch singer-songwriter Dotan was found to have a fake following of
140 user accounts, which were used between 2011 and 2017 to like the musician on social media,
edit the Wikipedia article on the artist, request his songs at radio stations and circulate
heartwarming stories about him across social media platforms. One of the profiles declared how
Dotan’s music helped her through a period of grief after a miscarriage; another tells how Dotan
welcomed one fan’s terminally ill brother in a meet-and-greet, throughout which the singer held
the boy’s hand. Both testimonials were false, as reporters of De Volkskrant found and Dotan
later confirmed (Misérus and van der Noordaa, 2018a; 2018b).
In 2018 the first large-scale global study of computational propaganda was published, examining
organised social media manipulation such as the use of fake followers in 48 countries, including
the Netherlands (Bradshaw and Howard, 2018). The study describes the different computational
tactics employed not so much by Russian influence campaigners but by political parties to
influence voters and the elections.2 It was found that the use of social media as an infrastructure
for the spread of propaganda and disinformation has become widespread. Under examination is
“cyber troop activity,” defined as “government or political party use of social media to
manipulate public opinion” (Bradshaw and Howard, 2018: 9).
While in more authoritarian regimes, social media manipulation fits into larger scheme of voter
suppression and election rigging, in “emerging and Western democracies, sophisticated data
analytics, and political bots are being used to poison the information environment, promote
skepticism and distrust, polarise voting constituencies, and undermine the integrity of democratic
processes” (Bradshaw and Howard, 2018: 5). The tactics described include the use of three kinds
of fake accounts. First, there is the creation of online commentator accounts that attack and troll
genuine users, spread divisive content, or “[divert] conversations or criticism away from
important issues” (Bradshaw and Howard, 2018: 11). A second tactic entails automated accounts
or political bots to automatedly flood particular hashtags, and astroturf by faking a follower base.
The bots also troll genuine users by reporting them and flag organic content thereby having both
suspended until a human moderator checks them. A third tactic is the use of hybrid accounts,
which are those that make use of automation (for the sake of speed and convenience) but are
actively curated by human users, who commonly manage multiple fake accounts or sock
2 The research conducted a content analysis of news articles reporting on cyber troop activity in a sample of 48

countries, supplemented by an in-depth secondary literature review. To collect the news articles, the researchers
used the following keywords in combination, in queries across Google, Yahoo!, Bing and LexisNexis: astroturf*;
bot; Cambridge Analytica; Facebook; fake; fake account; disinformation; government; information warfare;
intelligent agent; military; misinformation; persona management; pro-government; propaganda; psychological
operations; psyops; social media; sock puppet*; troll*; Twitter (2018:8).

25

puppets. This type of fake account is difficult to recognise, and thus to combat. The study finds
that automation is the most recurring tactic, seen in 38 of the 48 countries that were shown to
operate fake accounts.
Besides the use of fake accounts, other strategies involve the use of political ads and the
involvement of search engine optimization and activity on chat applications and across social
media platforms. Where Twitter is proven to be the platform most friendly for automation, the
study finds “cyber troop activity on chat applications or other platforms (Instagram, LINE,
SnapChat, Telegram, Tinder, WeChat, WhatsApp)” in one-quarter of the countries under study
(Bradshaw and Howard, 2018: 13). In the European countries in their sample, they find a distinct
fake news footprint per country. In Germany, junk news is rather marginal and was mostly
circulated by right-wing political actors during the 2017 federal elections. In Italy on the other
hand, a large and active “ecosystem” of it is connected to populist political forces such as the
Lega Nord (Northern League) and the Movimento Cinque Stelle (M5S, 5 Stars Movement),
which were at work during the 2017 constitutional referendum and the elections of 2018. Here,
fake news connects national politics to Euroscepticism, conspiracy theory, aliens and pro-Putin
propaganda. In the Netherlands, this analysis also finds that disinformation revolves around
politician Geert Wilders and in particular the spread of his anti-Islam video, which was broadcast
on television and shared in social media in the lead-up to the 2017 Dutch national elections. In
particular, the study finds that automated accounts have amplified Geert Wilders’ campaign
hashtags.
These results match the findings in a study that looked at troll-like behavior on Twitter, leading
up to the 2017 Dutch general elections, where sock puppets were found (Bounegru et al., 2018).
The study collected over 500,000 tweets mentioning at least one of the Twitter accounts of the
28 political leaders a month before the 2017 Dutch general elections. To retain the users that
demonstrated troll-like behavior, it narrowed down the set to retain only the 25 users who
consistently targeted one or more political representatives.3 The analysis showed that there was a
notable asymmetry in the distribution of targets of troll-like behavior and sockpuppetry across
the political spectrum, where left-wing politicians are most often targeted by negative mentions,
while right-wing politicians receive support. Troll content extended to reputable news sources
which cited it at least thirty times. Among the cited troll accounts were fake news organisations
with names as ‘Today in Syria’ and ‘WorldNewsPolitics’, political parties (including multiple fake
accounts for the Republican party in Tennessee) and concerned citizens, most of whom were
fiercely pro-Trump and anti-Islam (Kist and Wassens, 2017). In another analysis by the NRC
Handelsblad, a Dutch political party (DENK) also exhibited troll-like behaviour, including
sockpuppetry on both Twitter as well as Facebook (Kouwenhoven and Logtenberg, 2017).
While Dutch news consumers have been found to use a variety of news sources, the Netherlands
also has a steady junk news diet (Burger et al., 2019; van der Poel, 2019). From 2013-2017 Dutch
Facebook users consumed more low-quality, commercially driven clickbait than mainstream
news, as was found through engagement scores. As may be expected, there is also relatively more
clickbait on Facebook than quality news.
3 By @mentioning them at least 100 times in a one-month period.

26

The consumption and forwarding of clickbait, extreme clickbait as well as other problematic
information extends also to politicians and public figures. One Dutch researcher, Peter Burger,
has a collection of instances when Dutch politicians have retweeted anti-semitic or otherwise
disturbing content. In one example, a video purporting to show ‘Muslims vandalising Christmas
market in Lithuania’ was actually a recording of an event that took place in Baltimore in the US
(Burger, 2016).

Figure 1. Cartoon that ridicules the fake news taskforce, stating: “internet trolls are best countered by internet
hobbits.” Source: Reid et al. (2018).
Recognizing and countering disinformation in the Dutch online space
Various initiatives aim to detect and counter disinformation in the Netherlands and on an EUlevel. The EU taskforce (East Stratcom Task Force) against disinformation was heavily criticised in
the Netherlands after its project EUvsDisInfo mistakenly categorised articles by The Post Online,
GeenStijl and De Gelderlander as disinformation (van Keulen et al., 2018; Heck, 2018). (Figure 1
shows a cartoon about the fake news taskforce, stating internet trolls are best countered with
internet hobbits.) In a sense the dispute stemmed from misreadings of single stories, perhaps
without an appreciation of how settled some of the sources are in the Dutch media landscape,
despite their tendentious style (in the case of The Post Online and GeenStijl). For its part, De
Gelderlander had taken over nearly verbatim a Russian storyline concerning the perpetrator behind
the downing of the MH17 but did attribute it to its original source in a barebones reporting style.
The flagged cases were removed from the EUvsDisInfo site after complaints by the Dutch
media organization Persgroep (EUvsDisinfo, 2018).
Fact-checking as a journalistic practice has taken hold in the Netherlands. Many newspapers
have implemented (or revived) a fact-checking section, often dedicated to checking statements
made by political figures in interviews in newspapers or TV shows. There are also websites such
as Hoaxmelding.nl and Nieuwscheckers.nl that compile lists of instances of false news on
Facebook and elsewhere. For their study of disinformation, Rathenau researchers analyzed these
lists, comprising respectively 140 on Hoaxmelding (collected between 1 February 2014 and 18
December 2017) and 166 on Nieuwscheckers (between 3 February 2017 and 5 January 2018)
(van Keulen et al., 2018). They found that the items on the list of Hoaxmelding involved
examples of unfounded warnings (65), polarizing disinformation (32) and fake crime news (31).
Additionally, there were several examples of clickbait, benign as well as malicious. The content

27

steers users to advertising, “like-farming” and phishing sites (van Keulen et al., 2018: 38). Such
posts contain human interest stories that are “painful on a personal level” (van Keulen et al.,
2018: 45). The researchers found that only 25% of the disinformation concerned political
content and most clickbait serves a commercial goal, rather than a political one. On the list of
items collected by Nieuwscheckers, the Leiden University-based initiative, less than half was
found to have political content. Within the full set, the researchers found six examples of
polarizing content. Otherwise, many of the posts concern factually incorrect, public statements
by politicians, the investigation of which is how fact-checking is conventionally practiced.

Figure 2. “Detected and eliminated” fake news, with a warning issued by NU.nl and Nieuwscheckers. Source:
NOS (2017a).
Fact-checking now extends well beyond unpacking politicians’ statements, and Facebook has
entered into partnerships with many bureaus around the world, including the Netherlands, to
explore and catalogue dubious content. In 2017 Nieuwscheckers partnered with Facebook and
NU.nl and celebrated their first collaborative “successful detection and elimination of fake news”
that year when they flagged a tabloid-style, human-interest post about an Australian newborn
weighing 20 kilograms (see figure 2). In February of 2019, however, Nieuwscheckers withdrew
from the Facebook fact-checking initiative because of liability risks (Kist, 2019). Nu.nl continues
to work with Facebook on fact-checking, and they do so on a paid basis, an issue raised
repeatedly in the context of journalists’ being asked to address an issue of Facebook’s making on
a voluntary basis.
The effectiveness of fact-checking as a strategy in the Netherlands is a different question. As
mentioned above, fact-checks and fake news often have separate publics, and fact-checks may
lead people to fake news, rather than away from it. A recent study in the Netherlands found that
even when many people would agree with a fact-check, they are not interested in reading the
fact-checking article, prompting the scholars to advise journalists to make the fact checks an
engaging read (Hameleers and van der Meer, 2019). Another strategy to counter disinformation
concerns a strand of media literacy that involves developing skills to recognise fake user accounts
and disinformation. One is on a source level, the other on a story level. The Field Guide to Fake
News provides a method for the detection of trolling accounts by looking at their friends, or their
profile information (Bounegru et al., 2018). There are also courses and training modules for fake
28

news detection and fact-checking, such as those given by Radio Netherlands (RNTC, 2019). The
other format is the fake news quiz, such as those by de Volkskrant (2016) and the Guardian
(2016), as well as the New York Times ‘deceptive Facebook post’ test (2018). These quizzes make
it clear how challenging it is to recognise fake news. The Dutch serious game, titled Slecht Nieuws
(‘Bad News’), invites players to create fake news and by doing so gain insight into the strategies
behind it and become more astute in its recognition (NRC, 2018; DROG, 2018). It is part of
efforts that study fake news as risk and ultimately seek to inoculate populations against it
(Roozenbeek and van der Linden, 2018).
Voting aid applications
Voting aid applications (VAAs), often called stemwijzers in Dutch, are generally websites that
describe their purpose as helping undecided voters find the political party that best matches their
preferences and positions. As such, in the context of the study of fake news, they could be
regarded as a competing persuasion instrument, a preemptive measure against influence
campaigning, or even a potential site that may include it, either through parody, hoax or hack.
The literature on VAAs takes up the Dutch and Belgian cases, countries that together with
Germany, Austria and Switzerland have upwards of half the voter population accessing them
prior to elections. The work can be positioned broadly as pertaining to “the impact of internetbased applications on politics” and can be roughly divided into user studies, impacts of VAAs on
the voters as well as the methods behind them (Hirzalla and van Zoonen, 2015: 88). To date
these online voting aids have not been raised as recommended technology to combat fake news
and influence campaigning per se, though they do furnish a rather personalised information
experience that may be studied for its ‘influence’ effects, as discussed briefly below.
In studies of their usage, researchers have asked whether VAAs “mobilize the mobilized”
(Hirzalla and van Zoonen, 2015). And indeed, while VAAs have a heterogenous user base across
demographics, interests, attitudes and behaviour (Vassil, 2011), there is an overrepresented
subgroup of younger, mainly left-of-center, urban and well-educated male users who are
politically active or knowledgeable. This imbalance could lead to the conclusion that those who
would benefit the most from political advice are the least likely to seek it (Ruusuvirta, 2010).
A second set of literature concerns the impact of VAAs and assesses whether they have
influenced the voting behaviour of its users, though it is not clear whether the quality, reach and
graphical interfaces of the aids affected the extent of the influence. From those surveyed
anywhere from 1% to 15% using DoeDeStemTest (in Belgium) as well as StemWijzer and
Kieskompas (in the Netherlands) reported having been influenced by the aids (Walgrave et al.,
2009; Hirzalla and van Zoonen, 2015). While research has found that the politically
knowledgeable and engaged users that are common to use VAAs perceive them as useful, they
are also among the less likely users to be influenced by them (Alvarez et al., 2014; Dumont and
Kies, 2012).
A third set of literature concerns the methods used by the VAAs. Here there is a distinction
between the choice of the policy positions to include in the interactive system and the models
underlying the advice. The very selection of the policy positions is a crucial factor in the voting
advice given, where another set would lead to other advice (Walgrave et al, 2009). In general,
29

VAAs are found to select policy positions according to their saliency (for the election period),
and variability (in that different parties hold different positions) (Hirzalla and van Zoonen, 2015).
The editorial process differs, where certain VAAs select their statements solely with experts such
as political scientists or journalists (e.g., the Austrian VAA wahlkabine.at), while others co-create
the formulation of VAA positions, workshopping them with party representatives in the case of
the Dutch StemWijzer, or with an editorial board that consists of professional experts as well as
first and second-time voters in the German “Wahl-O-Mat” (Garzia and Marshall, 2017).
As the voters register their political views, and in certain cases add weight to them, the software
calculates the extent to which the voters’ preferences match the respective parties’ and presents
its results as a ranked list, bar chart, grid or radar chart. Several studies concentrate on the
workings and visual outputs of the different voting aids. Louwerse and Rosema (2014) dissect
them by examining how many dimensions are taken into account when ranking the political
parties. In their study, a one-dimensional model refers to the ranking the political parties based
on the level of agreement with the voter and presents its findings as a ranked list or bar chart. A
two-dimensional model places the political parties’ statements and the voters’ responses on a
continuum from left-wing to right-wing and proposes its match accordingly. The more elaborate
multi-dimensional model, employed by the Swiss smartvote application, plots the statements and
responses onto eight policy dimensions and presents its results in a spider plot that is more
complex to read (Louwerse and Rosema, 2014). In a comparative test of these models,
researchers took a dataset from the Dutch Stemwijzer and found that the different spatial
models would lead to very different matches (Louwerse and Rosema, 2014).
As mentioned above, the voting aids are rather popular in a series of European countries and
could be considered not only as another information input but also as one that competes with
campaigning. Though the influence (similar to campaigning) may again be minimal, it could be
considered as another approach or counter-measure in the discussion of how to address the fake
news problem.
Fake news may be pervasive, but is it persuasive?
If one were to divide the current period of fake news studies into waves, it could be argued that
the first related to the definitional issues and the production side (as mainly discussed above),
whilst the second is increasingly concerned with the study of its consumption (Boczkowski,
2016). In other words, fake news may be seen as “pervasive, but is it persuasive?” (Shaw, 1979).
Why do people consume fake news, and do these readers have particular demographics or
profiles? Which people deem these stories credible or at least have pass-along value? Are they
persuaded or even persuadable? In the US and in a growing list of other countries social media
platforms are increasingly a main source of news, and the manner in which they deliver news is
different from a newspaper or similar package or container (Gottfried and Shearer, 2016;
Poynter, 2019). One receives single stories, rather than an entire newspaper, each shared by
someone with whom the social media user has made a connection, most often directly. These
can be friends (Facebook), followers (Twitter), connections (LinkedIn), etc. Stories arrive in the
feeds algorithmically, meaning there is a filtering mechanism where certain of them are boosted,
based on signals such as activity and increasingly trustworthiness, or the amount of given and
measured meaningful engagement between individuals. Put differently, those who are close to
30

the user (by some special measure) are the ones whose stories more likely will be seen (Eslami et
al., 2015). Such observations have led to discussions of the re-application of the notion of the
filter bubble, a term originally associated with a user receiving personalised (rather than
universal) search engine results (Pariser, 2011; van Keulen et al., 2018; Puschmann, 2018).
Personalisation, however, has evolved from being the result of the information interactions of
one user searching to engagement with an entire social network. As such it shifts the bubble
from enveloping the individual to the group; it has prompted ‘bubble studies’ of not just social
media news environments, but those of health, science, fashion and other areas of collective
information production, sharing and recommendation (Pedersen and Hendricks, 2014;
Hendricks and Vestergaard, 2019). Indeed, fake news circulation and consumption are
increasingly experienced as an issue for the environment (e.g., climate change and its sceptics),
health (e.g., the anti-vaccination discourse) and a variety of other areas (Kitta, 2018).
Such findings have led researchers to define on the one hand the groups most likely to consume
and share fake news together with the dynamics of their bubbles, and on the other the meaning,
or sincerity attached to the sharing. In terms of the consumption of fake news, it could be said at
the outset that there have been two widely cited findings about their significance from the
journalistic arena. One found the most shared stories during the US presidential elections were
fake news (see figure 3), and the other that Russian disinformation campaigns had a far greater
spread than previously imagined as well as reported in testimony by Facebook before the US
Congress (Silverman, 2016; Timberg, 2017). These findings have since been put into a broader
context and compared to ‘normal’ political campaigning and the development of messaging
strategies, filtered through news. First, in the event, only a small fraction of the population
consumed such ‘news’ (Allcott and Gentzkow, 2017). Given the limited exposure, the impact, if
at all, would have paled in comparison to political TV commercials (Persily, 2017). There is the
larger question, however, of whether the messaging would have anything but “minimal effects”
(Lazarsfeld et al., 1948). As has been repeatedly found, the net effect of campaigning, albeit by
political elites, that persuades the prospective voter is exceedingly low or even zero (Kalla and
Broockman, 2017). The aim then is less to persuade than to “rous[e] the enthusiasm of existing
supporters” (Panagopoulos 2016).
Though they may have begun as symmetrical studies of the right and the left, of the most
circulated findings to date about the spread of fake news – both with respect to the contents as
well as its consumers – ultimately all have overwhelmingly concentrated on the right, be it
conservatives and the alt right in the US or other right-leaning, populist right or new right
publics in Europe (Bounegru et al., 2018; Benkler at al., 2018). It is of interest to note for starters
that both during the US presidential campaigning and thereafter the information spaces or
spheres of the right contained far more fake, junk, disinformation or otherwise dubious stories
and sources than the left (Faris et al., 2017). Thus, conclusions drawn about right-leaning publics
sharing information should take into account that they are disproportionately exposed to such
information; all else being equal, the right would share more of it (Marwick, 2018). In the
empirical studies it was found that the right (most notably Trump supporters) consumed the
most fake news. However, there seems to be an older, hard core of its consumers in the US
during the run-up to the US presidential elections in 2016 – “the 10% of Americans with the
most conservative information diets” (Guess et al., 2018). These are heavy media users, and
31

‘available audiences’, who have made time to consume media (Nelson and Taneja, 2018). Unlike
the majority of the media-consuming public, they are far more likely to read niche rather than
only establishment sources. There is, in other words, a normalcy to the consumption by those
audiences of fringe materials.

Figure 3. The birth of the fake news crisis, or ‘fake news’ outperforms ‘mainstream news’ on Facebook, in the
run-up to the U.S. elections in 2016. Source: Silverman, 2016.
The strand of work that considers why users share fake news should be prefaced by the
distinction between “earnest and ambivalent” internet users (Hedrick et al., 2018). Much of the
scholarship about internet culture has not considered that considerable cultural production and
sharing are undertaken not to be part of participatory culture, connective action and other
earnest forms of civic culture online but rather for unsympathetic amusement (aka ‘lulz’)
(Phillips, 2015). ‘Sharing’, a term that has mutated in digital culture from acting in a gift economy
to a dominant form of so-called platform capitalism, could have been prompted these days as
much by insincerity as by mindfulness (Barbrook, 1998; Belk, 2007; Srnicek, 2017). That is, the
rationale for making and sharing could go “either way … complicating an easy assessment of
authorial intent” (Phillips and Milner, 2018: 11). Such a tricky attribution of intent is especially
troublesome in the spaces where vitriolic exchange as well as extreme speech and content are
prevalent. It is difficult to disentangle whether one is sharing for amusement and to trigger a
reaction, or for substantive reasons.
As has been found in the US context, the fake news stories most shared on social media resonate
with particular grievances (about the bias of establishment sources) and resentments (concerning
economic opportunity) that underlie certain societal divides (Marwick, 2018). Moreover, the
stories do not stand alone in a mirror world of conspiracy theory but rather are contiguous with
more mainstream conservative news, anchored by Fox News; they are more extreme as well as
32

transgressive in their wording and presentation. Hence the notion of ‘hyperpartisan’, but also
there is also reference made to tendentious, anti-establishment sources. Here the Overton
Window is appropriately referenced, meaning the bounds of current, acceptable public discourse,
and the extent to which extreme speech in hyperpartisan and tendentious sources is moving
established norms (Daniels, 2018).
Fake news studies: Digital methods and data journalism
As we come to shortly, one research strategy for measuring the prevalence of fake news story
types and sources around national elections is to gauge their presence generally in scoping
exercises, but also more specifically in the most engaged-with content in social media concerning
elections, political parties, candidates and social issues. A more subtle analysis would examine the
top stories for the penetration of fake news narratives, measuring mainstreaming. Moreover,
through comparison of engagement with fake news, one also could determine which platforms
are most susceptible (or amenable) to hosting and circulating such content. Facebook in
particular has been held up as a fake news machine (Herrman, 2016). Empirically, it has been
found to host (proportionately) more of it, whether narrowly or liberally defined, than other
platforms (Guess et al., 2018).
In order to contextualise such measures it is worthwhile to consider the ways in which the scale,
reach and impact of fake news have been studied to date with media analysis, or more
specifically digital methods and data journalism. The methods generally could be considered
mixed quantitative/qualitative approaches. They often begin in the journalistic arena, with the
qualitative determination of the dubiousness of a set of sources and/or stories, and proceed with
digital methods that design queries and collect data from platform APIs, media monitoring
company dashboards, and social media companies that have furnished lists of banned trolls or
user accounts. Indeed, with respect to the dubious source lists, Buzzfeed News’ original list of
about 20 sources determined to be fake news inform a series of empirical studies (Silverman,
2016; Allcott and Gentzkow, 2017; Bounegru et al., 2018; Marwick, 2018; Grinberg et al., 2019).
For studies of the Italian news space, the lists relied upon are from BUTAC, Bufale and
Bufalopedia (Fletcher et al., 2018; Butac, 2018). Hoaxwijzer’s list of 92 Dutch-language “false
news” sites also informs certain of the empirical studies to date in the Netherlands (van Keulen
et al., 2018; Wieringa, 2017).4 But other work, such as the NRC Handelblad’s analysis of the extent
of the problem of fake news in the Netherlands in the run-up to the 2017 national elections,
looks at the sources afresh, making on-the-spot determinations of fakeness (Kist and Zantingh,
2017). These may conflict with previous listings. For example, Hoaxwijzer lists De Dagelijkse
Standaard as a “false news” site whereas the NRC Handelsblad did not determine it to be “fake
news”, but the “extreme right news site” fell among those they called “misleading” because it
reported that “1,000 crazy Muslims” had “torched” a church in Dortmund on New Year’s Eve
when instead a firework had landed on its roof causing slight damage. The NRC Handelsblad
determined that it did not meet its definition of fake news as a “fully fabricated story packaged as
news”.

4 As other studies also found, the list is dated; as of April 2019, 40 of the 92 sites are offline. It remains useful as a

list for older media corpora.

33

Indeed, the question of detecting fake news, on a source or story level, is often placed at the feet
of journalists, media organisations and fact-checking bureaus, where credibility and transparency
may be rated (NewsGuard, 2019). Masked sources are penalised, for example. As mentioned
above, for online stories, the determination of dubious content may benefit, too, from a genre
analysis (Lüders et al., 2010). Disinformation, conspiracy, clickbait and (automated) amplification
have styles (Rony et al., 2017). Disinformation tends to be a hard counterfactual presentation,
conspiracy has multiple characters and plot entanglements, clickbait is a cliffhanger that is often
painful on a personal level and (automated) amplification posts at particular intervals and in
coordination, as malicious social bot detection projects have found (Ratkiewicz et al., 2011; Bessi
and Ferrara, 2016; RoBhat labs, 2017). Other technical signatures of dubious news sites are of
interest. For instance, empirical work on the types of cookies and third-party elements in
mainstream and nominally fake news sites found distinctive types in each, with the mainstream
sites using customised trackers and the other off-the-shelf (Bounegru et al., 2018).
With the lists of fake news sources either in place or determinations still to be made, the next
step is to build a media corpus. Following Buzzfeed News’ method, many undertakings query
media monitoring services (such as Buzzsumo, Com Score and Facebook’s Crowdtangle) for
keywords and/or URLs, in order to build source sets of most engaged-with media and pull in
engagement scores per story. Certain of the techniques also include further interpretative coding
of stories, including grievance narratives (Marwick, 2018).
Whilst much attention has been directed towards Facebook, and the study of the election-related
stories most engaged with on that platform, Twitter is often used as the preferred data source,
given dedicated data sets (made available by Twitter or academic researchers) of accounts run by
the Internet Research Agency (Farkas and Bastos, 2018). There is a series of studies that rely on
Twitter’s curated sets as well as on the data robustly collected and shared among data
researchers, such as by Clemson University and FiveThirtyEight, mentioned above. In a form of
crowd science, the publication on GitHub of the Clemson data set led to numerous studies; in
the US widespread disinformation campaigning was found, as is known, but also more nichetargeting of politicians in such states as Maine (Roeder, 2018). As in the Netherlands, discussed
above, the data were put to use in other countries that according to journalistic accounts had
been previously understudied. For example, in Italy IRA trolls posted numerous pro-populist
party tweets in Italian, joining the ‘cacophony’ or media ecology around the right, as also
discussed in the OII work (Fubini, 2018; Fletcher et al., 2018). Twitter is also preferred given the
general ease of use of data collection through its streaming and search APIs, intermediate
services such as Hexagon Crimson for samples as well as the availability of historical data sets,
albeit at a price.
Other approaches (considering consumption and persuasion rather than definition and
production) should be touched on that rely on surveys, user data collection and experiments.
Numerous experiments have been performed on misinformation (Jankowski, 2018). Recently,
for example, a representative sample of the population consents to having their online media
consumption passively monitored, and subsequently surveyed thereafter (Guess et al., 2018). Or,
there are experiments that show fake news to consumers, and subsequently provide fact-checks
to them in order to determine whether the fact-checks should be ‘attitudinally congruent’ for
34

them to be persuasive (Hameleers and van der Meer, 2019). In another experiment in the
Netherlands, commissioned by the newspaper, de Volkskrant, respondents were presented with
fake news around one of four topics: vaccinations and autism, MH17, rape incidents in
connection to migration, or Sylvana Simons (a politician and media personality) and
discrimination. The study tests whether they became less certain about the facts after being
misinformed (I&O Research, 2017; Kranenberg, 2017).
With respect to platforms other than Facebook and Twitter, YouTube and especially Instagram
appear to be relatively understudied but significant, and Reddit and 4chan are being recognised
as breeding grounds for some of the more outlandish and consequential content such as
Pizzagate (New Knowledge, 2018; Tuters et al., 2018). There are platform-specific approaches
for building and analysing datasets for Instagram (through queries for hashtags and place names),
as well as YouTube, Reddit and 4chan (Rogers, 2018b; Rieder et al., 2018). Google web search
also has invited scrutiny, given the extreme content returned for queries such as the holocaust.
Buzzfeed’s work on detecting and analysing fake news on Facebook has been particularly
influential in data journalism research and subsequent studies that build upon it, and thus is
worthy of mention in some detail (Silverman, 2016). First, the researchers built a keyword list
concerning elections (and especially controversial election topics), and subsequently queried
those keywords in media monitoring software (Buzzsumo) that returns stories ranked by
engagement scores. With the aid of the results, they built a fake news and hyperpartisan website
list, which they merged with lists of the same that they curated previously through separate
reporting, including on the infamous collection of about 100 websites created by the Macedonian
clickbait makers, members of the same family of sites (with the same Google Analytics ID) of
WTOE 5 News that created the story about the Pope endorsing Trump, and a collection of
hyperpartisan sites (Silverman and Singer-Vine, 2016; Silverman et al., 2016). They also curated a
list of some 20 mainstream news sites.5 (All the accompanying data Buzzfeed also made available
through online Google spreadsheets, in keeping with emerging standards in data journalism.)
The engagement scores of the top mainstream news and top fake news stories are subsequently
compared. In the first study of this kind and perhaps the beginning of what could be called the
‘fake news crisis’ for Facebook, it was found that the fake news stories outperformed by
engagement scores those from the mainstream news in the three-month period before the US
presidential elections, thereby leading to conclusions about the comparable “power of fake
election news on Facebook” (see Figure 3) (Silverman, 2016). Follow-up reporting has
considered the extent to which such news continues to resonate more on Facebook than
mainstream news stories, despite incipient efforts by the company to curtail its impact. One of
the major studies commissioned by the US Congress found that fake news and influence
campaigning activity on Facebook and especially Instagram substantially increased after the US
elections (Howard et al., 2018).
In April of 2019, some two and one-half years after Buzzfeed News story, we found that only 4
5 Buzzfeed’s list contains the following mainstream sources: New York Times, Washington Post, NBC News, USA

Today, Politico, CNN, Wall Street Journal, CBS News, ABC News, New York Daily News, New York Post,
BuzzFeed, Los Angeles Times, NPR, The Guardian, Vox, Business Insider, Huffington Post and Fox News
(Silverman, 2016).

35

of the 13 top-performing fake news and hyperpartisan websites are still online: World News
Daily Report, Burrard Street Journal, Twitchy and Breitbart. The others appeared to have been
fly-by-night operations, which is another means of considering a source’s dubiousness. That is,
the other 9 sites, including two Macedonian-made ones (Denver Guardian and World Politicus)
and the highest-performing site (Ending the Fed) that spread the ‘Pope endorses Trump’ story
are all gone.
Facebook’s adjustments
After the US elections in 2016, Facebook CEO Mark Zuckerberg initially argued that “the idea
that fake news on Facebook influenced the election in any way, I think is a pretty crazy idea,”
and put forward that such material amounted to a small fraction of its platform’s content (Isaac,
2016). Two years later Facebook’s work to quell fake news, together with its more stringent
policies on (research) data access should be considered here. Addressing the fake news crisis,
there has been an increase in those hired to perform ‘content moderation’, referred to as the
janitors of social media, or even those doing the platform’s dirty work (Roberts, 2017). Facebook
also installed a political ad transparency tool; it lists on the ad itself who has sponsored it, and
there also is a political ad archive and an API (Hern and Waterson, 2018). No longer is the
targeted individual the only one able to view the hitherto ‘dark post’.
With respect to algorithmic changes, in 2018 Facebook began a three-pronged strategy that
would favour “meaningful connections” (family and friends), “trusted sources” (user-surveyed
media) and “local news” in the news feed over more far-flung “businesses, brands, and media”
(Abbruzzese, 2018; Flynn, 2018; Gartenberg, 2018). It should be remarked that these are global
initiatives, coming on the heels of well-reported Facebook-associated riots in Myanmar and Sri
Lanka but also the compilation of compendiums on the effects of fake news on Facebook all
over the world, as the OII’s global study have shown, but also the numerous governmental and
think tank (umbrella) initiatives such as disinfoportal.org.
Whether Facebook’s measures are working in some sense is unclear. The political ad library tool
may show a source, but who is behind it may remain unclear as in the case of a pro-Brexit
campaign group, Britain’s Future (see Figure 4), that spent hundreds of thousands of pounds on
ads in the run-up to significant UK parliamentary votes (Waterson and Hern, 2019). Efforts by
journalists to unmask the source behind Britain’s Future as well as other ‘dark money’
campaigners had for months been in vain (Monbiot, 2019). Significant political ads are also not
in the archive, as ProPublica found, before its tool crowdsourcing Facebook ads and targeted
individuals was purposively rendered inoperable by the company in what it called a “routine
update” that would prevent illegitimate “scraping” (Merrill and Tobin, 2019). Similar tools by
Mozilla and Who Targets Me also broke, thus making the verification work a difficult prospect.
The news feed tweak to boost “meaningful connections” was initially critiqued for its capacity to
exaggerate the importance of fake news, as was observed in Slovakia and elsewhere when
dubious sources saw their engagement scores rise (Frenkel et al., 2018). The prominence of
“meaningful connections” and “local news” in the news feed, according to Buzzfeed, stirred as
well as amplified the Gilets Jaunes protests in France, for their coverage on the local news made

36

the anger groups (groupes colère) and their posts more prominent in the news feeds, as
evidenced by engagement scores from Crowdtangle (Broderick and Darmanin, 2018).
Given the fake news crisis stemmed from the US elections, Facebook also has created specific
initiatives for future elections that would put political parties and their positions on issues in a
single, curated Facebook portal. One of the early projects was for Sweden’s national elections in
2018, which, it was found in a separate study (with Twitter data), suffered from ‘junk news’
quantities second in magnitude only to that surrounding the US elections, and much larger in
fact than such materials around the German, French and Dutch elections in 2017 (Hedman et
al., 2018; Kist and Zantingh, 2017). The Facebook elections project, rolled out in meetings with
social media researchers in 2018, also coincided with their new academic ‘partnership’ project,
Social Science One. It makes available to researchers data sets such as all the URLs that have
been posted to Facebook over the course of a year (King and Persily, 2018). At the same time,
however, Facebook revoked approvals to research software (such as Netvizz and Netlytic) that
made use of its Pages API, sparking academic protest about ‘locked platforms’ (Bruns et al.,
2018; Rieder, 2018). Seen as reactions to the Cambridge Analytica scandal, Facebook’s measures
could be described as curating the datasets researchers can use. The datasets notably do not
include Facebook pages themselves and their engagement scores – data that led to the very
knowledge about the fake news crisis and the scope of the Russian influence campaign in the
first instance (Albright, 2017).

Figure 4. Facebook political ad library tool, results for Britain’s Future, 13 March 2019.
Conclusions: Fakery and campaigning
The fake news phenomenon could be viewed as a revival of previous ones that typically have
occurred when a new media technology is introduced that destabilises production, distribution
and consumption of news and information, as was the case with eighteenth and nineteenth
century broadsheets and tabloids (respectively) but also the radio and newswire of the twentieth
century. The early web and the blogosphere also challenged existing news publication practices
and were considered unedited spaces populated by self-styled authors, providing speedy news

37

‘too fresh to be true’. Now social media platforms disrupt the trustworthiness of established
news and fact and reintroduce the idea of the web as ‘truthless medium’ (Marres, 2018).
The post-truth age, or condition, as it were, may be viewed in light of a conflict between what
counts as ‘fake’ (on a source or a story level), but it has been described rather as a contest
between facticity regimes, or even sets of sectarian expertise. Locating a network of so-called
‘fake news’ websites, for example, could be viewed as the discovery of an influence campaign,
but it just as well can be seen as an ‘alternative facts’ media ecology. When it is a hyperpartisan,
right-wing news ecology, as in the US in the run-up to the presidential elections of 2016, it could
be described as a part of the contemporary post-truth situation, or, as been often related, a
culture war.
Having the ‘trappings of news’ in terms of look and feel, fake news has been defined as
consisting of distinctive types with varying intentionality. For instance, disinformation and malinformation (the neologism) are meant to harm, whereas misinformation may be just as false but
its circulation unintentional. As a case in point, satirical stories and parody may become
misinformation, such as the story about the Pope endorsing Trump, which outperformed (by
engagement score) any other ‘news’ on Facebook during the US presidential election
campaigning in 2016.
In both the public as well as scholarly discourses, there has been a swing from the hype of the
fake news problem (perhaps well exemplified by the Dutch tabloid headline “landslide of fake
news”) to its gradual debunking, e.g., “researchers say fears about ‘fake news’ are exaggerated”
(Ingram, 2019). Such a view has resulted from a series of studies not just on engagement but also
on its consumption, including the rationale behind its sharing. Small, older populations appear
particularly active, as do ‘heavy news consumers’ and ‘available audiences’, or those who have at
their disposal time for fringe news consumption and spreading it among online friends. The vast
majority of news consumption remains of the mainstream sources, however. The evidence that
consumers have been influenced or persuaded is minimal.
Nevertheless, there appears to be agreement that social media platforms remain worthy of study
not only as the new ‘truthless medium’ but for their capacity to accelerate (or ‘supercharge’) fake
news distribution in a hybrid media system comprised of new and established media and media
formats. Despite increased content moderation, automated detection work, and a reorientation
of its news feed principles, does Facebook remain a fake news machine, comparable to the one
during the US presidential campaigning? Indeed, Facebook, at first hesitant to admit an issue, has
taken a series of measures since then that strive to produce more trustworthiness, such as
boosting posts by friends and family, crowdsourcing trusted sources as well as favouring local
news, though the effectiveness of these reengineered principles has been questioned. Indeed,
continuing empirical research on the most engaged-with, political news on Facebook could shed
light on the quality of the platform’s content delivery, however much data access may be
restricted to researchers. It remains to be seen how ‘oversight’ research will be affected when
Facebook closes research APIs and instead curates data sets for researchers, rather than allowing
them to create their own. Other oversight projects have been thwarted; in early 2019 Facebook’s

38

“routine update” blocked the software by ProPublica, Mozilla and Who Targets Me that was
collecting political ads and their targets, as mentioned.
The question of fake news as a campaign strategy – be it by Russian operatives, Russified
domestic actors, hyperpartisan or tendentious right-wing media-makers, and others – also has
been meticulously studied, with detailed ‘playbooks’ laid bare as tactics to create both a media
mirage (where fact and fiction are difficult to disentangle) as well as competing truth regimes,
offering counter-expertise as well as uncertainty. Governments around the world have
commissioned studies, revealing the breadth and scope of the problem, explaining the playbook
and putting forward policy recommendations such as increased media literacy and the regulation
of political advertising on platforms, including ‘dark’ posts. Platforms are asked to create public
archives, which also would benefit research as well as (data) journalism. Fact-checking also has
gone global, though it often remains a small-scale enterprise practiced by bespoke bureaus,
occasionally working in tandem with Facebook, checking posts that have been flagged by users,
and weighing in on the question of fakery.
Finally, there are scholars in the US and recently in Europe putting forward the argument that
studying Russian disinformation shifts the attention away from the home-grown hyperpartisan
news ecologies that have been emerging over the past few years, particularly on the right
(Benkler et al., 2017; Benkler et al., 2018; Rone, 2019). The point also fits with the ‘dark
globalization’ argument concerning how existing domestic divisions, displayed in this media, may
be exacerbated by foreign operatives but are not created by them. To date the effectiveness of
Russian influence campaigning in Europe, in either sowing or exacerbating division, has yet to be
compellingly demonstrated; the false and junk domestic news sources (e.g., the pro-Russian
sources re-narrating the cause of the downing of MH17) also appear to have scant reach
(Fletcher et al., 2018). In a climate of heightened sensitivity towards dubious sources and stories,
it remains to be seen whether they have staying power.

******
Appendix
Governmental efforts and discussions of countermeasures
A first step for many national governments and other regional political entities that wish to
counter disinformation is to install committees as well as task forces; it occurs across the globe,
from the much publicised hearings by the US Congress and UK Parliament on the Russian
involvement in the US elections and the Cambridge Analytica affair, to the task forces and other
entities formed in many of the nearly 50 countries where influence campaigning has taken place
(Bradshaw and Howard, 2018). Following from these convenings, there have been national calls
to regulate the “digital giants”, and the European Union, through its creation of a High-Level
Expert Group (EU HLEG) on fake news and online disinformation, has issued its
recommendations for countering disinformation, including calls for transparency, media and
information literacy, and tools for empowering journalism. In the European countries with
recent or imminent national elections there has been even greater urgency, with Germany and
39

France enacting legislation (online hate speech and ‘fake news laws’, respectively), and Sweden
and Denmark engaging in awareness-raising as well as media literacy campaigns. Denmark
installed a ‘digital ambassador’ (Gramer, 2017).
Below is a list of certain measures to counteract disinformation and fake news, gleaned from
recent governmental documents and related materials. They include social media company
regulation, codes of ethics, fact-checking and media literacy campaigning.
Social media company regulation
Many government committees agree that the large tech companies that have come to dominate
the online realm, such as Google, Twitter, and Facebook, should be regulated, but caution overregulation in forms that would curtail expression and press freedoms. The starting point for the
regulation of these companies to counter disinformation is to address political advertising on
social media platforms. It can include the verification of those paying for political advertisements
and disclosing them publicly. Additionally, all social media companies could be required to create
public archives of advertisements so that among other ad types ‘dark posts’ may be studied
(Bradshaw, 2018). In fact, as said, Facebook has such an archive (and an API), but it also
prevented watchdogs including Mozilla from verifying its collection techniques, equating their
methods with illegitimate data “scraping” (Merrill and Tobin, 2019).
Relatedly, the EU HLEG proposes the development of a ‘European-wide code of practices’ that
describes the roles and responsibilities of relevant stakeholders such as tech companies, and
media organizations but also research organizations and fact-checking initiatives, based on key
principles (2018). In short, they address the adaptation of political advertising policies (including
sponsored advertisements and other forms of content), and the provision of access to data for
research and fact-checking. They also propose the installation of advanced settings for users to
customise their user experience, collaboration with news outlets to facilitate users’ access to
trustworthy news, the facilitation of fact-checking and content flagging, and allowing users to
“exercise their right to reply” (EU HLEG, 2018: 32-33).
The UK Parliamentary report on fake news and disinformation speaks in an unusually piqued
tone of the importance of regulating social media platforms and related tech companies, singling
out Facebook as providing the “impression of working towards transparency”, but often
“obfuscating” how well it is capturing and archiving political ads (House of Commons, 2019:
85). Ultimately, they call for establishing an “educational levy” or charge on social media
companies to fund digital literacy as a fourth pillar of the education system after reading, writing
and maths (House of Commons, 2019: 87). There is also a recommendation that social media
companies should develop means to distinguish between those sources regularly furnishing
disinformation and those who do not, in a new system of “content regulation” (House of
Commons, 2019: 87). While carefully worded, that measures can count on the criticism that
similar proposals have faced concerning the restriction of the freedom of expression, while not
being effective measures against hateful or incendiary content (Access Now et al., 2018).
Nevertheless, legislation has been passed. Germany has established a law, NetzDG, that extends
its hate speech legislation compelling social media companies (with more than two million
40

registered users in Germany) to remove such speech rapidly or face hefty fines (Claussen, 2018).
More controversially, France has new legislation which applies to ‘false information’; the law
requires that three months prior to an election ‘false news’ be removed.
Detecting and removing fake content
The Reporters’ Lab at Duke University keeps track of fact-checking initiatives worldwide and
has identified some 160 active initiatives (Duke Reporters Lab, 2019). In European countries,
some fact-checking initiatives are attached to news organisations, but most are operating as notfor-profits (Wardle and Derakhshan, 2017; Graves and Cherubini, 2016). Many work in tandem
with Facebook; as of January 2019 some 50 fact-checking groups, who are party to the
International Fact Checking Network Code of Principles, independently assess fake news flagged
by users (Volpicelli, 2019). The expertise developed includes a variety of flagging and
adjudication systems such as NewsGuard’s “nutrition label” that evaluates some 2,000 online
news sources, or, as it relates, the sites that garner about 95% of engagement in the news sector
(2019).
Automation
Brief mention should perhaps be made of automation as offering methods for flagging dubious
or false content, however much it is rarely recommended in governmental reports. With respect
to fact-checking, if there are shared databases of ‘already fact-checked’ stories as well as sources,
then software could cross-check suspicious ones against those already debunked or evaluated, as
the UK parliamentary report mentions. The discussion concerning the need for human reviewers
for content interpretation and curation remains pertinent.
Counter-narratives
In Germany the government chooses to actively participate in spaces where disinformation is
spread. “On these platforms, the German Government provides both reliable information that
can be fact-checked and a narrative based on this information” (German Federal Foreign Office,
2018). In that vein, rumoursaboutgermany.info is a website for collecting and countering
disinformation about Germany spread by human traffickers. While Germany chooses to work
with counter-narratives, others have criticised this approach. A Canadian intelligence report
argues that developing counter-narratives is a “one event at a time approach” that “fails to
address the source and methodology of information campaigns” (Canadian Security Intelligence
Service, 2018: 66).
Media literacy and digital ‘hygiene’
The EU high level expert group on fake news and online disinformation makes a case for
increased media and information literacy to counter disinformation, which should be
“implemented on a massive scale in school and teacher training curricula” (EU HLG, 2018: 26).
This media literacy also should involve the development of tools and training modules for
journalists. As a particularly relevant method, the group proposes “more powerful tools to be
able to visually map online networks and connections to understand how disinformation is being
created, spread and amplified” (EU HLG, 2018: 28).

41

Some countries speak of ‘digital hygiene’ when referring to media literacy practices, for instance
in France when making a case for the development of skills to assess the validity of the
arguments and the reliability of the source. “This is a public hygiene measure—just as people in
the 19th century learned to wash their hands” (Jeangène Vilmer et al., 2018: 179). In Sweden the
word ‘cyberhygien’ is employed. The Swedish Civil Contingencies Agency has published a
handbook for communicators in public sector organizations for the countering of
disinformation, which includes strategies that range from source checking and recognizing a bot
to choosing an appropriate response to disinformation. The Swedish Media Council developed a
media literacy programme for young people, teaching them critical thinking and disinformation
detection; it includes a set of educational materials on ‘source criticism’ (‘Källkritik’)
(Government Offices of Sweden, 2017; Swedish Media Council, 2019). Several recent reports
stress the importance of better equipping journalists with tools and skills to recognise and avoid
disinformation, mentioning the importance of fact-checking, critical source assessment and
ethics (Jeangène Vilmer et al., 2018; Wardle and Derakhshan, 2017).
Investing in civil society and building public trust
A more general way forward that is presented in the literature is to invest in civil society, as it
“must remain the first shield against information manipulation in liberal, democratic societies”
(Jeangène Vilmer et al., 2018: 169). Such initiatives are specifically relevant around events such as
elections, in which civil society can be supported through non-legislative, pre-emptive measures
and multi-stakeholder collaboration of government with the industry, non-governmental sector,
and regional actors (Haciyakupoglu et al., 2018). In Sweden, the aforementioned Swedish Media
Council is an example in which politicians and media professionals collaborate and meet
regularly to discuss and counter disinformation and related challenges. Such regular, multistakeholder consultation both within and across European countries is among the
recommendations often given (Brattberg and Mauer, 2018).
Guaranteeing participation in public debate by all
Lastly is the admonition issued in the 2017 joint UN declaration on “fake news” that emphasised
the need for states to enable the participation of all in public debate. They should ensure that any
efforts to quell or thwart the practices of fake news-making and spread as well as that of
disinformation be handled within the context of the freedom of expression and the freedom on
the press (McGonagle, 2017).

42

JUNK NEWS ON FACEBOOK
DURING THE 2019 DUTCH ELECTIONS

Stijn Peeters and Richard Rogers

Junk news on Facebook during the 2019 Dutch elections
Stijn Peeters and Richard Rogers6
Introduction: Facebook
Since 2016 online disinformation and so-called fake or junk news have been virtually synonymous
with social media platforms, serving as their most significant conduits. The 2016 U.S. presidential
elections and the British Brexit referendum of the same year opened a period of increased scrutiny
of these platforms in how false or misleading information are published and amplified. Facebook,
the single largest social media platform of the past decade, has been an obvious focal point. It has
been the subject of a substantial and growing amount of studies that investigate its “challenge [to]
journalism” (Johson and Kelling, 2018: 817), the persuasiveness of fake news shared on it (Allcott
and Gentzkow, 2017) and the prevalence of it in the average user’s Facebook practice (Guess et
al., 2019).
One of the first well-publicized reports on this topic, and the one that informed some of the
subsequent research, was BuzzFeed News’ 2016 story on the prevalence of fake news in the three
months leading up to the presidential elections that saw Donald Trump elected the 44th president
of the United States. The report, entitled ‘This Analysis Shows How Viral Fake Election News
Stories Outperformed Real News On Facebook’ (Silverman 2016), outlines user engagement
with ‘fake news’, finding that in the last few weeks before the election it was engaged with more
often than mainstream news.
Following this piece and other coverage on the prevalence of fake news on its platform,
Facebook repeatedly announced initiatives that were ostensibly intended to prevent it from
happening again by employing third-party fact-checking organisations (Mosseri 2017a), giving
“more informative” content higher priority (Mosseri 2017b), providing more information about
the authors of news content (Hughes et al., 2018) and increasing content moderation. Despite
these changes, a few years after the 2016 U.S. elections the platform has still repeatedly been
found in studies to be spreading problematic content. It has been criticised because of its role in
spreading false and hateful content about minorities in Myanmar (Fink, 2018), live streaming the
2019 Christchurch mass shooting (Shead, 2019) and in inciting religious hatred in Bangladesh
through viral content that is misleading (Haque et al., 2018: 1). In an analysis of social media use
around the Mexican presidential elections in 2018, however, only “limited evidence of junk
content on [Facebook]” was found (Glowacki et al., 2018: 4). Similarly, a 2017 analysis of social
media usage by Dutch political parties found scant “dubious” content shared by Dutch political
Facebook pages (Wieringa et al. 2017, p.60), though their focus was Facebook pages associated
with political parties rather than a larger Dutch Facebook sphere.
Facebook therefore remains an interesting object of study. It is both the platform most
commonly associated with fake news as well as one that, at face value, has been relatively
proactive in deploying initiatives against its spread. Additionally, existing literature is inconclusive
with regards to the extent to which these measures have been effective, and there seem to be
6 The research reported here was undertaken in collaboration with Tim Groot.

43

significant regional differences in the penetration of fake news in the discourse on the platform,
and its effects. There is some existing research focused on the overall Dutch media sphere, most
notable a study on fake news during the 2017 Dutch parliamentary elections by the NRC
Handelsblad, the national newspaper. The NRC Handelsblad notably found little evidence of the
phenomenon; however, as both Dutch politics and Facebook’s platform have undergone
changes since then, the two Dutch elections of 2019 – the provincial elections (provinciale
statenverkiezingen) and the EU Parliamentary elections – present a useful case study through
which one may investigate the extent to which disinformation and fake news in a broader sense
play a role in this particular geographical context on the platform, three years after the 2016 U.S.
elections, and two years after the previous major national Dutch elections.
While ostensibly regional in character, the Dutch provincial elections nevertheless have a “strong
national component7“ (Hietbrink and van Voorst, 2011: 6) as they additionally determine the
make-up of the Dutch senate, which is indirectly elected by the ‘provincial states’ (provinciale
staten). As such they can serve as a national case study similar to that of the two other major case
studies by BuzzFeed News and the NRC Handelsblad that serve as a kind of baseline for this one.
In addition to provincial elections, only two months later, in May 2019, the Netherlands took
part in the EU parliamentary elections. Given the close proximity of these two elections, and
their different character, they together provide an opportunity to explore disinformation and
fake news in the media concerning Dutch politics.
In the following, we first discuss how their methods may be appropriated for this case study,
through an adapted query list and a more well-defined typology of ‘mainstream’ versus ‘junk
news’ sources, a term preferred over fake news, as we discuss in more detail below. We then
analyse the results in terms of overall trends and a characterisation of the sites found in the junk
news category. By way of wider contextualisation, these findings are further compared with
results found in other case studies contained within this volume. Finally, we offer a
characterisation of the platform-specific and cross-platform trends, and a qualification of the role
junk news plays in Dutch political news coverage.
The BuzzFeed method: results so far
The two aforementioned journalistic analyses that have investigated discourse on Facebook in
the context of national elections serve as a methodological starting point here. These are
BuzzFeed News’ landmark report into fake news in the lead-up to the U.S. presidential elections of
2016, and the NRC Handelsblad’s study of news shared on Facebook around the Dutch
parliamentary elections of 2017, which was inspired by BuzzFeed News’ and to a large extent
employed the same method.
Both of these studies used BuzzSumo, a commercial content aggregation and analysis platform,
to track the most engaged-with articles shared on Facebook in the chosen time period.
BuzzSumo defines ‘engagement’ as a “sum of likes, comments, and shares attributed to an
article” (Lee 2019). If the article is shared in multiple places (e.g., in multiple groups), the
7 Transl. from Dutch: “de statenverkiezingen hadden een sterk nationaal component” (Hietbrink and van Voorst,

2011, p.6)

44

engagement score represents the sum of all engagement that BuzzSumo has gathered from the
platform. After capturing this data through BuzzSumo, both BuzzFeed News and the NRC
Handelsblad categorised the results as of one of two categories, ‘mainstream’ and ‘fake news’. This
simple typology has the advantage of providing clear results, though is potentially limited
through its lack of nuance in terms of distinguishing between disinformation, conspiracy,
clickbait, and hyperpartisan (as discussed in the introduction to this volume), or related terms as
problematic information, misinformation and mal-information.
We adopt this basic method for our case study, but some refining is offered as the original
description could be said to lack specificity in some areas. Particularly, with regards to what
BuzzFeed News considers ‘fake’, the report is somewhat ambiguous, but it does provide the source
list in the form of open data. On the one hand, BuzzFeed News consistently refers to content as
either ‘mainstream’ or ‘fake’/‘false’, implying that all of the content in that category constitutes
articles containing untrue information. On the other hand, their definition of ‘fake’ is somewhat
expansive in the sense that hyperpartisan sites such as Breitbart News are included in their ‘fake
news’ category. Either way, the most engaged with content they found primarily consisted of
such false stories as the Pope endorsing Donald Trump, Hillary Clinton selling weapons to ISIS,
and a fabricated ‘leaked e-mail’.
While the NRC Handelsblad’s study broadly uses the same approach, its method differs in how it
categorises the articles it found. Rather than focusing on disinformation, the NRC Handelsblad
uses a broader category of “news that is taken out of context, strongly politically coloured, or has
a strongly exaggerated headline”8 (Kist and Zantingh, 2017). Approximately 10% of the content
they found fit this description. This would include hyperpartisan outlets, even if they do not
make false claims in their content. Their report notes that very little of the content they found
was actual false news, or consciously misleading, but that approximately 10% of the content they
found fit the broader description. Crucially, even with this broader definition their ‘nonmainstream’ category is far smaller than that of BuzzFeed News’ findings, and thus the NRC
Handelsblad answers its question of whether fake news (‘nepnieuws’) plays a role in Dutch
elections with a resounding ‘no’. In spite of these different outcomes, in different contexts, both
studies follow the same basic methodology of extracting results from a number of relevant
queries from BuzzSumo, which we follow here.
BuzzFeed News’ method, as described in their report, is relatively straightforward: a list of queries
is prepared, engagement for articles matching these articles is extracted from Facebook (via
BuzzSumo), the results are aggregated and divided into three-month periods, results are coded as
either ‘fake’ or ‘mainstream’, and the relative prevalence of both categories is plotted over time
(Silverman 2016). More practically, this data was collected by BuzzFeed by querying BuzzSumo
for a number of thematically appropriate queries. While no full query list is given, the examples
include names of election candidates ([“Hillary Clinton”] and [“Donald Trump”]) and phrases
reflecting topics of debate during the campaign, such as [Clinton AND emails]. They also
included a number of “known viral lies” such as [Soros AND voting machine]. It should be
8 Transl. from Dutch: “nieuws [dat] uit zijn context werd gehaald, sterk politiek gekleurd werd gebracht of werd

voorzien van een sterk aangezette kop” (Kist & Zantingh 2017)

45

noted that the latter inclusion is somewhat asymmetrical for it means the search for more
sensational and divisive subject matters is more precise and targeted than the search for
mainstream news topics, thereby seeking fake news. In any case, the question of asymmetry is
addressed in the case study at hand.
Query design: descriptions, issues and party leaders
Dutch provincial elections
We follow BuzzFeed News and the NRC Handelsblad in their general method in terms of query
design, querying BuzzSumo in order to find the most engaged-with content on Facebook. We
compiled a list of queries to search BuzzSumo following BuzzFeed News’ approach of mixing
names of political leaders with issues that were particular to the given election campaign. This
method also was used by the 2017 NRC study which queried “words like ‘elections’, ‘parliament’
and ‘polls’, and/or the name of a party, party leader, and/or widely discussed topics such as
‘health care’, ‘pensions’, ‘immigrants’ and ‘EU’”9 (Kist and Zantingh, 2017). We used the NRC
list as a starting point and adjusted it to fit the provincial elections rather than the national
elections they studied.
A complication here is the dual local/national focus of the elections. While candidate lists differ
per province, in televised debates, national rather than local party leaders participate, and they
can generally be said to dominate media coverage (though some local broadcasters organise their
own debates as well). In terms of media coverage, local leaders are simultaneously more
numerous (as there are far more local leaders than national leaders) and much less significant (as
news coverage and debates concentrate on national leaders). A national focus additionally was
particularly apparent in the 2019 elections as polls indicated the cabinet risked losing a senate
majority following the elections (Herderscheê and Meijer, 2019). For this reason, we limited our
party-based queries to the last names of the political leaders of the parties that currently
constitute the Dutch parliament,10 as well as the name of the Prime Minister, representing the
national government.11
Additionally, we queried a number of political issues that were topics of debate during the
election campaign. We looked at the manifestos of the larger Dutch parties and chose three
themes that were both significant across all parties’ manifestos and had been the topic of media
coverage during the ongoing campaign: [Klimaat] (climate), [Migratie] (migration), and [EU].
Finally, we queried two further general keywords, [verkiezingen] (elections) and [PS2019], a widely
used hashtag and shorthand for the elections at hand.
The queries were undertaken to capture the election campaign period from 18 February 2019
(the start of the first full week of campaigning, marked by the launch of various voting aids and
9 Transl. from the Dutch by the authors: “termen als ‘verkiezingen’, ‘Tweede Kamer’ en ‘peiling’, en/of de naam van

een partij, lijsttrekker en/of veelbesproken onderwerpen als ‘zorg’, ‘AOW’, ‘asielzoekers’ en ‘EU’” (Kist and
Zantingh, 2017).
10 [Asscher], [Baudet], [Buma], [Dijkhoff], [Jetten], [Klaver], [Krol], [Kuzu], [Marijnissen], [Segers], [Staaij], [Thieme],
and [Wilders].
11 [Rutte].

46

launch events hosted by a number of parties) to 5 March 2019 (five days after the elections), or
five full weeks after the start of the campaign for the provincial elections
EU parliamentary elections
Using the same general strategy, another set of queries was run to find discussion pertaining to
the EU parliamentary elections on 23 May 2019. As parties ran with national lists in this case, we
queried the lead candidates for each party in addition to the current political leaders of all parties
in the Dutch parliament.12 Querying these again was necessary as national leaders played an
active role in the election campaign, such as when Mark Rutte, the VVD Prime Minister, and
Thierry Baudet, the leader of the FvD, engaged in a televised debate on the eve of the elections.
We further queried general election-related phrases, as well as three themes that occurred across
multiple parties’ manifestos: [klimaat], [migratie] and [privacy]. As the elections coincided with a
government campaign seeking to make voters aware of the dangers of disinformation (Ministerie
van Binnenlandse Zaken en Koninkrijksrelaties 2019), we also queried [“fake news” OR
fakenews OR nepnieuws OR desinformatie OR junknieuws]. Finally, for this election we also
queried the names of all parties for which one could cast a vote.13
We queried these keywords using BuzzSumo, limiting ourselves to articles in Dutch, excluding
Belgian sources. As with the Dutch provincial elections, for the EU campaign we queried a
similar 5-week period between 19 April and 23 May (election day). Finally, we removed irrelevant
results such as those covering various Belgian election campaigns and those resulting from
ambiguous keywords such as [Klaver], the name of a party leader but also the word for clover.
Outlet coding: fake and/or junk news?
An important question here is how one identifies a source as either mainstream or its
counterpart, whether fake news, junk news or another terms (such as problematic information).
While mainstream appears rather straightforward to identify (though that also may shift in time),
its counterpart is a fuzzier concept. BuzzFeed News described their fake news as emanating “from
news websites that only publish hoaxes or from hyperpartisan websites that present themselves
as publishing real news” (Silverman, 2016). Here both types of sites purport to be “news”, but
not in the manner or with the substance that the mainstream publishes, given their hoaxes or
hyperpartisanism, or strongly politically coloured.
Another notion is ‘junk news’, and it may be preferred because it avoids the other, historically
fraught ‘fake news’ definition of the ‘lying media’, but is more ontologically flexible, at least as
scholars have described it. While this term has been used as a synonym for ‘fake news’
(Venturini, 2019: 10), Marchal et al. (2018) employ it to capture a broader category of content
that consists of “various forms of propaganda and ideologically extreme, hyperpartisan or
12 [“De Graaff”], [“De Lange”], [“in 't Veld”], [“van Dalen”], [“van der Spek”], [“van der Staaij”], [“van Lanschot”],

[Asscher], [Azmani], [Baudet], [Berendsen], [Buma], [Dijkhoff], [Eickhout], [Eppink], [Hazekamp], [Hoekstra],
[Jetten], [Klaver], [Krol], [Kuzu], [Manders], [Marijnissen], [Rutte], [Segers], [Thieme], [Timmermans], [Tonça],
[Wierda], [Wilders].
13 [50Plus], [CDA], [Christenunie OR SGP], [D66], [Denk], [FvD OR “Forum voor Democratie”], [GroenLinks],
[“Jezus Leeft”], [PvdA], [PvdD OR “Partij voor de Dieren”], [PVV], [SP], [VVD].

47

conspiratorial news and information” (2). This then would include BuzzFeed News’ notion, but
also part of the NRC Handelsblad’s broader category of tendentious sites that may more often
comment upon rather than deliver news, as we come to.
For their ‘Junk News Aggregator’, a Facebook junk news scraping project, researchers at the
Oxford Internet Institute identified a set of measures to define what qualifies as junk news,
consisting of 1) a lack of journalistic standards; 2) tendentious style; 3) low credibility; 4) clear
bias; 5) a mimicry of traditional news reporting aesthetics; or 6) aggregating content matching the
first five criteria (Liotsou et al., 2019: 3). A source was then considered junk news if it satisfied at
least three of the first five criteria, or the sixth. Herein lies the flexibility, but also the breadth of
the definition that may be suitable for the current analytical purposes in the Dutch case.
In its report, the NRC Handelsblad concluded that propaganda or disinformation did not play a
significant role in Dutch media. It also distinguished between mainstream and hyperpartisan
sources, where the latter is news that is purposively taken out of context, exaggerated to promote
a cause (i.e., tendentious) or strongly politically coloured. A number of Dutch outlets can be
qualified as both ‘tendentious’ and strongly politically coloured, while also being embedded in
the Dutch media landscape (and in that sense mainstream or mainstreaming). Originally a socalled ‘shock blog’, Geenstijl describes itself as tendentious, and gave birth to PowNed, a public
TV broadcaster with a similar signature style. Given its durability and link with the public
broadcasting company, GeenStijl could be considered both tendentious and mainstream, or the
hybrid category, tendentious-mainstream. Another case that is prominent in the BuzzSumo
results we found is The Post Online (TPO). It is a right-wing media outlet and could fit the NRC’s
definition as well as a broader definition of hyperpartisan sites as “openly ideological web
operations” (Hermann 2016). Putting it in the same category as more fringe sites such as
Ninefornews.nl (a site promoting conspiracies and UFOlogy) or De Dagelijkse Standaard (a farright outlet that regularly publishes virulently anti-immigrant articles) would not do justice to the
less extreme tone. Thus, we could dub it tendentious-hyperpartisan. In the analyses to follow
here we show the results with tendentious as a separate category made up of these two sources.
In other studies to follow (on Twitter), the results are compared when the tendentioushyperpartisan source is categorised as tendentious or as hyperpartisan (see Niederer and Groen,
this volume).
In the following we employ the fine-grained categorisation and continuum, distinguishing
between ‘mainstream’, ‘tendentious’, ‘hyperpartisan’, ‘conspiracy’ and ‘clickbait’, occasionally
linking the categories, as mentioned. These categories reflect the various sub-types of
mainstream, tendentious and otherwise lower-quality content discussed in the introductory
chapter. This also allows more nuanced categorisations of sites such as GeenStijl and The Post
Online. In the following analysis we offer this five-category coding as an addition to the binary
OII-based categorisation, as a way to illustrate the make-up of non-mainstream content found in
the data. This categorisation resulted from a collaborative coding effort across all case studies
found in this volume and provides a more detailed alternative to the binary ‘fake/junk’ versus
‘mainstream’ opposition found in, for example, the BuzzFeed News and NRC Handelsblad studies.

48

In all we therefore elect not to reduce the sources to fake but rather use a more inclusive
category of ‘junk news’, but then also pay special attention to the tendentious outlets. After
identifying the sites using this typology, we further removed all other sites from the results that
were either marginal or local. Marginal here refers to sites that received very low engagement
scores in the BuzzSumo results and were not otherwise notable in terms of content or overall
engagement. We also excluded local news sites, as our main concern for this analysis is outlets
with a national or otherwise substantial reach; regional outlets conversely typically have a limited
audience, and our list of ‘junk’ sites contained more nationally oriented outlets rather than
regional ones. This left a ‘mainstream’ category containing national outlets, mostly deeply
embedded in the traditional Dutch media landscape, such as various national newspapers, TV
broadcasters and a number of online news sites and magazines.
Data analysis: overall and per-query trends
Dutch provincial elections
We used the annotated source list (or expert list) to code the results for the BuzzSumo queries,
as discussed in more detail below. This allows for a per-query observation of the ratio between
mainstream and junk sources. Next to these separate analyses we also calculated an average ratio,
weighted by the relative engagement per category, on both an overall and a per-week basis. While
our categorisation method is slightly different from BuzzFeed News’, this per-week analysis
nevertheless allows for a trend comparison with the results of their over-time analysis of the U.S.
2016 presidential election campaign.

Figure 1. Engagement of mainstream (blue) and junk-like news (pink) articles found through provincial electionsrelated BuzzSumo queries, per week, between 18 February 2019 and 25 March 2019. Engagement scores have
been normalised.

49

Figure 2. Total Facebook Engagement of fake versus mainstream news. Results from election-related queries on
BuzzSumo, for the 20 most-engaged with articles during February and November 2018, per three-month period.
Source: Silverman 2016.
Notably, the trendline found in our over-time analysis (Figure 1) does not match the one in
BuzzFeed’s study (see Figure 2). While BuzzFeed News’ data saw a clear increase of engagement
of fake news in the weeks leading up to the elections, in our data junk news stayed relatively
constant in terms of engagement and even decreased slightly during the last few weeks. There
are, however, some differences between the two campaigns that complicate a direct comparison.
The U.S. election campaign is typically far longer than Dutch election campaigns, especially in
this case as the 2019 election was concerned with the provincial states and senate rather than the
lower house of parliament (typically the most important Dutch election). While the U.S.
campaign was analysed over a period of 9 months, the Dutch campaign and hence our data
spans five weeks only. Additionally, BuzzFeed News’ data resolution is quite low (one datapoint
per three months) while ours is more fine-grained (one per week).
Nevertheless, even considering these differences it is striking that the graphs indicate rather
different dynamics. While the BuzzFeed data points to a clear ramping up of fake news content
as the election date draws near, our data is more in line with the NRC’s earlier study and suggests
a more constant but persistent undercurrent of junk news that is a consistent part of politically
oriented media output. The above data is an aggregate of all queries performed on BuzzSumo,
however. While in aggregate there is no clear trend, this could be the result of summing up the
values, and more apparent trends exist in the results for individual queries.
50

Figure 3. Per-query engagement of mainstream (blue) and junk (pink) articles found through provincial electionsrelated BuzzSumo queries, per week, between 18 February and 25 March 2019. Engagement scores have been
normalised.
As can be seen in the overview in Figure 3, even on a per-query basis there are few clear trends
with regards to the prevalence of junk news engagement. There is an interesting uptick in the
prevalence of mainstream engagement for a few queries. Most notably, the data for [Segers], the
leader of ChristenUnie (a centrist Christian party), shows a sharp increase in the last week of
election campaign. This can however almost entirely be attributed to news coverage after the

51

elections about the implications of the election results for the cabinet, of which Segers’ party is
the smallest member. (Note the similar uptick for [Jetten], whose D66 party is the secondsmallest cabinet member.) Another notable bump in mainstream engagement occurs for a
number of queries ([PS2019], [Buma], [Kuzu] and [Dijkhoff]) around the middle of the election
campaign. A closer look at the articles responsible for this engagement reveals that this may be
an indication of the campaign coming into full swing and hence increasing media coverage of it.
The oft-quoted and feared BuzzFeed News pattern of fake news outperforming mainstream news
is thus not repeated on either an aggregate or query level in this case study.
What remains of interest is the relative performance of mainstream and junk news on a per-week
and per-query level, particularly on a number of occasions where junk news briefly outperforms
mainstream news in terms of Facebook engagement. For queries of politicians, it occurs most
notably for [Baudet], [Kuzu] and [Wilders] during the first week of the election campaign, where
the dominance of junk news is most pronounced. These politicians all lead relatively fringe
parties: Baudet leads the far-right Forum voor Democratie (FvD), Kuzu the left-wing and
immigrant-oriented DENK, and Wilders is the leader of the far-right Partij voor de Vrijheid
(PVV). A closer look at the junk news articles that are responsible for these surges shows that in
all three cases, these are not articles primarily concerned with the elections themselves but rather
coverage of other political issues (mostly around climate laws that were being discussed at the
time) by hyperpartisan outlets like De Dagelijkse Standaard. The relative prevalence of this
coverage is perhaps an indication that media had not yet started covering the election campaign
in earnest, rather than a dominance of junk news in election discourse. Overall, while in some
individual cases junk news outperforms mainstream news, these episodes are outliers and
represent less of an overall trend than one for particular parties. There is one general exception
to this rule, however, and it concerns the query for [migratie], or migration, where junk
outperforms mainstream for most of the period. Also, [klimaat] or climate, has a week where
junk news had more engagement that the mainstream. These are rather polarising issues, drawing
attention from hyperpartisan outlets.
EU Parliamentary elections
A trend analysis of the EU parliamentary results (see Figure 4) shows a pattern not too dissimilar
to the one found in the provincial elections data, similarly seeing junk news match the
performance of mainstream news particularly in the beginning of the query period. Recall that
during the provincial elections campaign junk news performed as well as mainstream news on
two occasions. Though this trend is still notably different from the one found by BuzzFeed News,
where junk news overtook mainstream news towards the end of the campaign, it is nevertheless
a significant finding that suggests an increasingly robust position for junk news in the Dutch
context.

52

Figure 4. Engagement of mainstream and junk-like articles found through EU elections-related queries on
BuzzSumo, between 18 February 2019 and 25 March 2019. Engagement scores have been normalised.
A closer look at this second week of the EU campaign data shows that the junk news
engagement can for a large part be attributed to an article in De Dagelijkse Standaard, which
discusses a video posted by the political party Denk on their Facebook page, accusing the party
of demonising Geert Wilders (of the PVV party).14 This article’s engagement is responsible for
about 36% of that week’s ‘junk’ engagement, providing a major boost.
More generally the relatively high engagement attained by junk sources can in many cases be
attributed to a small number of high-performing articles. This matches the findings from the
analysis of the provincial elections, where peaks in junk news engagement could similarly be
attributed to a smaller number of well-scoring articles. While junk sources perform relatively
well, especially in the earlier weeks of the data set, this success is thus attributable to a relatively
small number of sources and articles rather than a broadly successful and diversified ecosystem
or even a coordinated campaign.
Though the findings do not approximate those in the BuzzFeed News story, in the case of the EU
election campaign it is noteworthy that indeed junk news does on one occasion match the
performance of mainstream news, though not during the tail end of the campaign period as was
the case in the BuzzFeed News data. Overall, junk news is roughly as successful during the EU
campaign as it was during the provincial election campaign, and has a significant presence,
though over the whole campaign mainstream news still easily outperforms it.

14 “Video! Kuzu (DENK) wil dat Wilders gestopt wordt, voordat hij een tweede Srebrenica-bloedbad kan

aanrichten”: https://www.dagelijksestandaard.nl/2019/05/video-kuzu-denk-wil-dat-wilders-gestopt-wordt-voordathij-een-tweede-srebrenica-bloedbad-kan-aanrichten/

53

Figure 5. Per-query engagement of mainstream (blue) and junk (pink) articles found through EU parliamentary
elections-related BuzzSumo queries, per week, between 18 February and 25 March 2019. Engagement scores
have been normalised.
Characterising sources
It is useful here to briefly discuss the sites that make up both categories of content. Our category
of mainstream outlets (see Table 1) consists of well-known outlets with a national reach, which
in practice translates to a number of national newspapers, public broadcasting organisations,
national TV programmes and large online magazines. The junk category is comparatively more
54

diverse; the typology we use covers conspiracy sites, hyperpartisan online sources (including
independent self-styled journalists), and clickbait aggregators. Some of these are relatively large:
De Dagelijkse Standaard, a far-right weblog, appears in the top three of most engaged-with articles
for 15 of our 19 queries. Some other junk sites appear to be more focused on a particular topic;
this is especially apparent in the results for the provincial elections [Migratie] (migration) query, in
which fenixx.org – a far-right site advancing the racist ‘race replacement’ theory – appears often,
while it is far less prominent for the other queries, save for the [EU], in which it also appears
occasionally. This site was also noted by the earlier 2017 NRC study as being especially prevalent
in their ‘hyperpartisan’ category.
This ‘hyperpartisan’ category can then be seen to be comprised of roughly the same set of sites
in both data sets (see Table 1 and 2). This could be considered to suggest a hyperpartisan news
ecosystem of sites that enjoy a significant and stable readership. On the other hand, this
ecosystem is notably top-heavy; for both data sets De Dagelijkse Standaard (DDS) is by far the
most engaged-with site, almost four times as popular as the next site in the list. Following DDS
is a number of far smaller but simultaneously more outspokenly far right blogs such as Stop de
Bankiers, Fenixx and JD Report. Fenixx here is further notable as a site that was also mentioned as
a relatively prominent junk site in the 2017 NRC study. While we can thus identify a stable
sphere of hyperpartisan news sites that drive significant engagement, the success of this sphere is
still mostly reliant on De Dagelijkse Standaard, and with the exception of that site is quite marginal
compared to the mainstream sphere.

Mainstream

Junk news

Site

Engagement

Site

Enagagement

telegraaf.nl

102117

dagelijksestandaard.nl

98414

nu.nl

46962

stopdebankiers.com

26429

rtlnieuws.nl

46849

fenixx.org

13024

wnl.tv

39975

jdreport.com

8564

nos.nl

37319

ninefornews.nl

5975

nrc.nl

16010

tpook.nl

4431

metronieuws.nl

14746

ejbron.wordpress.com 4126

pauw.bnnvara.nl

10130

opiniez.com

2777

evajinek.kro-ncrv.nl

7412

dlmplus.nl

2110

Table 1. Top 10 sites per category (Provincial elections), for all queries combined, sorted by overall engagement
scores as reported by BuzzSumo.

55

Mainstream

Junk news

Site

Engagement

Site

Enagagement

telegraaf.nl

232327

dagelijksestandaard.nl

225006

nu.nl

192962

stopdebankiers.com

46892

nos.nl

141440

fenixx.org

25852

rtlnieuws.nl

99820

tpook.nl

17453

wnl.tv

91211

jdreport.com

9199

elsevierweekblad.nl

31150

opiniez.com

8302

metronieuws.nl

28038

ejbron.wordpress.com 6427

nrc.nl

27195

reactnieuws.net

5565

joop.bnnvara.nl

22509

ninefornews.nl

2047

Table 2. Top 10 sites per category (EU parliamentary elections), for all queries combined, sorted by overall
engagement scores as reported by BuzzSumo.
As discussed above, an alternative to the binary mainstream/junk opposition one may consider
the data for both election campaigns in terms of a more detailed five-category perspective (see
Figures 5 and 6). For both the provincial and EU elections it is apparent that the largest nonmainstream category by far consists of hyperpartisan sources. The only other category that has a
noteworthy impact are tendentious sources GeenStijl and The Post Online (which are both not
included in the other, binary, categorisation in figures 1 and 4). Conspiracy and clickbait sources
are present in the data but do not play a significant role compared to the other categories.

Figure 6. Engagement of mainstream, hyperpartisan, conspiracy and clickbait articles found for provincial
elections-related queries on BuzzSumo, between 18 February 2019 and 25 March 2019. Engagement scores
have been normalised. Geenstijl is considered ‘mainstream’ here, while The Post Online is classified as
‘hyperpartisan’.
56

Figure 7. Engagement of mainstream, tendentious, hyperpartisan, conspiracy and clickbait articles found for
provincial elections-related queries on BuzzSumo, between 18 February 2019 and 25 March 2019. Engagement
scores have been normalised. Geenstijl and The Post Online are considered ‘tendentious’ here.

Figure 8. Engagement of mainstream, tendentious, hyperpartisan, conspiracy and clickbait articles found for EU
parliamentary elections-related queries on BuzzSumo, between 19 April 2019 and 23 May 2019. Engagement
scores have been normalised. Geenstijl is considered ‘mainstream’ here while The Post Online is classified as
‘hyperpartisan’.

57

Figure 9. Engagement of mainstream, tendentious, hyperpartisan, conspiracy and clickbait articles found for EU
parliamentary elections-related queries on BuzzSumo, between 19 April 2019 and 23 May 2019. Engagement
scores have been normalised. Geenstijl and The Post Online are considered ‘tendentious’ here.
Dutch provincial elections
EU Parliamentary elections
Site
Engagement Site
Engagement
dagelijksestandaard.nl
stopdebankiers.com
fenixx.org
jdreport.com
ejbron.wordpress.com
dailypaper.org
opiniez.com
destaatvanhet-klimaat.nl
pallieterke.net
eunmask.wordpress.com

168668
35414
20757
15679
5285
4887
4554
3912
3228
2487

dagelijksestandaard.nl
stopdebankiers.com
fenixx.org
jdreport.com
opiniez.com
ejbron.wordpress.com
reactnieuws.net
xandernieuws.net
eunmask.wordpress.com
novini.nl

225006
46892
25852
9199
8302
6427
5565
2009
1296
862

Table 3. Top 10 ‘hyperpartisan’ sites for both data sets, sorted by overall engagement scores as reported by
BuzzSumo.
An examination of the most engaged-with sites in the hyperpartisan category (see Table 3)
further confirms that this category is the most influential one in the broader ‘junk’ (or ‘junk-like’)
sphere, with the top ten sites mostly matching those found in the top 10 of ‘junk’ sites identified
in tables 1 and 2. The top five is identical between all lists, and again De Dagelijkse Standaard is the
most important site. Notably, as the campaign draws on, mainstream engagement can be seen to
increase while junk news performance is relatively stable, meaning interest in mainstream news
coverage increases towards the end of a political campaign, while junk news remains stable.
Perhaps they serve different publics, though such a construal would require further work.

58

Generally, the junk news sites, of which hyperpartisan sites are the largest constituent, can be
characterised as right-wing, anti-immigrant, anti-EU and in some cases anti-Semitic or advancing
conspiracy theories (the latter especially applying to ninefornews.nl and jdreport.com). This
right-wing slant in our findings is consistent with other studies on junk news, including the 2016
BuzzFeed News analysis but also others that found that left-wing content was less prominent in
that category (Silverman, 2016; Neudert et al., 2017:1; Alcott and Gentzkow, 2017:223). In this
case study, next to the prevalence of hyperpartisan sites such as DDS the relatively large
engagement of especially conspiracy sites is notable; ninefornews.nl, which is the 5th-most
engaged with site in our data, regularly promotes conspiracy theories ranging from UFO
sightings to far-right theories such as Pizzagate and QAnon. The authors seem to be convinced
that this is accurate accounting of events. Overall, the data show that junk news, consisting
primarily of hyperpartisan and conspiracy theory sites, are a minor but constant and significant
factor.
A cross-platform appraisal
This case study focuses on Facebook, but a similar analysis may be performed for other
platforms. While Facebook has the dubious honour of being the platform with perhaps the
strongest association with fake news, other platforms have their own affordances that could
make them attractive for those seeking to spread forms of junk content. Just as this case study
builds on the analyses of BuzzFeed News’ and the NRC’s, with a number of methodological
tweaks, one could similarly move to other platforms as well, studying over-time engagement of
junk and mainstream content respectively. Multiple case studies in this volume employ a method
of this type.
The multiple platform analyses present an opportunity not only to investigate the prevalence of
junk news on individual platforms, but also to perform a cross-platform analysis in order to
investigate whether there are platforms that are particularly susceptible to junk content, or
whether some platforms have perhaps succeeded in combating the spread of it, given that the
phenomenon has been addressed for some time now, and the case studies in question take place
in early to mid 2019. While we present such a comparison in this section, it should be noted that
a direct comparison between platforms is complicated for a number of reasons.
One issue with a comparison between various platforms is that ‘engagement’ means different
things depending on the features a platform offers for interacting with content. On Facebook,
engagement means the sum of comments, likes (or reactions) and shares a post received. But
Reddit, for example, has no direct counterpart to some of those, as ‘shares’ are not a relevant
concept on that platform. It simultaneously offers metrics Facebook does not use (including
upvotes and downvotes). Moreover, on Facebook a dislike or angry reaction, for example, often
would be counted as a plus engagement, whereas a downvote on Reddit reduces a post’s score.
More specifically, the case studies in this collection use different time periods and in some cases
investigate, apart from election issues and leaders, certain polarised topics (such as MH17 and
Zwarte Piet) so as to seek disinformation or junk, as we discussed above in terms of
asymmetrical querying. Such query design may be justified, given that previous studies of
disinformation in the Dutch media context were borne of data curated by Twitter that consisted
59

of Russian IRA trolls, and found activity around the downing of the MH17 airliner in 2014 as
well as the terrorist attacks in the Brussels airport and metro in 2016. When examining on
Twitter the MH17 hashtag and keyword usage over the past number of years, one may find
increased activity around elections (such as during the national elections of 2017), thus further
justifying a renewed attention towards at least MH17 during the 2019 elections. Such
asymmetrical querying of course complicates comparisons, as the ratio between mainstream and
junk news engagement may be less balanced, given conspiracy and other sources’ continual
attention to such themes. Differences in time periods also pose issues, as there may be
particularly ‘junk-sensitive’ episodes from the past that are missing from the current analyses, and
for analytical purposes have been removed from the comparison.
Nevertheless, provided one is aware of the limitations in such a comparison, the results of such
an analysis for other platforms compared to the Facebook case study can provide an impression
of the relative penetration of junk across different social media platforms. While in the rest of
this volume there are separate case studies that investigate the individual platforms with methods
similar to this one, the graph below presents a rough impression of the results across platforms,
using data from this chapter and the other case studies.

Figure 10: Relative engagement of content categories across 4chan /pol/, Reddit, Twitter and Facebook. Geenstijl
is considered ‘mainstream’ here while The Post Online is classified as ‘hyperpartisan’. 4chan and reddit data from
1 Dec 2015 until 1 June; Twitter and Facebook data from 18 Feb 2019 – 25 Mar 2019 and 19 Apr 2019 23 May 2019.

60

Figure 11: Relative engagement of content categories across 4chan /pol/, Reddit, Twitter and Facebook. 4chan
and reddit data from 1 Dec 2015 until 1 June; Twitter and Facebook data from 18 Feb 2019 – 25 Mar 2019
and 19 Apr 2019 - 23 May 2019.
What is striking in the cross-platform comparison of results in Figure 7 is that the two
‘mainstream’ social platforms, Facebook and Twitter, show a higher prevalence of junk content
than 4chan and Reddit, the deep vernacular web platforms. This is interesting because the latter
two – the “seedy underbelly” of the internet (Bergstrom 2011) – are often characterised as
hotbeds of polarizing and alt-right political discussion, thus providing an environment where one
could expect particularly hyperpartisan content to thrive.
One plausible explanation of this is that especially on 4chan’s /pol/, the ‘politically incorrect’
sub-forum that was investigated in this case, those posting may position themselves in
opposition to mainstream sources. This positioning often goes hand in hand with linking to the
sources in question, thus increasing the share of mainstream content in the overall picture for the
platform. As such it underlines the notion that engagement does not necessarily indicate that one
agrees with the engaged-with content, and in fact high engagement may be taken to indicate
controversiality, as something polarizing that is hotly debated can be expected to be clicked on
and scrutinised by many of those posting about it.
Conclusions: Absence of disinformation and junk news prevalence
As discussed, a detailed cross-platform comparison is complicated by the different methods used
in each case study. While outside the scope of this research, further commensuration of these
methods and results for a more thorough cross-platform analysis presents an opportunity for
further research.
This particular case study, focused on Facebook, is informed by similar investigative (data)
journalism originating with BuzzFeed News and the NRC Handelsblad. Our results are not strictly
in keeping with theirs. It is of particular interest that the prevalence of intentionally false news
BuzzFeed found was not apparent in our data, indicating that this is far less of a problem in the
Dutch sphere than in the U.S. The data do seem to confirm the reputation of Facebook as an
especially fertile ground for junk news in comparison to other platforms and indicates that
despite its initiatives to combat such content, it is still endemic on the platform. In fact, whereas
61

the NRC found that “at most 10%”15 (Kist and Zantingh, 2017) of the engagement they analysed
concerned hyperpartisan and tendentious content, in our analysis a little over a year later we find
this share has risen to 25%.
While this difference between our findings and the NRC’s could partially be explained by the
differences in the criteria used to categorise the content, it seems justified to conclude that even
if junk news is in the minority, it is certainly not marginal, and seems to be a growing product in
the Dutch media landscape, on some occasions matching the performance of mainstream news
in terms of Facebook engagement. Though this case study is limited to the 2019 provincial and
EU elections, its findings suggest that a broader analysis of junk coverage of Dutch politics on
Facebook is warranted. Such an analysis could also investigate what ‘engagement’ means in
practice; as indicated by the cursory cross-platform analysis, engagement may not translate to
agreement, and if junk news is such a factor on Facebook it is important to understand the
motivations behind engaging with it if we are to understand the significance of it in the wider
political debate.
A silver lining (so to speak) is that there was virtually no outright (foreign) disinformation in the
data we found, and indeed across all platforms we investigated. While especially on Facebook
there is a solid undercurrent of junk sites including hyperpartisan content, and a number of wellshared conspiracy sites which give promote highly dubious content, there is no imminent reason
to expect so-called fake news affecting Dutch election coverage in the same way it appeared to
for the 2016 U.S. elections. Overall, our Facebook case study indicates that there is no immediate
cause for concern about disinformation about Dutch elections, but that junk news is a growing
factor that warrants closer scrutiny.

15 Transl. from Dutch by authors of “hoogstens 10 procent” (Kist & Zantingh 2017)

62

JUNK NEWS IN SEARCH ENGINES:
EXPLORING GOOGLE’S SENSIBILITY
TOWARDS HYPERPARTISAN SOURCES
DURING THE DUTCH ELECTIONS

Guillén Torres and Richard Rogers

Junk news in search engines: Exploring Google’s sensibility towards hyperpartisan
sources during the Dutch elections
Guillén Torres and Richard Rogers16
Introduction: Search engines as junk source space
As key entry points to the web, search engines serve as a site for the consumption of
information, including political information, and as such are a relevant space for the study of
both the presence of disinformation and junk news as well as approaches to combat it (Bowden,
2016). Although they are described in the industry as ‘organic’, the output of search engines
could be termed manufactured hierarchy (Hindman, 2008 Halavais, 2017). Sources are ranked
per query, and as such certain ones are offered as more relevant than others, as if naturally. Such
ranking practices are often considered inscrutable, since search engines generally do not provide
a means to save and study query results, e.g., through offering an API that enables it or terms of
service that allow it. In fact, the ranking algorithms are trade secrets jealously guarded by
corporations.
Since 2009, Google and subsequently other search engines have personalized results, be it for the
individual or the place where the search has taken place (Pariser, 2011; Puschmann, 2018).
Increasingly engines are thus both providing ranked political information but also tailoring it to
user preferences and/or location (Martens et al., 2018).
When it comes to events, such as elections, search engines become providers not only of topical
but also of timely information. These are particularly poignant moments to study the presence of
disinformation and junk news. As cases in point, there have been occasions when sources that
were otherwise insignificantly ranked rose to the top of engine returns during the ‘breaking news’
period of an event, such as in the immediate aftermath of the Las Vegas shootings in 2017, when
a 4chan post misidentifying the shooter rose to the top of the results (Robertson, 2017). Google
also prominently linked to rumours about the identity of a Texas shooter in 2018 as “a Muslim
convert, member of Antifa or Democrat supporter” (Lomas, 2018). One could point to artificial
manipulation, such as search engine optimization, as triggering the unexpected rankings and
sudden presence of dubious information. With respect to the 4chan post, the gaming of the
engine (if that were the cause) also may have been maliciously playful, introducing
misinformation as an act of trolling. In the case of the rumours about the Texas shooter, the
manipulation appears to have been hyperpartisan. Both spates of false news were not ‘corrected’
in the editorial sense of an erratum notice; rather, the dynamically published results are
continually algorithmically tweaked so “good information” is said to ultimately prevail (Waters,
2017).
In the study of engine returns and hierarchies (through manual capture techniques) it is often
pointed out that top placement matters, since engine users over the years have been browsing
fewer and fewer result pages (Jansen & Spink, 2003; Dan & Davison, 2016). Thus, in the above
16 The research reported here was undertaken in collaboration with Anja Duricic, Lisa Fluttert, James Ingleby and

Ziwen Tang.

63

examples, the significance of the location of misinformation, rumour and extreme results relies
on findings about how users gravitate to the top results, making them the most consumed and
thereby particularly worthy of study. An additional research strategy for inquiries into fake and
junk news presence concerns anticipatory search, also known as autosuggestion, which drives the
user to particular search terms. It also has been studied for the offensive associations made by
engines, such as the completion of “are Jews” with “evil” (Cadwalladr, 2016). Misogynistic
autosuggestions also were documented in earlier cases which led to a UN campaign in 2013
discussed in a longer study of “how search engines reinforce racism” (Noble, 2018). Other
extremist content has been similarly documented towards the top of Google results for the query
“holocaust” (Hern, 2017).
The discovery by The Guardian journalist of the offensive associations with the word “Jews” and
the resulting sites that surfaced (such as the neo-Nazi website, The Daily Stormer) have led to
discussions of not just how Google’s algorithms could be tweaked, but also the reach of the
sources producing and driving such information in the first place. Their presence (and top
placement) could be interpreted as a proxy for the significance and audience of such material
online, or as others have argued as evidence of a culture war, driven by the online boosting
tactics of ‘culture hackers’ (Albright, 2016; Confessore and Wakabayashi, 2017). Whilst they may
seek to correct the autosuggestions (and perhaps remove religious ones all together), companies
such as Google are hesitant to delist such extreme websites, given free speech concerns, which
also may arise if they are nudged downwards.
The present chapter studies the susceptibility of Google’s search engine to provide users with
questionable information sources in the results for queries related to Dutch political parties
during the Dutch provincial and European parliamentary elections of 2019. Our goal is to
identify the presence of junk sources in the results for political queries. Thus, the research
questions read as follows. Which rankings has the search engine assigned to fake and junk
sources when querying political parties and their issues? Are there particular political issues and
party spaces where fake and junk news are prevalent or entirely absent? Do the ranks and
amounts increase as the elections draw near?
We divided the chapter into six sections: a brief reflection about the methodological challenges
of studying search engines, the methodology for building our dataset, three sets of findings, and
a discussion of the limitations and further steps. In all it was found that fake and junk news,
specifically of the hyperpartisan variety, is rather pervasive in the search-demarcated political
space, but far more so for certain actors and their issues on the far right of the political
spectrum.
Studying personalization, junk news, or both?
The extent to which autosuggestions are personalized is understudied, but the personalization of
results more generally has been the subject of numerous inquiries and methodological
innovations that work around Google’s inscrutability through selectively scraping results or
soliciting data donations. The findings from scraping batch-queried results have shown relatively
low amounts of results affected by personalization in the Google search engine (Feuz et al.,
2011), and the same held for Google News (Haim et al., 2018), suggesting that original concerns
64

regarding the ‘filter bubble’ may not be as well founded any longer. Where the second method is
concerned, Algorithm Watch, the German NGO, created a browser extension for users to install
that regularly would make political queries, such as for “Angela Merkel” (Puschmann, 2017). The
results would be donated by the users to Algorithm Watch so as to enable a larger number of
those under study than is normally the case. Here again the findings have shown low levels of
personalization, but the study of the presence of certain junk sites (be they disinformation or
another genre) could be pursued further. Another technique, discussed below, is to selectively
scrape results in a manner that seeks to minimize personalization effects, thereby concentrating
on the overall presence of junk sites rather than on whether particular users, in a filter bubble,
are seeing more of them.
To begin to understand the amount and placement of junk news in search engine space, be it
around events or even after an algorithmic tweak, a query routine is designed, and a window of
activity is chosen. (Longer-term studies also may be undertaken, as in the Issuedramaturg project
that followed 9/11 query results for years, but Google often changes its output formats, breaking
automated tools (Rogers, 2013).) In order to reduce personalization, a research browser may be
deployed, which is a clean instance of a browser with the user not logged in. City-level
geographical personalization may be avoided through the use of advanced settings, choosing a
particular region, such as the Netherlands. For projects as ours, geographical personalization is
not viewed as a disadvantage in the sense of creating the conditions for a filter bubble to
materialise.
A brief mention should be made of the search engine under study. Among them Google is the
most popular, with the largest market share of users in most countries, certainly in the
Netherlands. As mentioned above, recently, the company has become entangled in the fake news
debate through the appearance not of Russian disinformation sources (though that to our
knowledge has not been studied in great detail), but owing first to the appearance of misogynistic
and extremist content that the company previously defended as ‘reflective’ of societal concern
rather than the product of algorithmic error or ‘culture hacking’. If one were to expand the
number of search engines under study (to include Bing and Yahoo!, for example), one could
triangulate results, and inquire further into the normalcy and regularity of misogynistic and
extremist content present in the top results, though one could not control for algorithmic
concentration or the extent to which the big engines’ algorithms are anyways similar. The extent
to which the results reflect societal concern would remain an open question.
Studying Google results
The presence of Junk news within Google search engine’s result is a multi-causal phenomenon
that may be credited to a number of factors. Among others, Google’s algorithm reacts and learns
from users’ own consumption of junk sources. It is trained using varied datasets, and content
producers’ attempt to game the search engine via Search Engine Optimization tactics (Finkel et.
Al, 2017). Given the inscrutability of Google’s tool, it is difficult to determine what could be
causing the presence of junk sources in the Dutch websphere or others. Here, rather than
attempting an explanation for the presence of junk news, we conduct a test of the tool’s
sensibility to connect politically relevant queries with junk sources.

65

As noted, the investigation relies on scraping as a method (Marres & Weltevrede, 2013), and
takes as its point of departure the question of junk news in search returns rather than the effect
of personalization in the creation of a filter bubble. The research seeks so-called junk news in
search engine results, which has been defined as “extremist, sensationalist, conspiratorial and
masked commentary” (Howard et al., 2017: 1). In keeping with Buzzfeed News’ definition of fake
news (Silverman, 2016), we also seek (foreign) disinformation, hyperpartisan sources as well as
clickbait, which itself may be extreme. In order to do so we rely on a list of sources expertly
curated by other researchers in the project.
Generally, the research employs the ‘source distance’ approach, inquiring into how far from the
top of the returns are the offending results (Rogers, 2013). More specifically, we investigate how
fake and junk webpages are positioned in the first twenty Google.nl results of various queries of
political parties and their most significant issues during the 2019 provincial elections campaign as
well as that of the European parliamentary elections. Thus, this case examines the susceptibility
of search engine results to fake and junk news, as defined above, rather than exploring the issue
of fakeness and junk in themselves or the effectiveness of counter-measures.
Engine returns as political spaces
In order to demarcate a political space in search engine results, we designed a query protocol
based on combining the names of political parties with specific issues associated with their
respective political agendas. In that sense, the underlying assumption is that junk news may have
a more significant impact when discussed in a specific political context, such as election
campaigns, when voters gather information to guide their choice. To be able to collect the results
of a large number of queries, we used the Search Engine Scraper by the Digital Methods
Initiative, a tool that allows one to scrape search engine results for a given query and commit
them to a database for further scrutiny through visualization.
The first step of the methodology consisted in constructing a list of Dutch parliamentary parties
and locating their websites as well as Facebook pages (see table 1). Another list was created
pertaining to the parties participating in the European parliamentary elections. In the next step
we identified the relevant keywords to build the queries; in the case of the Dutch provincial
elections, these were sourced from both the party webpages and the party Facebook pages. For
the parties competing in the European parliamentary elections, only the parties’ own websites
were consulted. The party webpages provided the parties’ issue keywords. The Facebook pages
furnished a more vernacular set of issue terms, as they contain issue keywords from users or
citizens in the comment space. The aim of sourcing these two sets of keywords was to enable us
to capture and compare the results for both official as well as more popular issue language.
The lists of party keywords were built by collecting the platform standpoints (standpunten) on the
party websites. There is one list for the provincial elections and another for the European
parliamentary elections. Most political parties mention between five and ten keywords on their
platforms, and all were collected. A few parties (e.g., the Staatkundig Gereformeerde Partij, SGP)
offered longer lists which were shortened on the basis of their key issues. In all, the political
party issue space consisted of 158 keywords across the thirteen parties for the provincial
elections, and four keywords across fourteen parties for the European parliamentary elections.
66

The vernacular list was made through a close reading of the comments made under the posts of
the Facebook pages of each political party. To build this list, the most commented posts around
the days of the elections were close-read, and the most representative keywords related to the
views expressed by the commenters were chosen. Identifying the most relevant issues in the
comment space on Facebook proved to be problematic, given that the comments were often
polarizing and emotive, especially those relating to the elections themselves. This citizenenriched political issue space consisted of five keywords for each of the thirteen parties, making
a total of 65.

Dutch Provincial Elections
Name of the Party
Short Name
Volkspartij voor Vrijheid en
VVD
Democratie
Partij voor de Vrijheid
PVV
Christen-Democratisch
CDA
Appèl
Democraten 66
D66
GroenLinks
GL
Socialistische Partij

SP

Partij van de Arbeid
ChristenUnie
Partij voor de Dieren
50Plus
Staatkundig Gereformeerde
Partij
Denk
Forum voor Democratie

PvdA
CU
PvdD
50plus

European Parliamentary Elections
Name of the Party
Short Name
Volkspartij voor Vrijheid en
VVD
Democratie
Partij voor de Vrijheid
PVV
Christen-Democratisch Appèl
CDA-EV
- Europese Volkspartij
Democraten 66
D66
GroenLinks
GL
ChristenUnie - Staatkundig
CU - SGP
Gereformeerde Partij
Partij voor de Dieren
PvdD
50Plus
50plus
Jezus Leeft
Denk
DENK

SGP

Forum voor Democratie

FvD

DENK
FvD

Van de Regio & Piratenpartij
Volt Nederland
De Groenen

VR - PP
VN
GN

Table 1. List of Dutch political parties under study
The three lists of keywords were inputted in the Search Engine Scraper along with the name of
each party. The results from the parties’ own websites were excluded. For example, for the
political party D66 and the keyword onderwijs (education), the following query was made:
[onderwijs d66 -d66.nl]. Using the advanced search features of Google, maximum results were
set to 1,000, and each day of our periods of interest (13-22 March and 22-24 June 2019) was
queried separately. The date range included the run up to the provincial elections on 20 March
and a short election aftermath period, and the days before and after the European Parliamentary
elections on May 23. The searches were conducted in a clean browser, in the Dutch Google.nl
domain, in the Dutch language, and in the Netherlands region (through the advanced setting).
The keyword and party were queried together so that the scraper tool delivered results that are

67

related to election politics, rather than a general overview by querying each keyword in isolation.
The keywords derived from Facebook were queried in the same format, using the same settings
and date range.
The outputs of the Scraper tool are the top twenty ranked Google.nl results per query. The
URLs in our lists were then truncated to their host names so that they could be cross-checked
with the list of known fake and junk websites curated by other researchers in the project. Here a
formula was used that effectively linked the search engine results spreadsheet to that of the
expertly curated list of fake and junk sites. The question for each source concerned its ranking
per query and its presence or absence in the expert list. All query results (per party and per
official or vernacular language type) were marked as fake and junk sites or not and listed in the
order they were returned.
Party platforms
Foreign affairs Europese Unie, Europa, EU, Nederland en Europa, buitenland, internationale zaken.
Polarizing
Islamisering, Islam
topics
Health
zorg, menselijke zorg
Environment klimaat, natuur, milieu, dierenrechten, natuur en milieu
belasting, economie, inkomen, pensioen, werk en inkomen, schone economie, eerlijk delen,
Finances
werklozen, economisch beleid, overheid en bestuur
Safety and
veiligheid, privacy, defensie, criminaliteit, rechtsstaat, terrorisme
security
waarden, gezin, respect, familie, samenleving, burgers, democratie, ouderen, onderwijs, goed
Society
onderwijs voor iedereen, vrijheid, verantwoordelijkheid, drugs
Future
innovatie, duurzaamheid, schone energie, energie
Migration
immigranten, migratie
Facebook
Foreign affairs Europa, EU, referendum
Polarizing
Islam, Moslim, racisme, discriminatie17
topics
Environment milieu, klimaat, kernenergie, energie
Finances
belasting, bezuinigingen, pensioen, onderwijs
Society
samenleving, democratie, toekomst crisis, vrouwen, vrijheid, Nederland, armoede
Faith
Islam, Moslim, Christendom, Christenen, geloof
Migration
migratie, immigranten, gelukszoekers, migranten
Table 2. List of categories and political keywords used in the study.

17 Islam is placed in both faith as well as polarizing topics categories, given how it is discussed as shorthand for a

social issue.

68

We zoomed in on those queries in which fake and junk news showed up consistently, that is, for
a minimum of four days within our period of interest for the provincial elections, and two days
for the European parliamentary elections. The keywords that produced fake and junk news
websites in their search results in the first case were then grouped thematically in the following
categories: ‘foreign affairs’, ‘polarizing topics’, ‘health’, ‘environment’, ‘economy’, ‘safety and
security’, ‘society’, ‘future’, ‘migration’ and ‘faith’, the most salient of which are described in
some detail below (see table 2). For the case of the European parliamentary elections, the four
keywords common to all parties were queried: Europese Unie (“European Union”), klimaat
(“climate”), migratie (“migration”) and economie (“economy”).
Political parties and issue keywords
Before analyzing the presence and positioning of junk news in Google search engine, we would
like to discuss briefly the keywords obtained for the provincial election campaign from the
official websites and Facebook pages of the political parties. Comparing the composition of the
categories that emerged from each of the two political spaces allows for showing differences
between the matters of concern as expressed by political parties and citizens or social media
users. Whereas political parties included keywords that could be grouped under the categories,
‘future’, ‘security’, and ‘health’, that was not the case for the Facebook users. In contrast, ‘faith’
was present in the Facebook comment space, whereas it was largely absent from the party
platforms (except for the SGP, with its long list).18 There are also matters of concern common to
citizens and political parties alike, such as ‘foreign affairs’, ‘economy’, ‘society’, and
‘environment’.
Within the shared concerns there are still differences between the way each political space is
constructed by political parties or citizens. For example, within the ‘foreign affairs’, ‘economy’
and ‘society’ categories, parties tend to refer to a wider variety of issues in comparison to the
concerns expressed by citizens, which are mostly focused on the European Union and the
referendum. In the economy cluster, political parties address ten issues, whereas citizens are
concerned with far fewer. The same holds roughly for the society cluster. Interestingly, this trend
reverses in the environment cluster, where users tend to express concerns about nuclear energy,
while the topic is not present in parties’ platforms (again, with the exception of SGP). Lastly,
even though the usage of some keywords is more or less the same in party platforms and the
Facebook comment spaces, for others it differed, as is the case for onderwijs (education). By the
political parties it is framed as a societal issue, whereas in the discourse on Facebook it is
discussed in terms of citizens’ ability to afford it.
Visualizations were made to facilitate the analysis; they show at a glance the presence and ranked
position of fake and junk news for each query over time. The columns represent the days in the
timeframe studied (13-22 March or 22-22 May 2019). Red cells indicate the source as marked as
fake or junk news. One also may read the distribution of it over time.

18 Islam was present in the party platforms (largely the PVV and SGP) but discussed in terms of a social issue.

69

Figure 1. Presence of fake and junk news in Google.nl search engine results for political queries related to foreign
affairs, 13-22 March 2019.
Political party standpoint space
The study of the political party standpoint space found overall that all fake and junk webpages
that appeared can be subcategorized as hyperpartisan. The one exception fell within the
environment cluster in the form of a conspiracy website in the top twenty. Three specific
websites make up for the biggest amount of junk: De Dagelijkse Standaard, Stop de Bankiers, and
Opiniez. A second general observation is that queries related to right-wing parties returned
hyperpartisan sources in a greater proportion than queries related to parties with other political
orientations. In particular, the queries related to the FvD were the most populated by
hyperpartisan sources. Thirdly, queries related to parties located at the center of the political
spectrum seem to produce results with less questionable sources. In those instances where
hyperpartisan websites appear among the top twenty results for center-oriented parties, the sites
mainly do not make it to the top positions (though there were exceptions).
In the political party standpoint space, most junk appears to be associated with queries related to
keywords within ‘foreign affairs’ and ‘polarizing topics’. The keywords within the foreign affairs
cluster mainly relate to the European Union and the Dutch relationship with it. Issues related to
political parties from the center of the political spectrum seem to be least connected to junk, as
may be noted for 50Plus, Partij voor de Dieren or D66 (see figure 1). Contrariwise, parties that
position themselves strongly against the European Union are linked to a high concentration of
junk results. For example, when looking at FvD and PVV, we can see that hyperpartisan sites
account for 37% and 47% respectively of all the returns discussing the European Union.
Moreover, the hyperpartisan sources are among the top five results throughout almost the entire
time period.

70

Figure 2. Presence of fake and junk news in Google.nl search engine results for political queries related to
polarizing topics, 13-22 March 2019.
Regarding the keywords within the ‘polarizing topics’ cluster (figure 2), two related issues in the
political party-demarcated space lead to search engine results with a large amount of
hyperpartisan sources: Islam and Islamisering (Islamization). These keywords are only discussed by
two right-wing parties, PVV and SGP, the latter from the religious right. In particular, when
PVV is queried together with the keyword Islamisering (Islamization), hyperpartisan websites
appear at the top of the results throughout the entire time span, occupying even the highest
positions. This changes only for three days (16, 19 and 20 March), when, however, the amount
of junk increases overall.
The queries for environmental keywords (figure 3) also lead to significant quantities of junk.
First, it is of note that the amount decreased as the elections neared, though junk websites still
maintained a prominent position among the first 10 results. Once again, queries mentioning
right-wing parties such as FvD are more prone to produce junk sources within their results. The
site, ninefornews.nl, was identified here as the only conspiracy website in the entire data set. It
emerged in the 8th position when [milieu FvD -www.fvd.nl] was queried, meaning that it appears
on the first page of Google results (with default settings at 10). The center party, D66, also
registered signficant amounts of junk for the query climate (klimaat), as did the center-left labour
party (PvdA).
The cluster related to economic issues, the second largest in our set of results, contains nine
keywords such as belasting (taxes), eerlijkdelen (fair sharing) and werklozen (unemployed) (see figure
4). Queries including FvD are once again among the most prone to produce hyperpartisan
results, particularly with the keywords economie (economy), pensioen (pension) and belasting (taxes).
When examining the remaining parties (with the possible exception of the PVV), the number of
junk results is relatively low overall and rarely occupy the first position of the search engine
results.
The ‘society’ cluster is the largest; twenty issues were identified in this cluster, ranging from
concerns about warden (values) to drugs (see figure 5). Social issues produced a varied distribution
of hyperpartisan sources overtime, with hyperpartisan websites appearing less prominently. Only
three queries, two including the FvD and one the VVD, stand out in this cluster as junk-ridden:
drugs, democracy and responsibility (verantwoordelijkheid).

71

Figure 3. Presence of fake and junk news in Google.nl search engine results for political queries related to the
enviornment, 13-22 March 2019.

Figure 4. Presence of fake and junk news in Google.nl search engine results for political queries related to the
economy, 13-22 March 2019.

72

Figure 5. Presence of fake and junk news in Google.nl search engine results for political queries related to societal
issues, 13-22 March 2019.

Figure 6. Presence of fake and junk news in Google.nl search engine results for political queries related to future
innovation, 13-22 March 2019.

73

Lastly, the thematic cluster, ‘future developments’ (figure 6), with such issues as duurzaamheid
(sustainability), innovatie (innovation), and schone energie (clean energy), are addressed by parties
located at the extremes of the political spectrum and the query results are populated by
hyperpartisan sources. This is most evident in the case of FvD, where questionable sources
appear nearly every day, and in four instances are returned in the top two positions, albeit not in
close vicinity of election day, which remains largely unaffected by junk sources in relation to
these terms.
Vernacular political issue space
Based on the keywords gleaned from the Facebook pages of the Dutch political parties, one new
category was created (‘faith’) on top of the other six from the previous exercise. Generally, the
results were similar. Queries mentioning right-wing parties such as PVV and FvD returned more
hyperpartisan sources in the top 20 Google results, and the positions of these junk sources tend
to be higher than in queries related to other parties.
In the vernacular political issue space, the foreign affairs cluster received the most junk news
returns: nearly 25% of the top 20 results are considered hyperpartisan websites. Migration and
polarizing topics have the second and third highest percentage of junk websites, with 19% and
17% of junk, respectively. Results associated with keywords such as immigranten (immigrants) and
Islam were significantly populated with hyperpartisan sources. Issues related to environment,
society, and finance obtained fewer junk returns (less than 10%). The faith cluster had the fewest
hyperpartisan returns.19
The environment cluster (which for many terms could have been merged with the future
innovations one) is the largest one in the vernacular issue space. In it the keyword klimaat
(climate) was discussed by Facebook users on the pages of five different parties, and milieu
(environment) on three. For Groenlinks, D66, PvdA and VVD, the keyword klimaat features
(right-wing) hyperpartisan sources in the top results (see figure 7). From the election day
onwards, fewer junk websites showed up in the top 20 results. The same pattern was observed in
the ‘foreign affairs’ cluster (see figure 8). Hyperpartisan websites occupied the first position for
five days during our 10-day research period for queries related to FvD and PVV. Queries for
FvD and EU were the most likely to return fake and junk webpages.
Regarding the keywords grouped under the ‘polarizing topics’ cluster, Islam was brought up by
Facebook users in the pages of four political parties: PVV, SGP, FvD and DENK.20 Among
them, results associated with FvD received the most junk webpage returns in the top 20 results,
and the hyperpartisan results maintained the first position for seven days (figure 9). On election
day, however, all the parties returned few to no such results in the cluster of ‘polarizing topics’,
with the exception of the FvD. The number of junk sources and the rankings dropped
dramatically on that day. Concerning DENK, although queries including it did not return many

19 Here again Islam is excluded, because it is considered a social issue, given the manner in which it is discussed in

the vernacular issue space.

20 For the political party, DENK, Islam, while discussed as a social issue, also could be categorized as ‘faith’.

74

junk websites overall, they were found in relation to the issues moslim (muslim), discriminatie
(discrimination) and racisme (racism).

Figure 7. Presence of fake and junk news in Google.nl search engine results for political queries related to the
environment, using language from the comment space of the political parties, 13-22 March 2019.

Figure 8. Presence of fake and junk news in Google.nl search engine results for political queries related to foreign
affairs, using language from the comment space of the political parties, 13-22 March 2019.

75

Figure 9. Presence of fake and junk news in Google.nl search engine results for political queries related to
polarizing topics, using language from the comment space of the political parties, 13-22 March 2019.
The migration cluster (figure 10) was more prominent in the discussion space on Facebook
pages than in the platforms of the political parties. Keywords as migranten (migrants), migratie
(migration), immigranten (immigrants) and gelukszoekers (fortune hunters) were often mentioned in
the comments on the Facebook homepages of political parties. (Gelukszoekers could be said to be
a pejorative term for economic migrants.) In particular, results for three parties (FvD, PVV and
VVD) had junk webpages on the top 10 Google search results, and they were spread evenly over
the ten-day research period. It is also noteworthy to see that there appears to be a decreasing
tendency of junk news from the election day onwards in this cluster, both in terms of the
amount as well as the rankings.

Figure 10. Presence of fake and junk news in Google.nl search engine results for political queries related to
migration, using language from the comment space of the political parties, 13-22 March 2019.
European Parliamentary Elections
The results for the political party standpoint space during the European Parliamentary elections
also show a consistent presence of junk news. Three major findings are worth mentioning.

76

Firstly, as figures 11 and 12 make evident, the presence of junk sources in the Google search
engine results was lower during the European parliamentary elections than during the provincial
elections. In many cases, our queries combining issues with parties did not produce links to
hyperpartisan material or only did so for one day. Only 25% of our queries returned junk for
more than one day during the three-day period under research. Of the four keywords queried,
the one that produced the least amount of junk in combination with party names was economie
(economy). In contrast, migratie was the most prone to returning junk, with queries related to five
parties consistently returning junk websites between May 22 and 24 (see Figure 11).
Queries in combination with the political party DENK were particularly junk ridden. On the day
of the election and the day after, almost half of the results provided by the Google search engine
are problematic. It is particularly relevant that during the 24th of May, hyperpartisan websites
occupied the top seven positions.
A second relevant finding is that the presence of hyperpartisan resources is more prevalent in the
day after the election than the day before, in contrast to what happened during the provincial
elections. Although this is the case for all parties and keywords where junk sources were
identified, it was especially prominent for the issues of migration, economy and climate
(keywords migratie, economie and klimaat). In most cases, junk sources also occupy the first
positions in the results during the 24th of June. We cannot answer the question of whether this
phenomenon can be credited to the Google search algorithm reacting to an increase in searches
related to the elections, to a surge in the activity of hyperpartisan websites after they took place,
or to some combination. A close reading of the results in the highest positions, however, shows
that the hyperpartisan sources behave as one would expect from any information provider
during election season, if only keeping their radical tone; before the election they provide
predictions about the results, on the day of the election they invite citizens to vote, and on the
day after they discuss the results. An article by the De Dagelijkse Standaard that consistently
occupied the top result for various keywords and parties consists of a reflection about the FvD
and leader Thierry Baudet’s need to tone down their radical discourse in order to become a more
powerful political force.
Lastly, whereas during the Dutch provincial elections the queries related to right-wing parties
were more strongly connected by the search engine with junk sources, during the European
elections this is not the case. For example, although queries performed in May involving FvD
also produced results pointing to hyperpartisan websites – similarly to the results obtained in
March – it was those related to DENK which, in aggregate terms, produced more junk (i.e., 25
for FvD and 27 for DENK). However, the case of DENK is difficult to assess given that, apart
from the name of a party, it is also a common Dutch word (“think”, in English). Although this
does not change the fact that people looking for information about this party were very likely be
exposed to junk sources, the content may not specifically relate to DENK. In fact, the highestranking result for the query [migratie DENK -bewegingdenk.nl] is an article in the hyperpartisan
webiste De Dagelijkse Standaard that makes no specific mention to this party, but instead generally
reflects on the results of the elections and specifically about the demise of the PVV.

77

Figure 11. Presence of fake and junk news in Google.nl search engine results for political queries related to
migration and European Union issues, 22-24 June 2019

Figure 12. Presence of fake and junk news in Google.nl search engine results for political queries related to
Cilmate and Economy issues, 22-24 June 2019

78

Conclusions: Junk in search engine results
The goal of this research is to locate the presence and ranking of fake and junk websites within
the first twenty Google.nl results of queries concerning Dutch political parties and their most
significant issues during the 2019 provincial and European parliamentary elections. The keyword
queries were built by combining political party names with keywords retrieved from political
party platforms and party Facebook page comment spaces (in the case of the provincial
elections). We clustered the keywords into categories to enable a comparative analysis. At the
outset the research questions were as follows: Which rankings has the search engine assigned to
fake and junk sources when querying political parties and their issues? Are there particular
political issue and party spaces where fake and junk news are prevalent or largely absent? Do the
ranks and amounts increase as the elections draw near?
Our results indicate that the junk websites present in the results of our Google.nl political
queries are almost exclusively hyperpartisan, rather than sources spreading disinformation,
conspiracy theories, or clickbait. Three websites, namely De Dagelijkse Standaard, Stop de
Bankiers and Opiniez, account for the largest portion of the junk sources identified. We did not
find fake advocacy groups (nepactiegroepen) or foreign disinformation operatives. Furthermore,
we found that queries involving right-wing parties were more prone to result in exposure to
hyperpartisan sources than those associated with centre-left or left-wing parties. For most
keywords, hyperpartisan websites appeared in the top positions, and certainly always within the
first page of results. Our findings suggest that on Google.nl there is a considerably high
probability that junk news is outputted on the first page of results when the queries concern
right-wing parties and their issues.
Another finding is that hyperpartisan results spread unevenly during our research period. For the
case of the Dutch provincial elections, they are more present before the day of the election and
drop their presence and ranking, in some cases dramatically, on election day and in the
immediate aftermath, for instance in the case of migration issues in the vernacular issue space. In
contrast, during the European parliamentary elections this phenomenon reversed, and junk
sources were more prominent the day after the elections.
Concerning the comparison between the two spaces, the vernacular space had the highest
percentage of junk news returns, especially in the categories of foreign affairs and migration. In
the political party standpoint space, the differences among themes is smaller. In other words, the
queries designed with the political language of Dutch Facebook users were more likely to result
in hyperpartisan results than the queries built with the standpoint language of political parties.
Although more research is needed in this regard, such a finding suggests that the discourse of
normal citizens, or those commenting on party Facebook pages, is more politically contentious
than that of political parties.
The results are indicative of the amount of hyperpartisan material in political space in Google.nl
rather than conclusive, for they derive from a particular query strategy and not from multiple
strategies and are only a snapshot from a particular event-related timeframe. The data set we
built could also be read more closely, and additional fake or junk sources could be found,
meaning that we could have undercounted (rather than overcounted).
79

Given that our intention was to determine the sensibility of the Google search engine to junk
news, the question remains whether the location of the results of politically charged queries can
be credited to an optimisation effort on the side of hyperpartisan content generators, an overall
susceptibility of Google’s search algorithm to provide questionable content to its users,
consumers’ preference for low quality information, or some combination of the three.
While discussing the two latter hypotheses would require more space, a few words can be said
about the first. A possible way to detect Search Engine Optimization strategies consists of using
one of the many online services providing SEO analysis. However, given the proprietary nature
of their methodologies, the results should be interpreted with caution. We submitted the three
most recurrent junk websites we found during our research to the service SEO Tester Online, a
tool that measures a website’s readiness to achieve top positions within search results. This tool
provides analysis in four different categories: basic, related to the overall online presence of the
website, content, which measures the richness of the keywords that trigger the website to pop up
in search results, web performance, which measures how fast the website can be rendered in
mobile and desktop devices, and social, which provides information about the websites
engagement with users through social networks. De Dagelijkse Standaard obtained a score of
56/100, faring the lowest in their web performance, and the highest in its social engagement.
Opiniez obtained a score of 62/100, faring the best in web performance and the lowest in
content, although it obtained an excellent score in the number of unique keywords. Stop de
bankiers obtained the highest score, with 74/100, including a 100/100 in the assessment of
keywords which may lead to the site through search engines. It also fared the best in social
engagement and the lowest in web performance.
The reports produced by SEO Tester Online, which for all websites included a considerable
amount of suggestions about how to improve the websites’ code, suggest that, at least currently,
content producers are not actively seeking to artificially alter the performance of their sites.
Consequently, the presence of these junk sources in our analysis is more likely related to an
inherent susceptibility of Google’s search engine or an organic result of users’ preferences.
Further research is necessary to paint a clearer picture regarding the increased consumption of
junk news. One could repeat the work for longer periods of time in order to ascertain the extent
to which the presence of these or other dubious source types is becoming more widespread or
even disappearing in the top results for political queries, as is currently the case with clickbait.
Furthermore, specific sources could be monitored over time to track the performance of their
content (and the changes to their code), in order to detect attempts to artificially increase
relevance and thus the ranking assigned by search engine algorithms.

80

TWITTER, JUNK NEWS AND TROLL-LIKE USERS
DURING THE DUTCH PROVINCIAL
AND EUROPEAN ELECTIONS

Sabine Niederer and Maarten Groen

Twitter, junk news and troll-like users during the Dutch provincial and European
elections
Sabine Niederer and Maarten Groen21
Introduction
In 2018 the Dutch daily newspaper De Volkskrant published an article entitled “The troll army of
pop artist Dotan” (“Het trollenleger van popartiest Dotan”), which revealed how the Dutch
singer-songwriter had made use of fictitious accounts pretending to be fans (Misérus and van der
Noordaa, 2018a). The fake fans were highly active across social media platforms (including
Facebook, Twitter, and Instagram) where they circulated heart-warming stories about the artist,
requested his songs on Dutch and German radio stations, and actively tried to edit the Wikipedia
pages about the artist and his mother (who is also a Dutch singer). At the root of these activities
were 140 accounts that the newspaper retrieved, at least one of which connected directly to the
artist’s own Gmail account, and others to accomplices. Dotan’s case is perhaps the most-known
example of artificially boosted accounts and content in the Netherlands, but certainly not the
only known case of such behaviour. The politicians Geert Wilders and members of the political
party Denk were found to have suspiciously inflated follower counts, which surfaced when
Twitter started deleting unvalidated accounts (NOS, 2018).
The present study builds on previous digital research in which the social media platform Twitter,
used by over 326 million monthly active users accounting for 500 million tweets per day, is
repurposed for social research (Omnicore, 2019). As with Dotan and the Dutch politicians
mentioned above, it similarly looks into social media use and the question of manipulation, in
particular in political spaces around elections. It studies troll-like and artificial boosting as well as
the circulation of fake/junk and tendentious news sources during two election campaign periods
in 2019. Initially intending to detect the possible presence of Russian disinformation in the
Dutch Twitter space, the study enquires into coordinated campaigning around divisive issues and
ascertains the extent of homegrown junk news in Dutch political Twitter, including
hyperpartisan, conspiracy and clickbait sources. So-called (and self-identified) tendentious
sources such as Geenstijl.nl and TPO.nl are labelled as such, and one could argue that they are
mainstreaming, given how they are shared, as we discuss below. These two sources are part of
the “anti-establishment established source” set, and as such are closely related to an emerging
right-wing media landscape (see Tuters, this volume).
In employing digital research methods and techniques, the analysis makes use of the platform’s
own features and cultures of use, which offer built-in structurings of the content being shared
(Rogers, 2019). These are repurposed for social or political research. Hashtags can be repurposed
as content categories or issue activity indicators, retweeting suggests “pass-along value”, and the
@reply and @mention functionalities network users and their content to fellow users and
content (Niederer, 2018). Through an analysis of @replies, Twitter can be studied “as a
conversation-maker, where one may explore the extent to which there is dialogue, or
21 The research reported here was undertaken in collaboration with Layal Boulos, Peter Fussy, Oana Patrici, Maria

Stenzel Timmermans, Emile den Tex, Carlo De Gaetano and Federica Bardelli.

81

broadcasting” (Honeycutt and Herring, 2009; Boyd et al., 2010). The @mentions may contribute
to the inquiry of dominant voice - certain understandings of issues can be shaped by the actors
most mentioned in a tweet corpus, and also by the actors that are the most vocal. Twitter can be
studied as a social network of professional information-sharers (Java, 2007). It also can be
considered a ‘rebroadcaster’ of (political) news, in which the platform’s built-in algorithms
reinforce the issues and framings discussed there as so-called trending topics (Kwak et al., 2010).
Furthermore, Twitter is often moving information faster than the news, and Twitter content in
some cases becomes news (Niederer, 2018). As news and mass media sources strive to make their
content “platform-ready” (Helmond, 2015), political news, other mass media content and new
platforms become further entangled, forming a hybrid media system (Chadwick, 2013). Here,
professional journalists include tweets in their stories, and when their work has been published,
they may post a link to that article on Twitter and other social media, using the platforms both as
a source of information and as a channel for the distribution of their own work.
Critiques of digital social research take issue with its dependency on the already problematic
hegemony of proprietary social media platforms. On a methodological level, scholars warn of the
sheer impossibility of distinguishing between the working logic of web platforms and exemplary
‘platform artifacts’ (Marres, 2015; Marres & Weltevrede, 2013; Rogers, 2013; Niederer, 2017).
How do we know whether the most-retweeted Twitter post is the most relevant, or the most
Twitter-friendly (Marres, 2015)? One way to approach this issue is to take into account the sociotechnical specifics of each platform, and to regard Twitter and other social media platforms as
distinct windows on an issue. Rather than questioning the relevance of the platform for the
elections, we then ask: how does Twitter present the elections? And how does this compare to
how other social media platforms cover the topic? Such lines of questioning open up avenues for
qualitative and empirical digital research across political events and social issues as they resonate
online and offer insights into the cultures of use of the various platforms. In this present study,
Twitter can be seen to produce political subspaces around divisive issues, in which a relatively
small number of highly active, troll-like users sow division and where fake/junk news at times
outperforms mainstream news.
Troll-like user activity during the 2017 Dutch general elections
The present study follows from an earlier one, which itself concerned Dutch elections. In the
lead up to the 2017 Dutch general elections for the national parliament, journalists revealed the
use of sock puppets (i.e., false online identities assumed to deceive and influence opinion) by the
political party Denk, in order to amplify their online messages and attack their political
opponents on Twitter and Facebook (Kouwenhoven and Logtenberg, 2017). In an empirical
study as part of the Field Guide to Fake News (Bounegru et al., 2017), we studied troll-like
behaviour in Twitter, developing a research protocol for identifying and analysing political
trolling, which in this case referred to repeated attacks of politicians on Twitter. It focused on
the sources of troll-like activity (i.e., which user accounts target politicians?), their targets (who
do these troll-like users address?), and the characteristics of these practices (what do troll-like
users do?) (Borra et al., 2017).
The detection of user accounts engaging in political trolling behaviour starts by compiling a list
of potential targets. The aforementioned study looked into the user accounts of 28 political party
82

leaders participating in the 2017 elections. The users that @-mention them were queried. For the
most-active users per @mention, their posts in which they @-mention the political leaders were
qualitatively studied. In a next step, only those who @-mention one or more political leaders at
least 100 times during a one-month period (8 February - 8 March 2017) were retained, and their
tweets coded for being favourable or unfavourable of the politician. The study found an
asymmetry in the troll-like behaviour across the political spectrum, as more left-wing politicians
were being targeted by negative mentions while most right-wing politicians were receiving
support (see figure 1). There are exceptions, such as Emile Roemer (SP) and Marianne Thieme
(PvdD), who in this time frame received only support by troll-like users, and Prime Minister
Mark Rutte (VVD) who received unfavourable mentions, in particular on his personal account
though less so on his official @MinPres account.

Figure 1: Political party leaders as troll targets on Twitter during the 2017 Dutch general elections. Red indicates
proportion of targeting. Source: Borra et al., 2017.
To classify the sources of political trolling, we used the same list of 24 highly active and troll-like
users (mentioning political leaders at least 100 times in the one-month time frame), and collected
their profile information (description, profile picture and banner) from the Twitter interface. If
the profiles had a profile picture, Google reverse image search was used to check these images
for authenticity. Then, using the Twitter API, the creation date for each of these accounts was
retrieved, in order to assess whether accounts in our dataset had been created on the same date.
This analysis provided a more nuanced view of the user accounts responsible for the trolling
behaviour. Of the 24 accounts still active at the time of study, three users appeared to be sock
puppets created for trolling activities. They had very similar profiles and had been created within
a short time-frame. Another six accounts in the data set promoted the same anti-Islam agenda,
but were not determined to be fake accounts.

83

To characterise the substance of the trolling practices, the study looked at the issues and the
media sources that resonate in the set of tweets. To identify the issues, the hashtags used by the
highly active and trolling users in their tweets (that @mention a political leader) were collected
and analysed. Most tweets that include hashtags were found to mention the right-wing populist
candidate Geert Wilders, and most hashtags referred to the issues in PVV’s political messages
from 2017 (“Nexit”, “StopIslam” and “BanIslam”), as well as those pertaining to expressions of
Dutch patriotism (Borra et al., 2017: 188). To assess which media sources were circulated by the
troll-like users, the most-circulated URLs in the tweets were collected and categorised. For the
2017 general elections, the most-tweeted media sources by the 24 trolling users were the Dutch
alt-right blog fenixx.org followed by the anti-Islam site Jihad Watch and the right-wing think tank
Gatestone Institute (Borra et al., 2017: 192).22
Research questions and data collection
For the study presented in the next section, the main research question is to what extent
fake/junk news sources and troll-like users are present on Twitter around both the provincial
and the European parliamentary elections in the Netherlands in 2019. To answer these
questions, we examine Twitter activity around the elections, the party leadership as well as
political candidates, and zoom in on potentially divisive issues, including Zwarte Piet and MH17.
In addressing these research questions, queries were formulated to demarcate the political and
issue spaces in Twitter (see table 1). The data was collected using the commercial social media
monitoring tool, Coosto, and the Twitter Capturing and Analysis Toolkit developed by the
Digital Methods Initiative (DMI-TCAT). Coosto was used to retrieve data from both the
provincial and European election periods, in order to conduct a comparative analysis of the
engagement with mainstream and fake/junk news across political and issue spaces, and the
presence of troll-like users in these spaces, as discussed in detail in the next sections. DMITCAT, a tool that “provides robust and reproducible data capture and analysis and interlinks
with existing analytical software” (Borra & Rieder, 2014), was used to analyse the engagement
with fake/junk and tendentious news sources and the users responsible for this engagement.
While some collections (or ‘bins,’ in the terminology of the TCAT-tool) were created only for
this study, others had been running for months prior, such as MH17, or in the case of Zwarte
Piet even years (with a bin that was created in December of 2017). The set for the Utrecht tram
shooting was created on the day that event took place, 18 March 2019. For this study, the sets
were limited to the provincial elections campaign period (18 February - 25 March 2019) and the
European election campaign period (26 April -24 May). The one exception was the Utrecht tram
shooting set, which was only included in the Provincial Elections campaign period, as it took
22 It is important to note the ethical implications of these lines of empirical inquiry. With the right tools, it is

relatively straightforward to assess which users are involved in attacking or amplifying political leaders. A qualitative
analysis can subsequently give further insights in the authenticity of a particular account and give context to the
political views it pushes by way of media circulation analysis. However, users who have created sock puppets may
make mistakes, too. It can happen (as it did in the 2017 study) that a fake user account created for trolling purposes
suddenly posts a proud story about his/her granddaughter, or a picture of him/herself in front of their newlyopened flower shop. It can be easy to then identify the people behind such activity, and publishing their names
becomes a consideration of weighing public interest against ethical and possibly legal privacy implications. In the
field guide, we thus mainly focused on networks and relationships between user accounts rather than singling out
users. We redacted personal identifying information from the data set and the visualisations.

84

place during that time frame.

Elections

Topic

Query

PS

General

Ps2019, Ps19, verkiezingen

EU

General

EUverkiezingen2019, euverkiezingen, ep2019, eu2019,
euelections2019, verkiezingen, verkiezingen2019, EU, Europa,
Europese Unie, europeseverkiezingen

PS

Party leaders

Mark Rutte, MinPres, markrutte, Geert Wilders, geertwilderspvv,
Thierry Baudet, thierrybaudet, Jesse Klaver, jesseklaver, Rob Jetten,
RobJetten, Liliane Marijnissenm, MarijnissenL, Marianne Thieme,
mariannethieme, Gert-Jan Segers, gertjansegers, Lodewijk Asscher,
LodewijkA, Tunahan Kuzu, tunahankuzu, Henk Krol, HenkKrol,
Klaas-Jan Dijkhoff, dijkhoff, Sybrand Buma, sybrandbuma,Kees van
der Staaij, keesvdstaaij

EU

Party leaders

SophieintVeld, Esther_de_Lange, mjrldegraaff, malikazmani,
arnouthoekstra, TimmermansEU, petervdalen, BasEickhout,
anjahazekamp, ToineManders, florens0148, atonca, paulbeasd,
djeppink, sentwierda, RLanschot, MinPres, markrutte,
geertwilderspvv, thierrybaudet, jesseklaver, RobJetten, MarijnissenL,
mariannethieme, gertjansegers, LodewijkA, tunahankuzu, HenkKrol,
dijkhoff, sybrandbuma, keesvdstaaij

PS and
EU

MH17

mh17

PS and
EU

Zwarte Piet

Zwartepiet, zwarte piet

PS and
EU

Climate

klimaat

PS and
EU

Fake news

Fake news, fakenews, nepnieuws, desinformatie, junknieuws

PS

Utrecht

utrecht, 24oktoberplein, gokmen tanis, gokman tanis

Table 1: Query overview showing the election campaign period (Provincial, EU, or both), the political or issue
space and the query made resulting in Twitter data sets.

85

Junk/fake news sources and troll-like users during the provincial elections on Twitter
During both the provincial and the European election campaigns we tracked the resonance of
mainstream, junk and tendentious sources in Twitter. We did so around the potentially divisive
issues of Zwarte Piet and MH17 and chose to include climate and fake news (as an issue).
Furthermore, we tracked the resonance of news sources for the political spaces of the (Provincial
and EU) elections, as well as the party leadership, and political candidates. For each of the
elections, we demarcated a five-week campaign period. For the Provincial elections this was 18
February - 25 March, and for the EU parliamentary elections in the Netherlands, 26 April - 24
May. Per political and issue space, and for each of the five weeks of the campaign, the mostshared links (up to a maximum of 500) were collected and coded (for mainstream or junk news
of various types, using the aforementioned expert list). The engagement scores for the
mainstream and fake/junk news source engagement per week were visualised as line graphs, as in
the well-known Buzzfeed News study (Silverman, 2016).

Figure 2: Line graphs visualising the engagement with mainstream news (blue) and fake/junk news sources
(magenta) during the Dutch Provincial election campaign (left) and the European Election campaign period
(right). Visualisation by Federica Bardelli.
For both election campaign periods, overall the mainstream news outperforms fake/junk news
(see Figure 2). When zooming in on the political spaces of the elections and the party leadership
and political candidates, the mainstream news sources garner far more engagement than
fake/junk news. A look at the top 500 most-engaged with links shows the rise and fall of
mainstream hosts circulated in the issue space, and the relatively small but steady resonance of
fake/junk news hosts, which during the provincial election campaign rises slightly in its last
week.

86

Figure 3: Line graphs visualising the engagement with mainstream news (blue) and fake/junk news sources
(magenta) for the issue of MH17 (top right) and Zwarte Piet (top left) during the Provincial elections, and the
EU elections (bottom right and left). Visualisations by Federica Bardelli.
Divisive issues: Zwarte Piet and MH17
Both for Zwarte Piet and MH17, there are instances in which junk/fake news outperforms
mainstream news. The line graphs in figure 3 include a zoomed-in view that render visible the
moments in which junk/fake news is more engaged with than the mainstream news. For the
controversial topic of Zwarte Piet, during the Provincial election period mainstream news
receives more engagement. Fake/junk news outperforms mainstream news in weeks three and
four of the European parliamentary elections campaign. The article mostly responsible for this
peak in week three is a short commentary on tendentious-hyperpartisan website tpo.nl about the
proposal by Dutch politician Sylvana Simons (addressed to the Amsterdam Municipality) to ban
the “racist caricature of Zwarte Piet” in the city of Amsterdam. (When one removes TPO from
graph, the results remain the same apart from the one week in May during the European
parliamentary election period where now mainstream news outperforms junk (see Appendix)).

87

Thus in that one week in May, the ) In week four, an article on Cultuurondervuur.nu (culture
under fire) entitled ‘Jerry Afriyie receives funding for anti-Zwarte Piet educational materials’ is
responsible for the increased activity. In it activist Jerry Afriyie is described as a “Zwarte Piet
hater” (cultuurondervuur.nu, 2019).
For the issue of MH17, during the Provincial elections campaign there are times in which
fake/junk news outperforms mainstream news in terms of engagement. For the European
parliamentary elections, the mainstream attracts more engagement, but during certain periods
junk/fake news performs on a similar level of engagement as the mainstream. The peaks that
occur during the Provincial elections are mainly caused by engagement with a piece from citizenjournalist Max van der Werff, on his website kremlintroll.nl, in which he demands rectification of
an article in De Groene Amsterdammer (from August 2018) about Russian internet trolls (van der
Werff, 2019). Two other articles that attract engagement are from the hyperpartisan website
jdreport.nl, questioning the integrity of the MH17 investigation, and in one Frans Timmermans
(who would win a seat for the PvdA in the EU parliamentary elections) is named as part of an
‘MH17-doofpot’, or cover-up (jdreport.nl, 2019). In week four of the Provincial elections
campaign period, the Kremlintroll piece requesting rectification is particularly actively shared.
Simultaneously, the interlinked article with the actual critiques of the article from De Groene
Amsterdammer is receiving more engagement.
During the EU election campaign, mainstream news receives more engagement. It is important
to note, however, that aside from a peak in mainstream news in week three of the campaign, its
engagement level is equal to that of fake/junk news sources. Where in the mainstream certain
events cause peaks in media coverage, it appears that for fake/junk news these divisive issues are
continuous and year-round. Zwarte Piet may not be a subject matter in the mainstream news in
Springtime, but it remains a matter of concern and a source of engagement in fake/junk news
media.
Troll-like users during the Dutch provincial and European elections on Twitter
For the Dutch provincial elections campaign period, a next step in the study is to look closely at
the user activity related to the Dutch provincial elections and the political party leadership, as
well as coverage of the potentially divisive issues of Zwarte Piet, MH17 and the Utrecht tram
shooting. As a first step, the URLs (hosts) were extracted from the sets of tweets and checked
against a collaboratively compiled expert list of junk/fake and tendentious news sources.
Similarly, the users active in each of the sets of tweets were checked against a list of flagged
users. Here, we made use of existing lists from the previous project in The Field Guide to Fake
News (Borra et al., 2017) and expanded these lists. To do so, we extracted top users from the data
sets of Zwarte Piet, MH17, Utrecht tram shooting, the Dutch provincial elections and the
political party leadership and followed an extensive protocol adapted from the aforementioned
study, and combined them with research on credibility metrics (Groot et al., 2019; Borra et al.,
2017).23
23 For this particular study, to identify potentially suspicious users in the data sets, the top 15 most-active users in the

set were selected, as well as the top 15 users who were highly active yet at the same time very low on visibility (i.e.
rarely or not at all @mentioned). Then, the profiles of these user accounts were checked for the following red flags:
mostly retweeting, or retweeting in several languages (as possible indicators of automation) which is of interest given

88

With the Compare List tool (Borra, 2013), the study assessed whether any of the flagged users
were active in one or more of the political issue spaces. Zwarte Piet had an initial list of 26
potentially suspicious accounts, five of which had already been taken offline at the time of
inquiry. Of the 21 remaining each was classified as suspicious; one of which described itself as a
retweet bot (in the user profile). For MH17, of an initial list of 26 potentially suspicious user
accounts, two were inactive at the time of inquiry. Of the remaining user accounts, 13 accounts
were flagged as suspicious, and 10 were not. For the Utrecht tram shooting, from an initial list of
23 potentially suspicious accounts, 10 were flagged as suspicious after examination. For the
provincial elections dataset, the list of potentially suspicious users entailed 24 accounts, 17 of
which were flagged according to our criteria and one of them described itself as a bot.
Subsequently, these flagged users were checked for activity in more than one issue. This would
make sense for those data sets that are of related topics, such as the provincial elections and the
political leadership. When users are active across distinct controversial issues such as Zwarte
Piet, MH17, and the shooting in Utrecht, which have in common their potential divisiveness,
such multi-issue users and the content they circulate would be further scrutinised. In fact, 14
suspicious accounts are common to all of the five political issue spaces, and as many as 29
suspicious users are common to four of the data sets, pointing to a suspicious effort to fuel
division during the election period.
Figure 4 gives an overview of the most-resonating hashtags, tweets, and sources as well as the
most-mentioned and most-active users during the time around the elections (18 February – 25
March 2019). Flagged users as well as fake/junk and tendentious news sources are highlighted,
with fake/junk being broadly defined to include hyperpartisan sources, as well as conspiracy,
clickbait and disinformation. The overview shows that there is no disinformation resonating in
the top 10 hosts per political and issue space. The top hosts are mostly (Dutch and international)
mainstream news media. The hyperpartisan/tendentious site Opiniez.nl is among the top 10
hosts for Zwarte Piet and the provincial elections space, and geenstijl.nl is shared for MH17 and
PS2019. Fake/junk sources are present across political and issue spaces around MH17, Zwarte
Piet, Utrecht, PS2019, and the Dutch party leadership. There are fake/junk news hosts that are
common across all five issues: Ninefornews.nl, fenixx.org, tref.eu, ejbron.wordpress.com,
drimble.nl (a particular story), and dagelijksestandaard.nl. Hyperpartisan and conspiracy sources
are mostly circulated by flagged users. However, some hyperpartisan and tendentious sources are
being mainstreamed, and circulated by regular (non-suspect) users. These include Geenstijl, which is
considered part of the mainstream but retained here because of its self-proclaimed tendentious
characteristics, tendentious-hyperpartisan host TPO, and hyperpartisan sources, De Dagelijkse
Standaard and Fenixx.

the wide distribution of easily acquirable retweet bots (McGarry, 2013); profile oddities such as suspicious user’s
profile images, which were checked with Google Image search to assess their authenticity, a recently created
account, or a high following count (of over 1,000), a username with over 3 numbers in it; and suspicious tweet
frequency as tweeting over 200 times mentioning the issue, posting 20 tweets or more times per day; and, whether
the user seems to mostly retweet more often rather than tweet his/her own content.

89

Figure 4: Top hashtags, users, @mentions, hosts and tweets during the Dutch provincial election period of 2019.
Visualisation by Carlo De Gaetano.
Looking at the time frame around the elections, flagged users are among the top most-active
users across issues. In particular for Zwarte Piet and MH17, six of the top ten users are flagged
accounts. The top @mentioned users vary per issue. Two flagged user accounts are among the
top 10 @mentioned users in tweets about Zwarte Piet and MH17. When analysing the mostused hashtags across the issues, what stands out are that the top hashtags used in the MH17 issue
space all seem to be Pro-Russian. Across the issue spaces of Zwarte Piet, MH17 and PS2019, we
see the resonance of right-wing political party hashtags, such as PVV and FvD. Zwarte Piet
contains hashtags both for pro-Zwarte Piet (e.g., ‘blokkeerfriezen’, referring to the Frysian
counterprotest in Dokkum against anti-Zwarte Piet protesters of ‘Kick out Zwarte Piet’, which
90

can be found in the data set with hashtag kozp, in which they blocked the highway to prevent
anti-Zwarte Piet protesters from entering their town) and anti-Zwarte Piet, e.g.,
‘SamenTegenRacisme’, which translates as ‘united against racism’.
For the EU election campaigns, we similarly investigated the activity of flagged users in the
political and issue space. For the political space, the top 1000 most active users were collected
for the general EU election hashtags and the political leaders relevant to the EU election
campaigns. For the issue spaces, the top 1000 most active users were collected on the topics of
climate change, Zwarte Piet, MH17 and nepnieuws (fake news). These lists of top users were
matched with the suspicious users list from the first part of the empirical study. Because some
topics were more active than others, the activity of the top 1000 users varies per dataset. For the
more generic EU set, the top 1000 users each posted more than 44 tweets in the EU election
period. In comparison, in the Zwarte Piet dataset the top 1000 users each posted two or more
tweets.
Of the flagged users list, eight users were active in all six issue spaces during the EU campaign
period. Three users were active in five of the spaces and another three users in four of the
spaces. Four of the eight users active in all spaces were also active in all the provincial election
period datasets. From the users active in all datasets, the top user posted 2781 tweets. 2578 of
those tweets were in the general EU and party leader dataset. This user is not only retweeting
other content, but also posts his own content. The content in the EU Elections period can be
characterized as anti-EU, anti-immigration, pro PVV/FvD and critical of all other parties.
The circulation of fake/junk and tendentious news during the provincial elections
To gain a better view of these troll-like, fake/junk and tendentious news activities, a next step
zooms in on the circulation of these news sources during the campaign period in each of the
political issue spaces. Visualised as network graphs, the analysis considers whether such news
sources are circulated by flagged or regular (non-flagged) users.24 Each host-user bipartite
network graph includes a short overview of the user and host types per data set, clearly
illustrating that the number of flagged users and the circulation of fake/junk or tendentious news
sources is outnumbered by unflagged users and the circulation of mainstream news. Thus, these
visualisations should be read as a zoom-in on a particular, small set of hosts that are of interest
to the study of the presence and circulation of fake/junk and tendentious news and the users
that circulate them.
In each issue space, hyperpartisan sources are circulated the most. And while the issue space of
Zwarte Piet is dominated by the circulation of hyperpartisan sources being shared by flagged but
also by regular users, the main fake/junk sources for MH17 are more diverse in composition.
Here, we see a mix of tendentious, hyperpartisan, as well as conspiracy hosts. For the Utrecht
shooting, tendentious and hyperpartisan hosts are circulated the most, by flagged and regular
users, making them appear as mainstream. The fake/junk and tendentious sources in both of the

24 Regular in this case in fact strictly speaking means not identified as suspicious. There could be suspicious users among

them, which have not (yet) been identified as such.

91

political spaces, PS2019 and the party leaders, revolve around mostly hyperpartisan and
tendentious sources.

Figure 5: Gephi visualisation of Zwarte Piet host-user network during the Provincial elections campaign period,
depicting only fake/junk and tendentious hosts and the users that circulate these sources. Visualisation by Carlo
De Gaetano.
The host-user network of the Zwarte Piet issue space (Figure 5) is dense and, as said, is
dominated by the circulation of hyperpartisan sources such as dagelijksestandaard.nl, fenixx.org,
cultuurondervuur.nu and opiniez.nl, and, at a slightly lower level, the tendentious source
geenstijl.nl. These central nodes are the sources of choice for the majority of the flagged users,

92

but also have been shared by regular users, who demonstrate a preference for the hyperpartisan
source, dagelijksestandaard.nl. One clickbait host (tpook.nl), which can be found in the outskirts
of the graph, stands out as being circulated by both flagged and regular users.

Figure 6: Gephi visualisation of MH17 host-user network during the Provincial elections campaign period,
depicting only fake/junk and tendentious hosts and the users that circulate these sources. Visualisation by Carlo
De Gaetano.
The network visualisation of the MH17 fake news source circulation (Figure 6) shows a different
source composition to that of Zwarte Piet which had hyperpartisan sources at its core. For
MH17, we see a more diverse set of sources central to the network: tendentious source
geenstijl.nl, hyperpartisan/conspiracy source novini.nl, and a set of two other conspiracy hosts
(ninefornews.nl and niguru.co), which have been widely circulated by flagged users.
The flagged users in this issue space mostly circulate tendentious hosts, such as geenstijl.nl, and
hyperpartisan and conspiracy sites, herstelderepubliek.wordpress.com and novini.nl. The source
most circulated by regular users is the tendentious geenstijl.nl.

93

Figure 7: Gephi visualisation of Utrecht shooting host-user network during the Provincial elections campaign
period, depicting only fake/junk and tendentious hosts and the users that circulate these sources. Visualisation by
Carlo De Gaetano.
In the issue space for the Utrecht shooting (Figure 7), tendentious and hyperparisan sources
(geenstijl.nl, tpo.nl and dagelijksestandaard.nl), populate the center of the network. Several
smaller clusters of fake/junk news sources that have been circulated by regular users are evenly
distributed on the periphery of the graph (e.g., drimble.nl (story-level), evendelen.net,
dagelijksekrant.nl or hardwaarheid.nl). Only a minority of flagged users circulate clickbait
(tpook.ml, nietbarkie.nl) and conspiracy pages (martinvrijland.nl, ninefornews.nl,
brekendnieuws.nl, ellaster.nl, wanttoknow.nl). It is important to note that overall the
hyperpartisan and tendentious sources in this network have been circulated by both flagged and
regular users, making them appear to be mainstream(ing).

94

Figure 8: Gephi visualisation of PS2019 host-user network during the Provincial elections campaign period,
depicting only fake/junk and tendentious hosts and the users that circulate these sources. Visualisation by Carlo
De Gaetano.

95

The PS2019 (Provincial State elections) host-user network appears to be organised around two
major hosts, hyperpartisan source opiniez.com and tendentious source geenstijl.nl (Figure 8).
The (marginal) presence of clickbait host aboutmedia.nl, is caused by the activity of only one
regular user. Conspiracy hosts ninefornews.nl and dimplus.nl have been only marginally
circulated by users who also shared other fake/junk hosts. Two recently created user accounts in
the network (created in December 2018) demonstrate an uncommonly high number of tweets
and likes. One of them has around 39,300 posts, and 31,900 likes within four months of
existence, a level of activity that suggests automation and artificial inflation.25

Figure 9: Gephi visualisation of Party Leadership host-user network during the Provincial elections campaign
period, depicting only fake/junk and tendentious hosts and the users that circulate these sources. Visualisation by
Carlo De Gaetano.
25 Their high number of likes is also inconsistent with the pattern of activity, which is mostly retweets and replies

with GIFs or funny images.

96

For the Party leadership network, the tendentious-hyperpartisan source tpo.nl and hyperpartisan
source dagelijksestandaard.nl are the largest nodes in the network and are circulated by both
suspicious and regular users (Figure 9). Smaller nodes of hyperpartisan sources, such as
fenixx.org, opiniez.com and verenoflood.nu, are positioned slightly more towards the periphery
of the network. A dense cluster of flagged users is situated in the heart of the network and has
circulated mostly tendentious and hyperpartisan hosts as well as conspiracy hosts, such as
ninefornews.nl or ellaster.nl. Regular users populate the rest of the network and have circulated
mostly tendentious and hyperpartisan hosts (e.g. tpo.nl, dagelijksestandaard.nl and opiniez.com)
and to a lesser extent, have circulated conspiracy hosts (e.g. donquijotte.wordpress.com or
stoppasfamiliedrama.blogspot.com) which are visible in the margins of the graph.
Conclusions: Suspicious activity in divisive issue spaces
As emphasized in studies of the campaigning by the Russian Internet Research Agency as well as
so-called home-grown actors, Twitter allows for easy automation, which makes the platform
susceptible to abuse by bot and troll-like users (boyd et al., 2018; DiResta et al., 2018; Howard et
al., 2018). We have identified suspicious activity during the Provincial elections of 2019, when
looking at political issue spaces as well as divisive issues. In fact, troll-like users are central across
political and issue spaces around MH17, Zwarte Piet, Utrecht, PS2019, and the Dutch party
leadership. In particular, 14 flagged users were found to be active across all political and issue
spaces, and the 29 that appear in four out of five, deserve scrutiny. Four suspect users active
during the Provincial election period were also (or still) active in all issue spaces during the EU
election period. Some of these users had already been flagged in previous research from 2017,
which means they have been operating and engaging in new and existing issues for over two
years. Overall, our study found that such flagged users tend to spread mostly hyperpartisan and
tendentious sources, followed by conspiracy websites. This points to the prospect of a
coordinated campaign to sow division, in which troll-like users include sock puppets, automated
accounts, and semi-automated user accounts that post both retweets and original content.
Divisive issue spaces are active year-round. From 18 February - the beginning of the official
campaign - to 25 March 2019, the issue spaces of Zwarte Piet and MH17 were still active, even
though Sinterklaas takes place in December and the downing of the Malaysian airliner was not in
the news, either through new developments or official memorial events. A significant number of
the most active users in each issue during this period display troll-like behaviour through their
high activity (30% in the case of Utrecht and 60% in MH17 and Zwarte Piet). Despite the
activity, most of these users’ influence is still limited, however. Only two of them appear among
the top ten most @-mentioned for each issue space.
At the same time, we identified at least three highly active new accounts that were created close
to the elections with a clear purpose of disseminating divisive content, indicating how the
platform may be employed around election time. When these troll or bot-like users are not
aggressively attacking the opposition, they function as amplification machines for web news
operations, ranging from tendentious sources such as Geenstijl and The Post Online to
hyperpartisan sources such as De Dagelijkse Standaard, Opiniez and Fenixx. Repeatedly, we have
seen how these tendentious and hyperpartisan sources are widely circulated by regular users who
97

crowd out the flagged users (in a network clustering sense). The uptake of tendentious and
hyperpartisan sources by such regular users leads to a “mainstreaming” of these hosts, in times
of elections.
In all, suspicious users tend to spread mostly tendentious and hyperpartisan hosts, followed by
conspiracy hosts, which appear in all datasets but seem to be more pervasive in tragedy spaces as
MH17 and the Utrecht tram shooting. Clickbait domains resonate marginally in all of the studied
spaces, while conspiracy host novini.nl, a host that can occasionally be classified (on a storylevel) as hyperpartisan or disinformation, were shared by users in all five data sets, resonating
more prominently in MH17 and only marginally in the political leaders set.
During the EU election period, on several occasions, fake/junk news outperformed mainstream
sources around the controversial topics, Zwarte Piet and MH17. On both issues, fake/junk news
outperformed mainstream news in two of the five weeks. During these weeks, there is not a large
increase visible in the engagement of fake/junk news sources compared to the other weeks.
Instead, the overperformance is mostly caused by a drop in the mainstream media attention for
the topics on hand, while coverage persists on the junk news sources, fuelling the debate.
According to these results, the Dutch political Twitter sphere does not appear to have a “fake
news” problem, though it is populated by some troll-like users, whose existence serves to
amplify certain voices. While we did not find a professional or large-scale trolling campaign, the
activity across issues in spreading divisive content was caused by various types of user accounts,
both bot-like (as in: automated) and troll-like (as in: repeatedly engaging with divisive issues and
targeting politicians) through what seems to be semi-automation. Divisive issues remain steadily
(even if marginally) active in fake/junk and tendentious news throughout the tested time frames,
suggesting these issues are year-round rather than seasonal (as may be expected with Zwarte
Piet) or event-based.

98

Appendix: Alternate figures

Alternate figure 2: Line graphs visualising the engagement with mainstream news (blue) and fake/junk news
sources (magenta) during the Dutch Provincial election campaign (PS) and the European Election campaign
period (EU), without the inclusion of tendentious-hyperpartisan sources. Visualisation by Federica Bardelli.

Alternate figure 3: Line graphs visualising the engagement with mainstream news (blue) and fake/junk news
sources (magenta) for the issue of MH17 and Zwarte Piet during the Provincial elections (PS), and the EU
elections (EU), without the inclusion of tendentious-hyperpartisan sources. Visualisations by Federica Bardelli.

99

THE PRESENCE OF FAKENESS
IN THE DUTCH POLITICAL INSTAGRAM:
FAKE CONTENT, SOURCES AND FOLLOWERS

Gabriele Colombo and Carlo De Gaetano

The presence of fakeness in the Dutch political Instagram: fake content, fake sources
and fake followers
Gabriele Colombo and Carlo De Gaetano
Introduction: fake followers, computational propaganda and the detection of fakeness on
Instagram
Though Facebook has been labelled the “fake news machine” (Herrman, 2016) and Twitter
studied as a matter of routine, owing to the availability of datasets, Instagram, when scrutinised,
has been found to perform well as an outlet for fake, junk or hyperpartisan news circulation,
artificially amplified engagement and other types of problematic content and users.
The relationship between Instagram and different types of “information disorders” (Jack, 2017)
has been studied in connection with the Russian effort to influence the 2016 American elections.
A study by New Knowledge (2018) found that Russian propaganda tactics played well on
Instagram. The report analyses data from a variety of platforms, in order to detect efforts by the
Internet Research Agency (IRA) to spread disinformation and divisive content. The study found
that Instagram, with “187 million engagements” of “116,000 Instagram posts across 133
accounts” (New Knowledge, 2018: 7), to be at the forefront of an IRA operation, with better
performing fake accounts and overall higher engagement than on Facebook.
The significance of Instagram, which “outperformed Facebook” (New Knowledge, 2018: 8) as a
battleground in the Russian disinformation enterprise, is linked, according to the report, to two
possible causes. First, since it is a platform designed around sharing visual materials, Instagram
may be well suited for the so-called “image-centric memetic (meme) warfare” (2018: 8), that is,
the weaponized use of image macros to stir conflict and foster division online. Second, the
report states that the considerably high engagement of content from IRA’s accounts on
Instagram may also be the result of click-farm activity, and some of the accounts in the dataset
appeared indeed to be linked to “a live engagement farm” (2018: 8).
With respect to the Dutch case, Russian influence has been studied mainly on Twitter, with the
detection of trolling activities, especially in the aftermath of tragic and divisive events both in the
Netherlands and in Belgium. For example, two journalistic studies found peaks in Russian
trolling activity following the downing of MH17 in 2014 (Kist and Wassens, 2018; van der
Noordaa and van de Ven, 2018a), while another study uncovered a (rather unsuccessful)
organized Russian effort in spreading anti-Islam content on Twitter after the 2016 Brussels
airport attacks (van der Noordaa and van de Ven, 2018b). Despite the lack of empirical research
regarding Russian influence on Instagram, one study from the NRC Handelsblad (Kist and
Wassens, 2018) suggests that a larger organized trolling activity may be found on other platforms
beyond Twitter, including Instagram.
The use of computational means to amplify misinformation and hyperpartisan content on
Instagram has not been linked exclusively to the Russian propaganda operation in the West, but
it has also been described as a domestic tactic, adopted by national campaigners as well. A
100

comparative, global study of social media manipulation in 48 countries (Bradshaw and Howard,
2018) describes different computational tactics for political influence online, including the use of
fake accounts to attack other users, automated accounts generating artificial engagement, and
human-curated accounts that employ automation to be more efficient. With respect to the
Netherlands, the study found such automated activity to be mainly linked to the boosting of
Geert Wilders’ hashtags on Twitter. While the study describes Twitter as the platform where
automation thrives the most, signs of “cyber troop activity” (Bradshaw and Howard, 2018: 13)
are also to be found in other platforms, including Instagram, among 25% of the countries
studied.
That fake following and artificial engagement flourish on Instagram may also be noted from
reported cases in the news. As a case in point, in June 2017, the Russian journalist, Vasily Sonkin,
posted an image of a vending machine, placed inside a shopping center in Moscow, that lets
users buy Instagram followers and likes. The news that for the (extremely cheap) price of 50
Russian rubels (about eur 0.70) one could buy 100 fake Instagram likes was reported by
numerous tech or news media outlet (Matsakis, 2017; Feldman, 2017; Tan, 2017).
There have also been efforts by Instagram itself to counter artificially amplified activities on the
platform. In December 2014, Instagram announced a crackdown on fake (or improperly
obtained) profiles, in the so-called “Instagram rapture” (Lorenz, 2014) that resulted in the
deletion of hundreds of thousands of accounts. And later, in December 2018, a “Christmas
crackdown” (Lorenz, 2018) resulted in the shutdown of 500 meme accounts, some of which
with millions of followers, suspected of using stolen or traded profiles. On the same note, in
April 2019, Facebook filed a lawsuit against one company based in New Zealand, accusing it of
providing “fake likes, views and followers to Instagram users” (Romero, 2019). The lawsuit is
presented as part of a larger effort by the platform to prevent “inauthentic behavior” on
Instagram. While actions have been taken to cope with artificial engagement as well as fake or
improperly obtained accounts, the platform has been apparently less active in limiting the spread
of extremely colored or hyperpartisan content, but rather has become the “Alt-Right’s new
favorite haven” (Sommer, 2018), offering refuge to extreme right-wing personalities, after their
accounts are deleted from Twitter.
The presence of a large automated engagement infrastructure on Instagram is also indicated by
the deluge of fakeness-detection tools offered by commercial services online. The detection of
inauthentic automated activity on Instagram may work by fingerprinting one account’s follower
base by nationality, and associate specific geographical locations, such as Brazil, Turkey or China,
to suspected bot activity (Maheshwari, 2018). For example, among the available tools,
Hypeauditor, a “100% AI-powered” service to expose “fake followers and engagement” on
Instagram, flags certain countries, such as Brazil, as geographical locations that may signal the
presence of fake followers (Komok, 2018).
Despite being understudied, specifically in the Dutch context, Instagram appears to be a
platform prone to the presence of various instantiations of fakeness. Fakeness (also referred to
as junk news) may refer to the presence of content that can be described as false as well as

101

merely hyperpartisan and divisive, but deliberately pushed online in order to stir conflict in a
political space, both from outside the country and from within. It can also concern various
computational tactics (such as bot work, fake likes and fake following) employed as a means to
artificially amplify that same content.
Studying the presence of fakeness in the Dutch political space on Instagram
As few studies have described Instagram as fertile ground for the circulation of content with
varying degrees of fakeness, particularly for the distribution of inflaming content in the form of
memes, but also as a well-performing infrastructure for the artificial amplification of
engagement. In this empirical research project, we devised three complementary approaches for
the assessment of fakeness in the Dutch political Instagram: fakeness with respect to the content
shared on the platform, fakeness of the most relevant information sources within the space, and
fakeness in the sense of inauthentic followers that may generate fake engagement (see figure 1).
In the first part of this study, we search for fakeness in the content shared on the platform, by
asking to what extent the most liked content in a demarcated Dutch political space on Instagram
can be defined as fake or junk (i.e., disinformation, conspiracy, clickbait or hyperpartisan)?
Second, as the estimation of fakeness also can be made through “online source criticism”
(Rogers and Niederer, this volume), we expand the work by detecting fakeness on Instagram at a
source level. Here, we study the mainstreaming of fake sources by exploring the affinity of the
follower bases of Dutch political entities with those of fake (or junk) news providers (flagged as
such by experts), and we ask, to what extent do Dutch political entities share an audience with
fake (or junk) news sources on Instagram? Thirdly, in order to account as well for the tactics of
artificial engagement that political parties and news sources may employ on Instagram to boost
their content, we search for signs of inauthentic activity in the follower bases in the Dutch
political space and we ask, to what extent are efforts of artificial boosting (by means of fake
followers) present around divisive topics on the Dutch Instagram?
In this research we employ a “digital methods” approach (Rogers, 2013), repurposing Instagramspecific features to gather data from the platform. In particular, we compile a list of hashtags and
profiles in order to demarcate the Dutch political space on Instagram. Within this space, we
collect and analyze most liked posts (i.e., posts that receive a certain amount of likes) to study
fakeness in the shared content. Furthermore, we collect followers of the accounts of Dutch
political entities, mainstream news sources, and fake news sources, to study the intersection
between their audiences, and more generally to assess the degree of fakeness in the Dutch
political space at the level of the sources. Finally, we rely on profile features (such as the 150
characters bio in one’s profile, or post captions), to evaluate the amount of fake followers of the
Dutch political space.

102

Figure 1. Diagram of the research protocol, showing the type of hashtags and accounts used for querying
Instagram, and the tools used to collect, visualize and analyze the data.

Detecting fakeness in the most liked content
The aim of the following analysis is to identify engaging content in the Dutch Instagram political
space and observe the extent to which this space is fake – in the sense of how much engagement
is generated by content that is either disinformation, conspiracy, clickbait or hyperpartisan.
To outline the Dutch political space in Instagram, we compiled a list of hashtags (see figure 2)
that are used on the social network to identify the leaders of Dutch political parties (e.g.,
#markrutte), the 2019 Dutch provincial elections (#PS2019) as well as politically charged issues
such as climate change (#klimaatverandering). We used the Instagram Scraper tool,26 offered by the
Digital Methods Initiative, to collect the 1,000 most recent posts per hashtag (data collected
between the 25th and 28th of March 2019), together with their metadata (date of the post, media
URL, caption, number of comments and number of likes). For each hashtag we selected only the
20 most liked posts, manually filtering out posts that are not relevant to the search criteria, or
identical posts that prevent more diverse results from reaching the top 20.27
In this curated list of most liked posts, we conducted a close reading by looking at post captions
and embedded media (images and videos) to understand how political party leaders and

26 The tool is available at this link: https://wiki.digitalmethods.net/Dmi/ToolInstagramScraper

27 For example, we filter out posts about the Slovak professional footballer Marek Hamšík, who plays with the

number 17 and is referred to in Instagram with the same hashtag of the Malaysia Airlines Flight 17 (#MH17). We
also do not include in the dataset the identical posts of condolence messages for the Utrecht attack posted by Dutch
national football team players with the hashtag #Utrecht.

103

politically charged topics are discussed within the limits of the Instagram Dutch political space,
and specifically to flag the presence of fake content.
As a result of this evaluation (see figure 3), we found that out of the 400 most liked posts within
our dataset there are (only) 45 posts that can be flagged as fake or junk, 4 satirical posts, and 351
posts that do not appear to be junk. Looking at the engagement generated by these posts, fake
content was liked 79,466 times, satirical content 37,532 times, and non-fake content 838,794
times.
In figure 4, the 400 most liked posts are divided in hashtag-dedicated columns, in which they are
also ranked from the most liked post in the first row to the least liked one in the last. Fake
content is flagged using three different colors: light blue for hyperpartisan content, magenta for
conspiracy, and blue for click-bait. Satirical posts are color-coded in dark blue. Finally, columns
are ordered from left to right according to the amount of fake content, calculated on the total
number of likes for each hashtag.
The analysis shows that the #zwartepiet, #geertwilders and #tunahankuzu hashtags represent
the most divisive political spaces, with respectively 56.1%, 42.8% and 42.7% of their total
amount of likes directed to fake or junk content. Moreover, we find that the majority of the
posts flagged as such can be considered hyperpartisan, mostly supporting and/or opposing
particular right-wing ideology or figures, while only one post can be considered as clickbait, and
one conspiracy. Generally, we did not find any trace of disinformation linked the content that
receive most likes. The findings suggest that certain issues or political leaders, such as the Zwarte
Piet debate and the leader of Denk political party, Tunahan Kuzu, draw more divisive content
than others. Of the 20 most liked posts, however, we found no strong presence of fakeness or
junk.

Hashtags related to

Hashtags related to

Dutch political party leaders

politically charged discussions

#markrutte, #rutte, #geertwilders, #wilders,
#thierrybaudet, #baudet, #jetten,
#tunahankuzu, #jesseklaver, #lodewijkasscher,
#alexanderpechtold, #gertjansegers,
#sybrandbuma, #mariannethieme

#PS2019, #klimaatverandering, #immigranten,
#utrecht, #zwartepiet, #MH17,
#24oktoberplein

Figure 2. Lists of hashtags pertaining to political leaders and politically charged discussions used to demarcate the
Dutch political space on Instagram around the 2019 provincial elections.

104

Figure 3. Proportions of most liked content shared around the 2019 Dutch provincial elections, categorised as
fake, satire, and not fake. Data source: Instagram Scraper; data collection: 25-28 March 2019; pie-charts.

In general, we found a relative scarcity of fake or junk content in this high-engagement political
space. In the top results for the Dutch provincial elections, #PS2019, we found only positive
content, either celebrating preliminary poll results or encouraging people to exercise their right to
vote. The hashtags, #24oktoberplein and #utrecht, returned mainly condolence posts and the
news that the attacker was spotted and arrested. Almost all of the content we considered as fake
or junk is hyperpartisan, mainly about particular right-wing ideology or figures. We found no
presence of disinformation in the most liked results within the demarcated political space.
In order to ascertain the presence of fakeness on Instagram surrounding the 2019 European
Parliamentary elections in the Netherlands, we conducted a second hashtag analysis concerning
content posted in the months before the election day (23 May). With the goal of demarcating the
Dutch political space around the 2019 European elections, we compiled a new list of hashtags
(see figure 6) used to identify Dutch political parties (e.g. #fvd) and their leaders (e.g.
#thierrybaudet), the European elections (#EUverkiezingen2019, #EUverkiezingen), and various
politically charged issues such as immigration (#immigratie, #migratie, #immigranten) and
climate change (#klimaat, #klimaatverandering).

105

Figure 4. 20 most liked posts per hashtag shared around the 2019 Dutch provincial elections, sorted from right
(most fake) to left (least fake). Data source: Instagram Scraper; data collection: 25-28 March; Image wall.
With the Instagram Scraper tool, we collected the 1,000 most recent posts per hashtag (data
collected on the 22nd of May) and their metadata. For each hashtag we only retained posts
shared after the 28th of March, in order to focus the detection of fakeness on the period prior to
the European election, but after that of the Dutch provincial elections. For each hashtag we
selected the 20 most liked posts, excluding those included in the dataset but not relevant to the
search criteria.28 Subsequently, in order to ascertain the amount of fakeness in the dataset, we
looked at embedded media and textual captions and flagged each post as fake or junk (making
the distinction among disinformation, conspiracy, clickbait and hyperpartisan content).
The analysis (see figure 7) confirmed the relative lack of fake or junk content in the Dutch
political space, also around the 2019 European parliamentary elections: out of 452 most liked
posts, we found only 41 that can be considered junk (specifically hyperpartisan), counting for less
than 10% of the total amount of posts. Moreover, hyperpartisan posts score low even in terms
of engagement, generating only the 4.66% of likes out of the total amount.

28 The query for some of the less popular hashtags returned less than 20 posts in the specified date range.

106

Figure 5. Example the posts flagged as hyperpartisan or satire. Data source: Instagram Scraper;
data collection: 25-28 March; Image wall.
In figure 8, most liked posts are organized in hashtag-dedicated columns. Columns are grouped
by type of hashtag and sorted from right to left according to the amount of likes generated by
hyperpartisan content. The analysis shows that the hashtags related to political parties attracting
more divisive content are #pvda and #christenunie, with respectively 27.6% and 25.1% of likes
directed to hyperpartisan content. Compared to the dataset around the provincial elections,
Geert Wilders (#geertwilders, #wilders) remains the political leader receiving the highest
percentage of likes directed to hyperpartisan content (28.4%), followed by Jesse Klaver
(#jesseklever) with 13.4%, who instead scored low in terms of fake content in the previous
analysis. We did not find traces of hyperpartisan content in the most liked posts around other
political leaders. As it was the case with the hashtags used to refer to the Dutch provincial
elections, #EUverkiezingen2019 and #EUverkiezingen present mainly invitations to exercise the
right to vote. Among the issues under study, #zwartepiet remain the most divisive one, with
22.9% of likes directed to hyperpartisan content.
In general, we did not find evident signs of fakeness in the most-liked content around the 2019
European elections, except for few hyperpartisan posts. The finding is aligned with that of the
hashtag analysis conducted around the 2019 Dutch provincial elections.

107

Hashtags related to

Hashtags related to

Hashtags related to

Dutch political party leaders

Dutch political parties

politically charged discussions

#markrutte, #rutte,
#geertwilders, #wilders,
#thierrybaudet, #baudet,
#jesseklaver, #jetten,
#mariannethieme,
#tunahankuzu

#cdavandaag, #pvv,
#socialistischepartij, #pvda,
#christenunie,
#partijvoordedieren,
#50pluspartij, #groenlinks,
#fvd, #stempiraat,
#voltnederland, #d66,
#degroenen

#duurzaamheid, #klimaat,
#klimaatverandering,
#immigratie, #migratie,
#immigranten, #mh17,
#zwartepiet

Figure 6. Lists of hashtags pertaining to political leaders and politically charged discussions used to demarcate the
Dutch political space on Instagram during the months before the 2019 European elections.

Figure 7. Proportions of most liked content shared around the 2019 European elections, categorised as fake/junk
and not fake. Data source: Instagram Scraper; data collection: 22 May 2019; pie charts.

108

Figure 8. 20 most liked posts per hashtag shared around the 2019 European elections, sorted from right (most
fake) to left (least fake) and grouped by type (elections, issues, political leaders, and parties). Posts flagged as
hyperpartisan are colored in red. Data source: Instagram Scraper; data collection: 22 May 2019; Image wall.
Follower ecologies and the fakeness of sources
In order to detect the relevance of fake news sources within the Dutch political space on
Instagram, and to assess whether and how much fake news sources are becoming mainstream,
we studied the overlap between followers of Dutch political entities, mainstream news sites and
Dutch-language “junk news” sites.29 Specifically, we asked, to what extent followers of false news
providers are shared with those of Dutch political entities?
First, we demarcated the Dutch political space on Instagram, by compiling three lists of profiles:
a list of Dutch political parties and their leaders, a list of Dutch mainstream media outlets, and
the profiles of Dutch information sources flagged as fake in the expert list. We then used the
API Instagram Follower Collector by Phantombuster30 to collect the follower list of each Instagram
account, and then, by creating a co-follower network, we looked at the amount of shared
followers between the political entities and the suspicious Dutch information sources from the
expert list.
29 We use the list of sites flagged by the hoax-wijzer (www.hoax-wijzer.be), which was edited and enhanced by

University of Amsterdam researchers, and is dubbed the ‘expert list’.

30 Phantombuster is an API store that “provides ready-made cloud APIs to collect data from various social networks

and improve marketing strategies” (phantombuster.com).

109

In mapping the follower network of the Dutch political space, we found three distinct follower
ecologies (see figure 9). First, an ecosystem of followers of mostly established mainstream news
organizations, such as the Dutch public broadcasting station, NOS. The follower bases of these
news organizations are the largest in the network, which suggests that the Dutch mainstream
news providers are still more relevant that those flagged as junk, at least in terms of follower
count. Few sites from the expert list are close to (or part of) the cluster of mainstream news
organizations, due to a relative high number of shared followers. Shared followers among
mainstream news organizations and suspicious news sites may indeed suggest a special affinity
among them, or rather be the signal of the mainstreaming of fake news providers.
A second ecosystem is made up of political parties and their youth organisations. The
distribution of parties is laid out from left-wing parties to right-wing parties, whilst still being
tightly clustered together. This may suggest that most followers either follow multiple parties on
the same side of the political spectrum or follow all political parties regardless of political leaning.
What can also be observed is the relative distance of the cluster of political parties to that of
news organizations, suggesting that followers of political entities are mostly not shared with
those of news organisations.
A third cluster is made up of far right-wing political entities, which are far from other political
entities, closer to few hyperpartisan or clickbait sites and to few, less established, mainstream
news providers. Within this cluster, the account of PVV leader Geert Wilders is surrounded by
GeenStijl, a tendentious, right leaning mainstream ‘shock blog’ and PowNed, the public
broadcasting station that is an offshoot of GeenStijl. The official profile of FvD (Forum for
Democracy) and the youth organization of the same party are even more distant and isolated
from other parties: they are surrounded by individual political commentators and share a high
number of followers with the hyperpartisan news site, De Dagelijkse Standaard. This topology may
suggest that although these parties and personalities share some followers with those from other
sides of the political spectrum, they are mostly on their own and produce content consumed by a
unique audience.

Fake followers and artificial engagement
In order to profile the follower base of the previously demarcated Dutch political space, we feed
each account31 (of political entities, but also of mainstream media, and of those from the expert
list) in the HypeAuditor tool to check the authenticity of the accounts and look for signs of
artificial boosting and fake followers. With HypeAuditor one can profile an Instagram account to
determine the authenticity of its follower base. To assess the extent to which Dutch political
accounts are employing artificial engagement tactics, we use reports from HypeAuditor, regarding
the percentage of real followers, and their geographical origin.32 The percentage of fake followers
31 HypeAuditor analyzes only accounts with more than 1,000 followers. For this reason, we limited the detection of

fakeness to the accounts with more than 1,000 followers.

32 According to HypeAuditor, the geographical origin of one follower base is detected by analysing profiles

biographies and place names in post captions (twitter.com/hypeauditor/status/1077143110432538624).

110

returned by the tool is then used to rank each account from less fake to more fake (see figure
10). Furthermore, we zoomed in on those accounts with a higher percentage of fake followers,
to observe their geographical provenance (paying particular attention to suspicious countries), as
well as the segmentation of the follower base provided by HypeAuditor, which breaks down
followers in “real people”, “influencers”, “mass followers” and “suspicious accounts” (see figure
11).

Figure 9. Follower ecologies in the Dutch political space, visualized as a co-follower network and manually
annotated. In the network, accounts with higher amounts of shared followers (pink) are placed closer to each other.
Data source: Phantombuster; data collection: 25-28 March; network graph.

Generally, we found that the majority of profiles do not have a suspicious follower base, with
most accounts scoring higher than 70% in the real follower metrics provided by the tool. There
are some accounts, however, that are suspect of having a fake follower base. For instance, the
media entity Powned has 32.6% of suspicious followers. The clickbait site Prankster also scores
111

relatively high in terms of fake following. Within the group of political entities, the personal
account of Mark Rutte and the account of Geert Wilders have the highest number of fake
followers. Strikingly, the “work” account of the prime minister, Mark Rutte, has a lower
percentage of fake followers than that of his personal account. On the other hand, the account
for the political party, Christenunie, has hardly any fake followers, just as the SGP (Reformed
Political Party) and that of the minister Gert-Jan Segers.
When we look closer to the nationality of the follower bases, we found no suspicious results,
with most of the accounts followed by users based in The Netherlands. For both of Mark
Rutte’s accounts, the followers are mostly based in the Netherlands. On the contrary, Geert
Wilders account has 36% of his followers from Brazil. This raises some questions regarding the
legitimacy of Geert Wilders’ follower base, for Brazil is often mentioned as one location that can
signal the presence of fake followers (Maheshwari, 2018).
In all the follower analysis does not show an organised effort of artificial boosting within the Dutch
political Instagram sphere, and it indicates, with the exception of Geert Wilders, a rather authentic
follower base.

Figure 10. Degree of account fakeness according to report by the HypeAuditor tool. Accounts on the further right
have more suspected ‘fake followers’ than accounts on the right side of the graphs. Data source: HypeAuditor; data
collection: 25-28 March 2019; beeswarm plot.

112

Figure 11. Visualisation of the follower base of Mark Rutte’s personal and work accounts and Geert Wilders
account, based on results from the HypeAuditor tool. Each follower base is segmented based on “audience type”
and geographical provenance. Popular suspicious countries, that may suggest an inauthentic follower base, are colored
in red. Data source: HypeAuditor; data collection: 25-28 March 2019; pie charts.

Conclusions: Findings and limitations
The goal of the present research is to detect signs of fakeness in the Dutch political Instagram
sphere. More generally, it can be considered an attempt at applying to the Dutch context the
argument in the New Knowledge report (2018) that Instagram performs well in terms of fake
content circulation and artificial amplification strategies. It also takes up the invitation from the
NRC Handelsblad study to inquire into other platforms than Facebook and Twitter for
disinformation campaigning and computational propaganda.
The presence of fakeness (or lack of thereof) has been studied on three levels: at the story level
(by looking at the circulation of fake content in high-engagement political spaces on Instagram);
at the source level (by looking at the intersection between the follower bases of Dutch political
entities and that of news sources flagged as junk); and also through the detection of artificial
engagement tactics, specifically fake followers, among the profiles of Dutch political entities as
well as Dutch information sources.
In general, we found a rather healthy political space, with only few evident signs of fakeness.
Most liked content in the Dutch political space proved to be fake to a very small degree,
113

although we found a small amount of hyperpartisan and divisive content centered around rightwing figures and issues around the 2019 Dutch provincial and European elections. With respect
to the alignment of the audience of Dutch political parties with that of (mainstream or fake)
news providers, we found mainstream news organizations to be still more relevant in this
political space, somehow confirming the argument that in the Netherlands “the vast majority of
news consumption remains of the mainstream sources” (Rogers and Niederer 2019, this
volume). Furthermore, the analysis of the follower base of Dutch political entities (and that of
news sites, both mainstream and fake) revealed an apparent authentic audience with almost no
signs of artificial engagement.
Within a relatively healthy political spectrum, it is at the extremes that fakeness surfaces. With
the current research we have pointed out a special affinity between far right-wing political
entities and some information sources that may be defined as fake and junk (or at least
hyperpartisan). Furthermore, the few indications of artificial engagement we have found are
located at the far right of the political spectrum, with Geert Wilders account being the most
suspected of inauthentic activity.
In the co-follower analysis, we found right-wing extremes to have a unique follower base, not
shared with other parties or mainstream news sites. Right-wing political entities are also relatively
closer (in terms of shared followers) to suspicious sources (a few of them flagged by the expert
list). Above all, Geert Wilders account is the closest (according to shared followers) to
hyperpartisan news sources. Relatedly, Geert Wilders’ account is the only one of those under
study that may reveal signs of artificial engagement, as suggested by a geographically dubious
follower base. This finding resonates with the 2015 scandal about a suspicious increase of the
follower count of Geert Wilders Twitter profile. In addition, the already mentioned comparative
study of social media manipulation strategies by the Oxford Internet Institute (Bradshaw and
Howard, 2018) also refers to Geert Wilders as making use of various artificial boosting strategies
in The Netherlands, reporting on an analysis by a social media analytics firm that in February
2016 found 26 fake accounts amplifying the #geertwilders hashtag on Twitter.
The determination of the relative absence of fake content, dubious sources and fake followers in
the scope of the current research has a series of methodological limitations. First, in the search
for fakeness in the shared contents, we collected data based on a limited list of hashtags related
to Dutch politicians and controversial topics. One could repeat the analysis to include other
politically charged issues. Furthermore, we have considered only the top 20 most liked posts per
hashtag, whereas we could have also counted the number of comments per posts to analyze
most-engaged-with content. Moreover, we could have included in the analysis a larger set of
posts that do not necessarily make it to the top (because they receive fewer likes, or have less
comments), in order to evaluate the presence of fakeness in less-engaged-with spaces. In
addition, given that for data collection we made use of the DMI Instagram Scraper, which “scrapes
Instagram to retrieve posts” (Digital Methods Initiative, 2019), this research is dependent on the
limits of such scraping, including Instagram’s rate limits which are not documented and
unknown security challenges (Instaloader, 2019). It is also not a platform that invites research
through scraping. As others have pointed out, social media platforms are designed to increase a
platform commercial value, rather than to meet researchers’ needs (Borra and Rieder, 2014). To
114

overcome the limitations, one could use additional tools for data collection and compile a richer
data set.
Secondly, we established the fakeness in the Dutch political follower base using the metrics
provided by one single tool (HypeAuditor). We could have compared the results with those by
other similar services (and auditing the auditors, so to speak). Moreover, we searched for signs of
inflated engagement in the Dutch political space only by looking at followers’ demographics,
while we could have paid attention to other signals such as patterns of repetition in posts
comments. For example, to account for other tactics of artificial engagement on Instagram, one
could perform a co-hashtag analysis33 in a demarcated issue space, and detect signs of (semiautomatic) boosting, such as the use of long list of popular unrelated hashtags, deliberately added
in the post captions to increase content visibility.34 Moreover, one could trace back the users
involved in this activity and profile them in order to evaluate their authenticity.

33 In addition to the most recent lists of posts, the Instagram Scraper tool returns a network of hashtag co-occurrence,

that is a file that contains the hashtags used at least once together with the hashtag under study. For each pair of
hashtags, the tool returns a numeric value representing the total number of posts in which the two hashtags appear
together in the data set. A similar approach is largely used for empirical research on Twitter: with co-hashtag analysis
one can get a sense of the relationship between sub-topics in a conversation (Borra and Rieder, 2014); or find
additional and/or more “significant hashtags” (Rogers 2017) to be queried to expand a corpus of data; or spot
hashtags practices aimed at enhancing the visibility of particular content (Wang et al. 2016), or overturning its
original meaning through hashtag hijacking practices (Berg, 2017).
34 Unlike Twitter, which has a character limit of 280 characters, Instagram character limit is 2,200 characters, and
users can include up to 30 hashtags in the caption and comment sections of the post. This results in certain users
adding blocks of more or less related hashtags to the posts to enhance their visibility. Even if Instagram is applying
countermeasures to block the use of certain hashtags (Drewe, 2016), there are several websites that provide lists of
safe and popular hashtags that users can copy paste directly in their posts (for example tagblender.net).

115

DUTCH JUNK NEWS ON REDDIT AND 4CHAN/POL/

Sal Hagen and Emilija Jokubauskaite

Dutch Junk News on Reddit and 4chan/pol/
Sal Hagen and Emilija Jokubauskaite35
Introduction: The understudied, deep vernacular Web
Recent debates on online fake news and disinformation have largely been discussed with respect
to the social media behemoths in the context of a “platformized” internet ecosystem (Helmond,
2018), with Facebook, Twitter, Instagram and YouTube in the spotlight. It is not without reason;
given their gigantic user bases, open publishing and micro-targeting, they are vulnerable to
disinformation campaigns and dubious information, not so unlike the Web itself. Regardless,
they do not exist in a vacuum. On the fringes of the Web, yet difficult to characterise as
marginal, are pseudonymous or anonymous platforms like Reddit and 4chan. Instead of publicfacing “e-celebs” or otherwise identifiable accounts, these spaces are characterised by “masked”
users with distinctive subcultural styles, vernaculars and iconographies. The pseudonymous and
anonymous users on Reddit and 4chan do not only congregate around shared interests or
common goals, but also – and in some cases predominantly – around a deep understanding of
shared subcultural knowledge and norms. The unconventional and sometimes downright
esoteric cultural productions some of these groups create feed into community members’ selfimagination as “underground”, “countercultural”, or “internet native”. 4chan and (parts of)
Reddit can be associated with the term “deep vernacular Web” (Tuters and De Zeeuw, 2019),
referring to online discussion forums that lack stable user identities and whose masked
participants frequently transgress the boundaries of “mainstream” conventions, often through an
entangled mix of sincere ideology and ironic play.
While Facebook, Twitter, Instagram and YouTube have already been studied in relation to issues
of “fake news”, the abovementioned “virality-oriented subcultures” of the deep vernacular Web
are also said to play a “crucial role in the system” of the circulation of various types of “junk
news” (Venturini, 2019). 4chan and certain parts of Reddit have indeed been characterised as
hotbeds for disinformation (Shiebel, 2017; Collins and Russell, 2018; Lagorio-Shafkin, 2018),
trolling campaigns (Phillips, 2015), and conspiracy theories (Marwick and Lewis, 2017; Tuters et
al., 2018). Despite their relatively marginal number compared to more mainstream platforms,
users of 4chan and areas of Reddit are considered particularly skilled in “setting the agenda” of
broader news media (Phillips, 2018). In a 2017 report, Marwick and Lewis highlight how an
underground current of Internet subcultures associated with 4chan and Reddit “take advantage
of the current media ecosystem to manipulate news frames, set agendas, and propagate ideas”
(Phillips, 2018: 1). In a more recent report, Phillips builds on this research by exploring how and
why the false narratives of these online antagonists were amplified by major U.S. news outlets
(2018). As she identifies, journalists were keen on reporting the narratives with false information
or dark undertones partly because of a fascination with their bizarre cultural phenomena or
simply due to a lack of time required to decipher their problematic code language. The reporting,
she argues, “amplifies” their overall presence. By 2019, there are now well-known byproducts of
this cycle of the normalisation of false content emerging from fringe online spaces. To provide
but one example, the “Pizzagate” conspiracy theory, originating on 4chan, presumed the
Clintons were maintaining a child sex trafficking ring, which led to media coverage and an actual
shooting in a U.S. pizza parlour (Tokmetzis, 2018; Tuters et al., 2018). A related conspiracy
theory, “QAnon”, gradually spread from 4chan to Reddit and mainstream news sources (Hagen

35 The research team includes Lucie Chateau, Gabriele Colombo, Ognjan Denkovski, Carmen Ferri and Holly

Foxton.

116

et al., 2019), even sprouting international support groups including “QAnon Nederland”
(QAnon Netherlands).
The influence of fringe internet subcultures on the news ecosystem has mostly been scrutinised
in relation to English-language spheres and U.S. politics. As such, it remains unclear to what
extent the propagation of problematic content from the “deep vernacular Web” affects other
news ecosystems such as the Dutch. Such activity already has caught the attention of Dutch
media outlets. For example, the QAnon conspiracy was covered by major outlets like RTL
Nieuws (2018) and Algemeen Dagblad (Van Huet, 2018), while De Correspondent untangled the
related Pizzagate conspiracy in some depth (Tokmetzis, 2018). De Volkskrant discussed Dutch
users active in the far-right “politically incorrect” subforum of 4chan, /pol/, by observing an
increasing prevalence of anti-Semitic conspiracies (Kranenberg and Bahara, 2018). In a broader
sense, conspiratorial rhetoric native to the deep vernacular Web seems to be normalising in the
Dutch political and media discourse at large. For instance, the concept of “cultural Marxism” has
increasingly appeared in Dutch news media (Van den Bos, 2018). It concerns a theory assuming
a Marxist and/or Jewish network pulling the strings of European institutions – a narrative
particularly popular on 4chan/pol/. Dutch politicians have subsequently flirted with such
sweeping theories. For instance, the party Forum voor Democratie tweeted that Mark Rutte was
a puppet of the Jewish philanthropist George Soros,36 while the party’s leader Thierry Baudet
supported the most conspiratorial aspects of the “cultuurmarxisme” debate, tweeting that the
European Union is “a cultural Marxist project aiming to destroy European civilisation”.37 NOS,
the public broadcaster, subsequently published an article framing George Soros as an “influential
meddler with tentacles deep in world politics”,38 which was later withdrawn after heavy criticism
pointing out the framing’s commonalities with anti-Semitic rhetoric (Peek, 2018). While direct
ties between such mainstream attention and fringe internet platform rhetoric are not to be
drawn, each incident in its own right could be situated in the aforementioned dynamics of
amplification in a Dutch context.
The falsehoods cooked up on the deep vernacular Web are hard to grasp through the concepts
of “disinformation” or “fake news”, since their “fakeness” is broader than deliberately
coordinated campaigns or clearly false information. Rather, they speak to what Muirhead and
Rosenblum (2019) call a “new conspiracism”, whereby sweeping accusations are made
independent of evidence or coherent explanations, and complex phenomena are “explained”
through “conspiracy without theory”. Such conspiracism is said to be dangerous since it
delegitimizes the knowledge-making institutions at the foundations of democratic societies
(Muirhead and Rosenblum, 2019). Importantly, this conspiracism can be fueled by or work
alongside a mix of foreign interferers, sincere believers and hyperpartisan actors.
To understand these broad range of actors and interests that stimulate the emergence of
problematic information, the concept of “junk news” (or “pulpnieuws”) is more apt. Junk news
shifts the focus from clear and coordinated falsehoods towards a broader notion of news crafted
to be engaged with and to circulate, which, in turn, stimulates polarising or “simple” information
that “saturates public debate” (Venturini, 2019). Junk news thereby forms an umbrella term for
36 The original Dutch tweet by Forum voor Democratie notes: “@MinPres [i.e. Mark Rutte] draait er niet eens meer

omheen: De belangen van NDO’s (lees: Soros) gaan boven het beleid van de democratisch gekozen regering van
#Hongarije. Hoogste tijd dat deze loopjongen van het grootkapitaal nu van het toneel verdwijnt. Reken af met
#Rutte op 20 maart! Stem #FVD” (@fvdemocratie, 14 Sep. 2018).
37 The original Dutch tweet by Baudet noted: “Omdat de Europese Unie een cultuurmarxistisch project is dat tot
doel heeft de vernietiging van de Europese beschaving” (@thierrybaudet, 19 Aug. 2017).
38 In their original article, NOS used the title “George Soros: invloedrijke bemoeial met tentakels ver in de
wereldpolitiek” and noted: “De jood Soros steunt organisaties die regeringen openlijk bekritiseren [...]. Dat moet
stoppen, zeggen tegenstanders” (Peek 2018).

117

conspiracies, hyperpartisan slander, “ironic” falsehoods, low-effort clickbait articles, as well as
deliberate disinformation. The circulation of these types of junk news has a plethora of reasons,
but, as noted, “tightly-knit communities” (Zannettou 2017) and “virality-oriented” subcultures
creating and engaging with this highly “shareable” content are said to be a crucial factor in their
effectiveness (Venturini 2019).
Much has been said about the grassroots production of false narratives within spaces like Reddit
and 4chan (Marwick and Lewis, 2017; Phillips, 2018; Tuters et al., 2018; Benkler et al., 2018).
However, a more elementary question is usually left untouched: what kinds of news sources do
these actors rely on themselves? Zannettou et al. (2017) found that “‘fringe’ communities often
succeed in spreading alternative news to mainstream social networks and the greater Web” (1),
employing a statistical model (Hawkes process), indicating that fairly marginal spaces like
Reddit’s pro-Trump subforum r/The_Donald and 4chan’s /pol/ board are often first to post a
URL to alternative news, that only later catches attention on Twitter. They furthermore traced
which alternative sources where shared on Reddit, 4chan, and Twitter, showing that alternative
news was shared more often on 4chan/pol/ and select subreddits than on Twitter, noting the
popular use of breitbart.com, rt.com, infowars.com, and sputniknews.com across the three
platforms.
What about junk news in a Dutch context? Is there a Dutch alternative “junk news” network
within the Deep vernacular web, or do these platforms mostly rely on mainstream sources? If
found, how vast is the presence of Dutch junk news on these spaces? In identifying linked-to
websites, can signs of coordinated disinformation campaigns be discerned? Or are the types of
junk news shared mostly hyperpartisan, clickbait, or some other “junk” category? These
questions are of interest when applied to fringe and “extreme” spaces like 4chan/pol/ but can
also aid in positioning more widely used yet still “alternative” spaces, like the largest Dutch
subreddit, r/thenetherlands. The research reported here thereby begins with the question, where
does Dutch junk news appear (if at all) on Reddit and 4chan/pol/? Subsequently, it asks, what
kinds of junk news resonate? It concludes with a brief section on YouTube as a possible
alternative news network by following the links to Google’s video platform.

Demarcating the Dutch spaces and Dutch junk news in the deep vernacular Web
Case studies: Reddit and 4chan
For Dutch cases of virality-oriented subcultures, we focus on Reddit and 4chan/pol/. Although
less known than the likes of Twitter, Facebook, Instagram and YouTube, Reddit is one of the
largest discussion sites globally, with Alexa metrics currently showing 234 million unique visitors
per month. The platform is divided into different subreddits dedicated to the discussion of
specific topics, such as r/tennis or r/politics. Posts on these subreddits can be “upvoted” or
“downvoted” by users. The higher the post’s score (upvotes minus downvotes), the higher it is
placed in a ranked list of content and the more visibility it gains. In the comment section
underneath every post, “redditors” discuss, debate, or simply joke around. Reddit’s Dutch user
base seems to be growing (as is shown below), with the largest Dutch subreddit r/thenetherlands
amounting to 236,000 “subscribers” at the time of writing. Its growing popularity makes it an
increasingly important object of study in a Dutch context. This is heightened by the fact that
Reddit has been identified as a target of multiple Russian disinformation campaigns, with “at
least a hundred” IRA accounts influencing the 2016 U.S. elections and campaigns continuing
into late 2018 (Collins and Russel 2018; Lagorio-Shafkin 2018). Exploring whether such

118

campaigns have also transpired within Dutch spheres of Reddit is thus part of the objective of
this research.

Figure 1. The frontpage of Reddit (retrieved 11-Jun-2019).
The second case study is 4chan, an infamous imageboard where users post anonymously within
one of its subforums (called boards) dedicated to different topics like videogames or fitness.
4chan is ephemeral, meaning posts are deleted from the site after a few days or even hours. It is a
visual environment conducive to the production of viral content and generation of junk news
(Venturini 2019). The space’s creativity extends beyond the generation of alternative theories, as
4chan is also infamous as the “birthplace of internet memes”, as well as a hotbed for nebulous
political movements. The latter include “Anonymous”, the loose “masked” collective of geeks
and hackers infamous for trolling and DDOSing the likes of the Church of Scientology and
MasterCard (Coleman 2014), as well as more recently the “alt-right”, once characterised as an
“amalgam of conspiracy theorists, techno-libertarians, white nationalists, Men’s Rights advocates,
trolls, anti-feminists, anti-immigration activists, and bored young people” (Marwick and Lewis
2017, 3) but now arguably pertaining to the extreme side of those far-right actors. For this
research, we chose to focus squarely on 4chan’s politics board, /pol/. This is the most relevant
board in relation to the research questions, for it is currently among the most active boards on
the website39 and is a fertile ground for conspiracy theories (Tuters et al., 2018) and alternative
news sources (Zannettou, 2017). 4chan/pol/ is a far-right space, identified as a recruitment zone
for neo-Nazis (Wendling, 2018) and connected to various acts of extreme violence (Hankes and
Amend, 2018). This partisanship naturally affects the types of news shared on this platform. For
balance, other partisan areas of the deep vernacular Web were also considered (e.g.,
8chan/leftypol/) but were ultimately found too insignificant in terms of Dutch activity.

39 At the time of writing, the website 4stats.io, which tracks activity on each 4chan board, lists /pol/ and /v/ (video

games) as the most active boards, with almost 50 posts per minute and 120 thousand posts per day (taking the last 4
weeks as a benchmark). These numbers are supported by metrics from our own tools (Peeters and Hagen 2018).

119

Figure 2. The index page of 4chan/pol/ (retrieved 11-Jun-2019).
Tools and timeframe
In contrast to mainstream platforms like Facebook and Twitter, data from Reddit and 4chan is
rather accessible. For most of the data collection, we used 4CAT (Peeters and Hagen, 2018), a
tool developed by the Digital Methods Initiative that captures data from a variety of sources,
including 4chan/pol/ since November 2013. For Reddit, 4CAT makes uses of the Pushshift
API, which allows access to an archive of nearly all Reddit posts and comments (Baumgartner,
2018).
We chose a timeframe from 1 December 2015 up to 1 June 2019, spanning 4 1/2 years in total.
Whereas most other studies in this volume present timeframes based on specific events, this
long-term timeframe is more suitable here for multiple reasons. Firstly, as we will show, the
activity in relation to junk news posting on these platforms was shown to be fairly marginal in
comparison to more mainstream social media websites. A larger timeframe thereby aids to arrive
at patterns in this relatively small stream of data. Secondly, to the best of our knowledge, it is the
first time the presence of questionable Dutch news is researched on these platforms, so it makes
sense to start with a high-level perspective on the object of study instead of limiting it to a
particular case. Moreover, this timeframe includes a variety of major political events in the
Netherlands, including the general elections in 2017 and more recently the provincial and
European parliamentary elections in 2019.
Analyses: Haystack to needle and needle to haystack
As the research focuses on the presence of junk news linked to on Reddit and 4chan/pol/, it
takes URLs as the primary research objects. To provide an overview of the types of news linked
to, we decided to focus on domain names (sources) instead of links to individual articles (stories).
To identify and categorise domains, we used two related approaches, referred to metaphorically
as ‘haystack to needle’ and ‘needle to haystack’. The haystack to needle approach denotes a
macro to micro inquiry where all domains posted were categorised in order to subsequently
identify the presence of Dutch junk news within this larger pool of data. The needle to haystack
120

does the reverse and starts from an expert list of Dutch junk news domains40 and subsequently
enquires into what is shared most often, where these sources appear, and, for Reddit, what kinds
of users post them. The next two subsections describe these approaches in more depth.
Haystack to Needle
The haystack to needle approach moves from a high-level overview to the categorisation of
particular linked-to domains, specifically by parsing (1) news from non-news, (2) Dutch news
from non-Dutch news, and (3) types of junk news (mainstream/junk and types of junk). To do
so, a Dutch sphere first had to be defined for Reddit and 4chan/pol/ from which an initial list of
domains could be extracted. For Reddit, the full dataset of opening posts was filtered for a list of
Dutch subreddits (thus excluding comments, for URLs appear less here and are less visible). The
relevant subreddits were compiled from a set of “related communities” posted by administrators
of r/thenetherlands41 and supplemented through querying Dutch issues on Reddit. This resulted
in a final collection of 182 subreddits (see Appendix I). On 4chan/pol/, all posts show a flag
icon indicating the location of the IP address of the poster. To identify a Dutch sphere on /pol/,
all posts with a country flag of the Netherlands were extracted. It is important to note that this
only results in a partial sample of Dutch posters, since users can also choose to display a custom
flag (like “Hippie”) instead of one based on geolocation, or they can spoof their IP addresses. In
all, the dataset collected consists of over 2 million posts with Dutch country flags, forming a
large enough sample to gauge the presence of Dutch junk news using the haystack to needle
approach.
Having demarcated Dutch spheres on Reddit and 4chan/pol/, domains from URLs posted were
extracted from all posts. For Reddit, this resulted in 3,489 unique domains. To qualitatively
analyse a sample of this large dataset, the domains that were posted five times or more were
retained. This resulted in a list of 372 domains. Similarly, domain names from the Dutch
4chan/pol/ posts were extracted using 4CAT, yielding 8,048 domains.42 To arrive at a
comparative sample, we kept the domains that were posted twenty times or more, resulting in
352 unique domains.
The two lists were then categorised according to whether (1) the domains were news websites,
(2) they concerned either news websites in the Dutch language or Dutch affairs, and (3) the
category of news websites they would fall in. “News websites” here refers to a fairly broad
selection of websites focusing on the production of news and opinionated columns which
contain a section dedicated to timely updates. They include blogs on current affairs, special
interest news, and websites of TV news programmes. Thereafter, the news sources were
categorised as follows:
● Mainstream: Reporting by “established” general news outlets with a predominantly
neutral tone of voice.
● Other mainstream: All other mainstream news websites concerning special interests,
such as business or sports news.
● Disinformation: Sources deliberately publishing false information, often with harmful
intention, and part of a network or campaign.
40 The expert list is comprised of an original list by De Hoax-Wijzer, edited to remove inactive sources, with

additional sites added through qualitative analysis by University of Amsterdam researchers.

41 See: https://www.reddit.com/r/theNetherlands/wiki/related. Accessed 25 March 2019

42 This is a higher number than for Reddit because for 4chan, not only the first posts in a thread were kept, but also

the replies since this matches 4chan’s infrastructure of more “horizontal” conversational threads.

121

● Hyperpartisan: Extremely coloured and “openly ideological” reporting and
editorialising from a far end of the political spectrum (Herrman 2016).
● Clickbait: Sources consisting mainly of articles with sensational headlines and gossip,
often in the form of cliffhangers and listicles, with a financial incentive to gain
advertising revenue.
● Conspiracy: Sources mainly dedicated to propagating a range of explanations to events
behind which are secret plots and multiple actor entanglements.
Two researchers categorised the domains and discussed the debatable cases with other
researchers in this volume for higher intercoder reliability and used external sources like
mediabiasfactcheck.com. Mostly, these discussions were held for websites that could be
categorised with multiple labels or that fall between hyperpartisan and mainstream such as
tendentious ones (Peeters and Rogers, this volume). A caveat to this method is that categorising
websites on a source instead of story level results in stories being labelled, for example, as
“hyperpartisan”, even though the categorisation would differ on a story-by-story basis. Websites
like The Post Online, for instance, contain stories from press agencies as well as tendentious and
hyperpartisan ones. Despite this, the rigorous domain categorisation did allow preliminary
overviews, which is why it was fitting for the “bird’s-eye” perspective of this research. To show
these different categorisations (news or non-news, Dutch or non-Dutch, types of news), they
were visualised in treemap diagrams using the software RAWGraphs (Mauri et al., 2017).
Needle to Haystack
Next, the needle to haystack approach was employed to analyse the prevalence of Dutch junk news
in the entirety of Reddit and 4chan/pol/, now by starting with a list of URLs that were already
identified as questionable. This list was constructed by combining an edited list by De HoaxWijzer (“Valse Nieuwssites”, n.d.) with websites found through engagement analysis by
researchers in this volume (see Appendix II below). The list refers to Dutch domains known to
present news of questionable validity, with an overwhelming partisan tone but also occasionally
showing traits of conspiracism. The list was coded by the researchers who compiled it and
contains the categories hyperpartisan, clickbait, conspiracy, and disinformation.43 We fetched all
the posts containing these domains with 4CAT, resulting in 1714 posts on Reddit and 443 on
4chan/pol/.
We then “scoped” how often junk news appears over time, plotting it as histograms. To
compare these junk posts to all of Reddit, Google BigQuery was used. The total number of posts
within subreddits where at least one Dutch junk domain appears was fetched to calculate the
relative presence of junk news. Additionally, the “size” of the Dutch Reddit sphere and the
entirety of Reddit was retrieved through fetching the total number of posts on Dutch subreddits
and on Reddit overall. The data was mapped as circle-pack diagrams with RAWGraphs. For
4chan/pol/, we used 4CAT to fetch all posts (both opening posts and replies) mentioning one
of the domain names from the expert list in the full timeframe. In order to identify temporal
trends, the amount of posts with Dutch junk domains was plotted per month as histograms.

43 It also has the category tendentious-hyperpartisan, which seeks to capture sources like The Post Online that have

stories from press agencies as well as hyperpartisan columns and other contributions that could be described as
“edgy”, anti-establishment and against political correctness. (Tutors, this volume).

122

Characterising junk news propagation on Reddit
Finally, to characterise the kinds of actors propagating Dutch junk news and the effectiveness of
their activities, various metrics were calculated for (further anonymised) junk news posters on
Reddit. A similar analysis was impossible for 4chan/pol/ owing to the imageboard’s anonymity
and lack of “repurposable” objects to shine light on the posters. Taking the needle to haystack
approach, 4CAT and the Pushshift API were used to retrieve all posts by Reddit accounts who
posted a source from the expert list at least twice. The retrieved users were considered “junk news
propagators” for the purposes of this research. The following metrics were calculated for the
total corpus as well as for individual users:
●
●
●
●
●
●
●

Subreddits most posted often in.
Average score of all posts.
Average score of posts referring to Dutch junk news domain.
Most linked-to domains.
Total posts with domains to Dutch junk domains.
Percentage of posts linking to Dutch junk domains.
Total posts by user.

The Reddit users’ pseudonyms were further anonymised, since not the identity but rather the
characteristics of the users is of importance here. The first four metrics in the list above were
plotted for the whole corpus in histograms and circle diagrams, while all metrics were also
visualised in a matrix for the ten most active Dutch junk news posters, i.e., those who linked to
the domains from the expert list most often.
Following and categorising YouTube links
YouTube emerged as one of the most popular websites linked to on “Dutch Reddit” and
“Dutch 4chan/pol/”. Since the video platform is often described as offering alternative news
consumption, we also followed the links to YouTube videos in all posts in Dutch subreddits and
4chan/pol/ posts with a country flag of the Netherlands. Having collected these links, we used
4CAT’s “YouTube metadata” module (in turn using YouTube’s API) to retrieve metadata on the
videos linked to, such as video title, views, and topics. We then plotted the thumbnails of the
100844 videos that were linked to most often on image walls with a custom Python script. To
visualise what types of videos these concerned, we plotted YouTube’s “video categories”
(selected by the uploaders) on top of the image wall. We finally ranked the most-linked to
YouTube’s channels, derived from the full list of videos linked to on 4chan and Reddit, to gain a
grasp of the type of video content posted.
Scoping Dutch junk news
This section explores the scope of Dutch junk news on both platforms under study. We do so by
showing the volume of posts linking to one of the URLs in the expert list (i.e., the needle to
haystack approach). These are then compared to the overall volume of (Dutch) posts on Reddit
and 4chan/pol/.

44 We settled on the peculiar number of 1008 since we wanted a large sample and this number would make the

image wall adhere to the common 18:9 screen aspect ratio.

123

Scoping junk news on Reddit
First, as a way to ground the corpus, figure 3 shows the total amount of posts made on one of
the Dutch subreddits (appendix I). Just like activity on Reddit in general, Dutch activity is
increasing: in December 2015 there were just over 2,000 posts and comments per month,
whereas in January 2019 this number had grown to 14,000 and seems to be rising.
Figure 3. Total amount of posts and comments on one of the Dutch subreddits (appendix I).

Data source: 4CAT and Puhshift; timeframe: from 1-Dec-2015 to 1-Jun-2019; line graph.
Does this increase in activity also mean an increase in Dutch junk news linked to on Dutch
subreddits? As is evident in figure 4, the amount of posts linking to one of the domains from the
expert list started at a maximum of just eighteen instances in 2016. Two subsequent spikes can
be observed. The first one, in April to July 2017, speak to the “spammy” nature of some areas of
Reddit, since one user frequently posted a Dutch junk news domain (ninefornews.nl) to an
English subreddit. The second spike is more varied, however, showing a range of websites like
boinnk.nl, worldunity.me, and ninefornews.nl. Upon closer inspection, these were again posted
by a single account, mrthirdeye, the closest one will find to a “fake news troll”, though its posts
received little to no engagement (discussed in more detail in section 4). The subsequent dip in
November can be attributed to a content policy change in 2017, possibly leading to the banning
of this malicious account (Alexander 2017). In subsequent months, junk news sharing increased
somewhat compared to 2016 but remained fairly consistent with around fifty instances per
month. An increase of posts linking to a tendentious-hyperpartisan website, The Post Online, can
be seen in 2019, although no significant spikes during the 2019 Dutch provincial elections and
2019 European elections can be discerned. In perspective, these numbers do not seem extremely
troubling, especially since most posts link to hyperpartisan sources instead of outright
disinformation (see section 3), and furthermore do not receive a lot of engagement (see section
4).
124

Figure 4. Frequency of posts linking to Dutch junk news domains on Reddit.
Data source: Google BigQuery; timeframe: from 1-Dec-2015 to 1-Jun-2019; Stream graph.
To further put the scope of Dutch junk news on Reddit in perspective, figures 5 to 8 contain
circle pack diagrams that show its amount compared to the entirety of Reddit, as measured in
terms of posting activity (excluding comments). Figure 5 shows the size of all subreddits where a
link to a Dutch junk news source was shared at least once. The Dutch subreddits are tiny in
comparison to non-Dutch subreddits (figure 5), given the dominance of English-language
subreddits on the site. There are a few occasions when Dutch junk news was shared on very
large subreddits, such as r/viral, r/news, and r/worldnews, as well as the infamous pro-Trump
subreddit, The_Donald. Notably, however, in the Dutch subreddits, the proportion of junk news
is very low as well. As will be touched on, only occasionally does a subreddit have over 5% of its
posts linking to Dutch junk news. For the majority of subreddits, this figure is less than 1%.

125

Figure 5. Dutch versus non Dutch subreddits in which Dutch junk news appears. Size of circle represents the
overall number of posts in that subreddit within the timeframe, and colour represents the relative amount of posts
with junk news. Data source: Google BigQuery. Timeframe: 1-Dec-2015 to 31-Jan-2019. Visualisation: circle
pack diagram.
When zooming in on Dutch subreddits (figure 6), a clearer variation in the volume of junk news
is observable. The subreddits where these sources appear are still in small amounts, with the
highest percentages appearing in r/Forum_Democratie (5,37% of all posts),
r/meerderheidnederland (5,67%), r/de_thierry (4,93%), r/Duindorp (13%), r/The_Wilders
(1,97%), and r/FreeDutch (3,04%). Most of these subreddits are related to right-wing political
parties, ideologies or politicians, such as Geert Wilders or Thierry Baudet. These subreddits
appear mostly because of the frequent posting of links to hyperpartisan websites such as De
Dagelijkse Standaard.
When compared to the overall Dutch sphere on Reddit (figure 6), quite a large area of the Dutch
subreddits has at least some presence of junk news from the expert list. Still, the largest and most
mainstream Dutch subreddits (r/thenetherlands, r/cirkeltrek, r/Amsterdam) contain a negligible
amount. Dutch junk news can most notably be seen within already polarised or partisan spaces,
such as the right-wing subreddits listed above. “Neutral” subreddits like r/thenetherlands seem
fairly immune, likely because of a different user base and content moderation.
Lastly, figures 7 and 8 provide a zoomed-out visualization of the relative amount of Dutch junk
news in the entirety of posts on Reddit. While some Dutch junk news appears in a number of
both Dutch and non-Dutch subreddits, it pales in comparison to the total number of posts in
other subreddits in the research timeframe. Moreover, even though some Dutch junk news
appears on a number of large international subreddits (in turn, making the sphere appear large),
the relative number of appearances of Dutch junk news in those subreddits is close to zero.

126

Concluding, then, in terms of frequency, links to questionable Dutch-language news sources on
Reddit is a small issue outside of a few partisan subreddits.

Figure 6. Dutch subreddits where Dutch junk news appear compared to the size of all Dutch subreddits. Size of
circle represents the overall number of posts in that subreddit, and colour represents the relative amount of posts
with junk news. Data source: Google BigQuery. Timeframe: from 1-Dec-2015 to 31-Jan-2019.
Visualisation: circle pack diagram.

127

Figures 7 & 8. All Dutch and non-Dutch subreddits where Dutch junk news appear compared to the size of all
of Reddit. Size of circle represents the overall number of posts in that subreddit, and colour represents the relative
amount of posts with junk news. Data source: Google BigQuery. Timeframe: from 1-Dec-2015 to 31-Jan-2019.
Visualisation: circle pack diagram.
Scoping junk news on 4chan/pol/
4chan’s infrastructure allows less of a comparative approach than that of Reddit, but some
metrics can shine light on the relative appearance of Dutch junk news on /pol/. First, to scope
the Dutch sphere, the amount of posts with the country flag of the Netherlands is fairly stagnant
since late 2015 (figure 9). Each month, around 40,000 “Dutch flagged” posts are made. The
amount increased in March 2017, owing to the Dutch general elections. While these numbers are
128

lower in comparison to Dutch users on mainstream platforms like Facebook, they are fairly
comparable to Reddit, and a non-negligible number - a fairly significant insight considering the
extreme political ideas present on /pol/. It is impossible to tell how many individual people
these numbers of posts denote, however.

Figure 9. Line graph of posts with Dutch country flags on 4chan/pol/. Data source: 4CAT; timeframe: from 1Dec-2015 to 01-Jun-2019; line graph.
Despite the frequent Dutch posts on /pol/, the amount of posts linking to Dutch junk news is
quite low (figure 10). Links to Dutch junk news domains appear only around ten times per
month. One significant spike occurs in March 2017, caused by links mostly to The Post Online and
De Dagelijkse Standaard, again concerning the general election on March 15. Interestingly, a similar
spike associated with the elections is absent from Reddit. Afterwards, however, the amount of
posts linking to Dutch junk news drops, remaining low for both the 2019 Dutch provincial
elections and the 2019 European elections. Considering the total amount of posts by Dutch
/pol/ users (averaging around 40,000 posts per month), the amount of references to junk news
URLs should be considered negligible. This should not be equated with a lack of problematic
news content, however, as is discussed below.
Categories of Dutch junk news
How sizable of a role do online news media play within 4chan/pol/ and Reddit? What types of
domains are linked to when categorising news domains posted in these forums? What types of
junk news can we discern? This section uses the haystack to needle approach to walk through a
number of tree maps, each showing a different categorisation of the most-linked to domains.
First, the proportion of news websites is compared to non-news domains. After, the news
websites are sorted by Dutch or non-Dutch. Finally, the categories of these news sources are
outlined and discussed (mainstream, hyperpartisan, disinformation, etc.).

129

Figure 10. Frequency of posts linking to Dutch junk news domains on 4chan/pol/. Data source: 4CAT;
timeframe: from 1-Dec-2015 to 01-Jun-2019; streamgraph.
Firstly, figures 11 and 12 show which domains from the most-posted domain sample are
categorized as “news”. For Reddit (figure 11), 21,6% of all posts on Dutch subreddits refer to
“news” websites. Notably, tweedekamer.nl appears 15,694 times, caused by the bot
u/kamerstukken-bot posting parliamentary texts to the subreddit r/kamerstukken. Removing
this bot increases the news proportion to 50% - quite a considerable number. Other non-news
websites include reddit.com itself, often used to host images and text, and youtube.com and
youtu.be, appearing 951 times cumulatively.

130

Figure 11. Links to news (red) and non-news (blue) sources in posts in Dutch subreddits. Data source: 4CAT
and Puhshift; timeframe: from 1-Dec-2015 to 01-Jun-2019; treemap diagram.
4chan/pol/ paints quite a different news/non-news picture. At 16.6% the proportion of links to
news websites is lower than Reddit’s 50%. After twitter.com and en.wikipedia.org, a staggering
50% of URLs point to YouTube. Considering this major presence of Google’s video service, it is
further scrutinised as an alternative news sphere in section five.
Figures 13 and 14 show the news domains on Reddit and 4chan/pol/, respectively, according to
their origin (Dutch or non-Dutch). On Reddit, the domains shared on Dutch subreddits are
almost exclusively of Dutch origin. This is likely due to content moderation in these spaces,
requiring posts to be specifically about the Netherlands (e.g. on r/thenetherlands). The news
sources on 4chan/pol/ (figure 4), on the other hand, are predominantly from Anglophone
sources, such as The Daily Mail, The Guardian, BBC, and Reuters. This is fairly unsurprising
considering 4chan/pol/’s designation as an English language space, unlike the Dutch subreddits.
Still, it is worth noting that Dutch users on 4chan/pol/ are mostly concerned with English
sources and are thus more internationally oriented in terms of news propagation than users on
Dutch subreddits. This also implies foreign news sources might significantly influence their news
consumption. As shown below, English junk news is indeed posted by these Dutch “anons”.

131

Figure 12. Links to news (red) and non-news (blue) sources in Dutch posts on 4chan/pol/.
Data source: 4CAT; timeframe: from 1-Dec-2015 to 1-Jun-2019; treemap diagram.

Figure 13. Links to Dutch (orange) and non-Dutch (blue) news on Dutch subreddits. Data source: 4CAT and
Puhshift; timeframe: from 1-Dec-2015 to 01-Jun-2019; treemap diagram.

132

Figure 14. Links to Dutch (orange) and non-Dutch (blue) news on Dutch subreddits. Data source: 4CAT;
timeframe: from 1-Dec-2015 to 01-Jun-2019; treemap diagram.

Figure 15. Categories of news domains in posts on Dutch subreddits.
Data source: 4CAT and Pushshift; timeframe: from 1-Dec-2015 to 01-Jun-2019; treemap diagram.

133

Next, we explore the types of news sources, and if “junky”, how they can be categorised. Figures
15 and 16 show the categorisation of the shared news domains as mainstream, other
(mainstream), conspiracy, disinformation, hyperpartisan, and clickbait, as defined in section 1.3.1.
As is evident in the visualisations, mainstream or special interest (other mainstream) sources
make up the largest share of URLs posted on both platforms: 99,6% for Reddit and 81% for
4chan/pol/. Despite the frequent characterisation of pseudonymous spheres like Reddit as
“alternative”, these results are thus somewhat counterintuitive since mainstream sources make
up the dominant proportion links shared. On both sites, NOS.nl is the most-linked to news
source, meaning the established source is highly relevant. For Reddit especially, the lack of
problematic content in Dutch spaces is remarkable, as in these most-posted domains almost no
websites from the expert list can be found, save for a few instances of dagelijksestandaard.nl and
tpo.nl. Indeed, the platform and Dutch users show they seem to be inoculated against
“pulpnieuws”.

Figure 16. Categorised types of news from news sources posted 4chan/pol/.
Data source: 4CAT; timeframe: from 1-Dec-2015 to 1-Jun-2019; treemap diagram.
Dutch-flagged posts on 4chan/pol/ show a more problematic, hyperpartisan nature. Here, 21%
of top news domains are “junk”, with hyperpartisan sources making up most of these. Some of
these are foreign state-influenced and/or hyperpartisan, such as RT and Breitbart, and others are
outright extremist, like the neo-Nazi website, The Daily Stormer. As alluded to above, Dutch junk
news seems to play less of a role here. A few sporadic instances of far-right disinformation
appeared in the Dutch posts (shown in orange), all originating outside of the Netherlands. These
include links to Sputnik, the large Russian news website that has been known to propagate
disinformation (MacFarquhar, 2016; EUvsDisinfo, 2017), as well as two far-right websites that
post a large amount of Islamophobic stories, Speisa.com and The Gatestone Institute. In posts
134

linking to Russian sources, like RT and Sputnik, the top URLs are usually referring to refugee
slander, particularly in Sweden (see appendix IV). It is impossible to tell who posted these links
considering 4chan’s built-in anonymity, but it could potentially point to foreign interference.
Characteristics of Dutch junk news propagation on Reddit
What are the characteristics of online actors who share Dutch junk news? And are their actions
effective? While these questions are nearly impossible to answer for 4chan, considering its
anonymity, Reddit does afford “natively digital” (Rogers 2013) objects to explore the
characteristics and effectiveness of junk news propagators. This section will therefore discuss a
range of metrics and lists concerning Dutch junk news on Reddit.

Figure 17. Mean Reddit posts scores by Dutch junk news propagators (users who posted a link to a Dutch junk
news domain at least twice). Data source: 4CAT and Puhshift; timeframe: from 1-Dec-2015 to 01-Jun-2019;
bar graph.
298 Reddit accounts were found within the timeframe that linked to domains from the expert list
of Dutch junk news domains. Out of those, 193 accounts only posted a Dutch junk news URL
once. Only sixteen accounts did so ten times or more, meaning there is a long tail of occasional
junk news posters. When these junk news sources are linked to, they furthermore receive a lower
score on average than other posts these propagators make (figure 17). To reiterate, Reddit scores
are created by users’ “upvoting” or “downvoting” a post, with a high score meaning a post will
move to a higher position on a subreddit, thus receiving more visibility. As can be seen in figure
17, posts to non-junk news by these propagators outperform posts linking to one of the sites in
the expert lists, with the propagators’ mean score being 9.8 and the mean for their posts linking
to a Dutch junk source being 5.6. This is mostly caused by automated, “spammy” posts. The
median for each of these is 1 and overall, 1.24 of the 1.72 of junk news posts have a score of 1 or
less (72%), meaning the Dutch junk news posts receive little visibility and approval on average.

135

These low average scores do not mean that junk news stories are totally void of success,
however. 33 of the 1,761 posts received a score of 50 or more. Reddit’s infrastructure stimulates
a snowball effect of “rich get richer” posts, and some of these even scored higher than 1,000.
Zooming in on a URL instead of a domain level, it shows that most of these stories are
hyperpartisan of tone. Table 1 shows the top three highest-scoring posts on Reddit linking to a
domain from the expert list (see appendix V for the top 25). All three best-performing spots are
“junky” and Islamophobic in tone. The first concerns a story by De Dagelijkse Standaard on rape
and refugees. The second and third are both linking to the same story by Fenixx that framed a
man who drove a car into a group of people at Amsterdam Central Station as a Moroccan
terrorist, even though he was officially declared as unwell and confused. Interestingly, these
stories are posted in English-language subreddits, notably the pro-Trump r/The_Donald and the
now-banned r/CringeAnarchy, showing how junk news from the Netherlands spreads to foreign
spaces.
subject

url

subreddit

timestamp

score

Amsterdam Square driver
(terrorist) before declared a dagelijksestandaard.nl/201
confused and sick Dutch
7/08/onthullingnational is now revealed to werkelijke-naam-van-debe KHALID K. from
amserdamseCasablanca. The media
stationsrammer-blijkt-duscover up doesn't stop!
khalid-karmaoui/
The_Donald

22/08/2017
13:31

1811

fenixx.org/2017/05/14/ja
Japan ONLY Admits 27
pan-only-admits-27Muslim “Refugees”, Two muslim-refugees-twoAlready Arrested For Gang already-arrested-for-gangRape.
rape/
The_Donald

15/05/2017
06:02

1211

fenixx.org/2017/05/14/ja
Japan ONLY Admits 27
pan-only-admits-27Muslim “Refugees”, Two muslim-refugees-twoAlready Arrested For Gang already-arrested-for-gang10/08/2017
Rape.
rape/
CringeAnarchy
5:13
936
Table 1. The top 3 best performing posts linking to a Dutch junk comain on Reddit. Data source: 4CAT and
Pushshift. Timeframe: 01-Dec-2015 to 01-Jun-2019.
In summary, the overall performance of Dutch junk news throughout Reddit is fairly marginal.
Moreover, the high-scoring stories are usually hyperpartisan instead of clear-cut disinformation.
Dutch junk news thereby can garner considerable engagement on Reddit, but it does not do so
on a regular basis. In this sense, Reddit is more “resistant” to junk news than (for example)
Facebook is said to be (Burger et al., 2019).

136

Figure 18. Subreddits where Dutch junk news domains are most often posted in. Data source: 4CAT and
Pushshift; timeframe: from 1-Dec-2015 to 31-Jun-2019; circle pack diagram.
In which subreddits are Dutch junk news domains posted? Figure 18 shows that r/viral links to
most Dutch junk news with 543 instances, but much of the prevalence is caused by a single
“spam” account, receiving no engagement whatsoever. More interestingly, r/Forum_Democratie,
the unofficial subreddit for the currently the largest party in the Dutch Senate, comes in second
with 312 posts to junk news sites. Other right-wing partisan and hyperpartisan subreddits appear
further down the long tail, such as r/The_Wilders, r/FreeDutch, r/meerderheidnederland,
r/The_Donald, and r/de_thierry. This is mainly caused by posts on these subreddits linking to The
Post Online and De Dagelijkse Standaard.
According to figure 19, the junk domains that are linked to often are mostly the well-known
right-wing tendentious and hyperpartisan blogs, with The Post Online and De Dagelijkse Standaard
ranking on top. The “alternative” news website NineForNews, which has been host to
conspiracies and hyperpartisanship (Roermund 2017), is also amongst the most shared domains,
but this is mostly due to an automated bot posting links to the website (u/ninefornews). As such,
most of the shared junk news domains can be categorised as hyperpartisan, often with an
“alternative” right-wing stance. Signs of disinformation or coordinated Russian influence are
137

fairly marginal, with Novini appearing 22 times, a website known for pro-Putin sentiment (Heck
2017). As such, from this needle to haystack method, partisan and hyperpartisan content is easy to be
found, but coordinated disinformation or Russian meddling appears to be less of an issue within
these spaces.

Figure 19. Most linked to Junk news domains on all of Reddit. Data source: 4CAT and Pushshift; timeframe:
from 1-Dec-2015 to 1-Jun-2019; circle pack diagram.

author

Avg.
score
with
%
Dutch
Dutch Dutch
junk Avg. junk
junk
source score posts posts

Top domains

Top Dutch
junk domains

Top subreddits

user1

1

1

13.5

youtube.com: 416
welingelichtekringen.nl: boinnk.nl: 82
153
earth-matters.nl: 77
stopdebankiers.com: 61 viral: 4390
591 rt.com: 125

user2

1

1

100

294 ninefornews.nl: 294

ninefornews.nl: 294

news: 294

138

Total
posts

4390
294

user3

user4

user5

user6

user7

user8

user9

1

11.2

16

11.5

8.4

25.6

1.5

1.1

28

18.2

74.7

9

23.7

1.5

0.1

112.international: 9891
unian.info: 6238
70 liveuamap.com: 6210

dagelijksestandaard.nl:
23 fenixx.org: 9
politiek.tpo.nl: 4

russiawarinukraine:
6470
meerderheidnederla
nd: 788
oekraineukraine: 707

3.9

i.redd.it: 431
twitter.com: 352
60 imgur.com: 106

dagelijksestandaard.nl:
22
tpo.nl: 19
politiek.tpo.nl: 12

Forum_Democratie:
1337
FreeDutch: 83
The_Donald: 21

1525

12.7

twitter.com: 126
youtu.be: 60
62 tpo.nl: 50

tpo.nl: 50
opiniez.com: 10
tpook.nl: 1

Forum_Democratie:
489

489

16.8

geenstijl.nl: 52
twitter.com: 42
48 tpo.nl: 28

tpo.nl: 28
politiek.tpo.nl: 12
tpook.nl: 2

Forum_Democratie:
263
The_Donald: 11
FreeDutch: 9

286

3

youtube.com: 118
twitter.com: 91
29 i.redd.it: 64

dagelijksestandaard.nl:
13
tpo.nl: 11
opiniez.com: 3

Forum_Democratie:
954
test_forum: 3
JFVD: 2

961

7.7

youtube.com: 32
imgur.com: 14
19 twitter.com: 14

verenoflood.nu: 6
politiek.tpo.nl: 5
opiniez.com: 4

The_Wilders: 113
The_Europe: 60
The_Donald: 46

246

0.1

youtube.com: 5859
gellerreport.com: 1698
19 bitchute.com: 1677

fenixx.org: 19

news: 5813
worldnews: 5537
worldpolitics: 4232

22477

Table 2. Metrics of users who shared the Dutch junk news on Reddit. Data source: 4CAT and Puhshift.
Timeframe: form 01-Dec-2015 to 01-Jun-2019.
Finally, we highlight the Reddit accounts most active in propagating junk news to profile actor
types. Table 2 shows various metrics on the ten accounts ranked by the amount of posts linking
to one of the domains in the expert list. As indicated by total posts and average score, some of
the accounts post frequently but receive no engagement. Most of these are “spam” accounts or
automated bots. Interestingly, user 1, the aforementioned u/mrthirdeye68, has posted many URLs
to Russian and pro-Russian websites as RT.com and novini.nl, as well as mainstream sources and
hyperpartisan websites like the far-right website Red Ice TV. It is possible that user 1 is a Russian
“troll”. It received no upvotes, however, and only posted links to the obscure subreddit r/viral,
meaning it did not garner any engagement. As such, it is likely this user is an automated bot, or
some hybrid. Other bots seem more effective, however. User 2, for instance, is the
abovementioned ninefornews.nl bot, posting a hundred percent of posts to this website in the
global news subreddit r/news. Of interest here is that user 2 does receive engagement, with a
fairly high average post score of 570 and a junk news post score of 50. As such, Reddit is at least
somewhat susceptible to manipulation, depending on the “strategy” of its users.
In terms of issues, it can be discerned that the most active accounts are either concerned with
Dutch right-wing parties or topics surrounding Ukraine. Despite their frequent linking to junk
news websites, the most active accounts still link most often to platforms like YouTube and
Twitter.45 A cohort of four right-wing partisans can be observed, who are most active on
45 Further research might scrutinise what YouTube videos or Tweets are linked to, for instance to identify further

“newsy” sources or influencers.

139

66860

r/Forum_Democratie and frequently link to websites like GeenStijl and The Post Online. Most of the
accounts actually use the Dutch language, and, upon closer inspection, are also likely Dutch
natives. As such we may conclude that there is not a significant attempt of foreign accounts
meddling with Dutch affairs, with the possible exception of the now-banned u/mrthirdeye68.
YouTube as an alternative news network
Thus far, this text has handled “news” in the conventional sense of designated outlets publishing
on current affairs. As discussed in the introduction, however, the consumption of both amateur
and professional reporting increasingly occurs on social media. These modern, alternative ways of
news consumption cannot be identified when the “needle” is formulated as traditional news
outlets. As we have identified in section 3, URLs linking to YouTube are frequent, especially on
4chan. As discussed elsewhere in this volume, the video hosting site is host to various spheres of
alternative news commentary and opinions, leading Zeynep Tufekci to describe it as “the Great
Radicalizer” (2018). Can we indeed outline an “alternative news network” working in tandem
with 4chan/pol/ and Reddit? This section briefly touches on this question by visualising and
categorising the most-posted videos on 4chan/pol/ and Reddit, as well as the most popular
channels.
Figures 20 and 22 display the thumbnails of the 1008 most-posted YouTube videos on our
Dutch Reddit and 4chan/pol/ corpora. Figures 21 and 23 shows the “video categories” for each
of these videos. For the top videos on Dutch subreddits, 161 are concerned with “People and
Blogs”, 129 with “Entertainment”, and 118 with “News & Politics”. From this, the type of
content shared is fairly diverse. 4chan/pol/ is more concentrated on news and politics, with 196
videos categorised as such, with “People & Blogs” following at 95 and “Entertainment” at 64.
The number of missing videos for 4chan (the black labels) is notable, comprising almost half of
the total, indicating 4chan’s extremism as well as YouTube content moderation. The amount of
deleted videos is visibly less on Reddit.
If one takes the videos labelled as “News & Politics” as an indicator of a “news source”, as we
categorised in the sections above, it becomes possible to quantify the role of YouTube as a news
source on the two platforms. The “News & Politics” category comprises 11.7% of the stillonline videos for Reddit in the sample above, and 19.4% for that of 4chan/pol/. Considering the
total amount of links to still-online YouTube videos in this timeframe – 7,667 for Reddit and
26,635 for 4chan/pol/ – one can estimate that around 896 “News & Politics” videos were
posted on Dutch subreddits and 3,748 on 4chan/pol/ by users with a Dutch flag.46 Comparing
these numbers to those presented in section 3, for YouTube news videos would constitute the
largest and second largest source of news content. On Reddit, they would form the secondlargest news source, only behind NOS.nl with 1,615 mentions. For 4chan, YouTube is by far the
largest player in relation to news circulation, since the next most popular source, NOS.nl (861
mentions), comprises only one-third of the amount YouTube news videos. As such, the role of
YouTube as a new player in the circulation and consumption of news should not be understated.
Is this dominant presence of YouTube of great significance in the study of junk news? Table 3
shows the 25 most-occurring channels from all of the YouTube links in our two Dutch corpora.
Here, the platforms differ significantly. On Reddit, some partisan channels can be discerned, like
the one for Forum voor Democratie and PVV pers, but the list mostly consists of “established”
46 In reality, these numbers will be somewhat lower because not every YouTube URL points to videos (they can also

refer to e.g .channels), although the vast majority in our corpus does.

140

sources like NOS, GeenStijl, and VPRO Zondag met Lubach. On 4chan/pol/, however, they are
far more extreme and potentially harmful. The most-posted channel is SouthFront, dedicated to
videos on the Syrian civil war. Below that is Stefan Molyneux, a popular Canadian YouTuber
who promotes “scientific racism” and white supremacist views. Further down the list are
(hyper)partisan news channels like Fox News as well as the Russian RT and Ruptly. Other farright YouTubers and channels also appear, like Paul Joseph Watson and Rebel Media, as well as
some left-leaning channels like The Young Turks and VICE. Together, the channels referred to
by Dutch posters are thus of a hyperpartisan, sometimes far-right makeup. As such, YouTube
videos on Dutch subreddits seem to align with consumption of “established” and “traditional”
news media outlets, while those on 4chan/pol/ show a highly hyperpartisan and polarised
landscape.

Conclusions
Despite the frequent characterisation of Reddit and 4chan as “alternative” zones on the Web, the
results presented in this text generally do not reveal a large share of alternative news networks
spreading disinformation within the platforms, at least in a Dutch context. Despite a few
instances of pro-Russian websites like Novini and one suspicious Reddit account, coordinated
campaigns of malicious users posting links to disinformation seem largely absent. Dubious
content can certainly be discerned but compared to overall activity (as shown in section two) it
should be considered fairly marginal within the spaces we scrutinised. Reddit seems especially
resilient against the circulation of junk news. In turn, the characterisation as actors within 4chan
and parts of Reddit as influential “agenda setters” should therefore likely be taken with a grain of
salt.

Figure 20. The top 1008 most-posted YouTube videos in Dutch subreddits. Black labels denote deleted
videos/channels. Ranked left to right, top to bottom. Data source: 4CAT, Pushshift, and YouTube API.

141

Figure 21. The top 1008 most-posted YouTube videos in Dutch subreddits, with video categories as an overlay.
Black labels denote deleted videos/channels. Ranked left to right, top to bottom. Data source: 4CAT, Pushshift,
and YouTube API. Image wall.

142

Figure 22. The top 1008 most-posted YouTube videos in 4chan/pol/in posts with a Dutch country flag. Black
labels denote deleted videos/channels. Ranked left to right, top to bottom. Data source: 4CAT and YouTube
API. Image wall.

Figure 23. The top 1008 most-posted YouTube videos in 4chan/pol/in posts with a Dutch country flag, with
video categories as an overlay. Ranked left to right, top to bottom. Black labels denote deleted videos/channels.
Data source: 4CAT and YouTube API. Image wall.

143

4chan/pol/ - top 25 most-occurring
channels
channel

count

South Front

191

Stefan
Molyneux
Fox News
RT
The White
House

channel

Reddit - top 25 most-occurring channels

count

channel

count

channel

count

FOX 10
Phoenix

49

AFC Ajax

476

AT5

38

177 sanderson1611

46

VitesseTV

269

FvD Meems

37

41

Forum
Democratie

143

De Speld

36

39

Omroep
PowNed

107 Football-Oranje

32

38

Politie
#PRO247

95 LISSAUER

31

82

vpro.nl

30

156

PewDiePie

155

Paul Joseph
Watson

Acts17Apologe
136
tics

Ruptly

125

Rebel Media

37

BRAXATOR
ES

Right Side
Broadcasting
Network

112

VICE

37

VPRO Zondag
met Lubach

69

De Telegraaf

30

Omroep
PowNed

108

Fullwhiskey

36

GeenStijl

67

NOS op 3

30

U.S.
Department of
State

76

VICE News

34

Cafe
Weltschmerz

51

Hoop Stront

29

CNN

68

ABC News

34

NOS

51

PVVpers

28

Forum
Democratie

67

DeroVolk

33

WNL

48

RTL Z

27

Fox Business

65

corbettreport

31

Xbox

47

Politie Den
Haag

24

The Young
Turks

56

TopNotch

43

Table 3. The most-occuring YouTube channels from all YouTube links posted in the Dutch Reddit and

4chan/pol/ samples. Data source: 4CAT, Puhshift, and YouTube API. Timeframe: from 01-Dec-2015 to 01Jun-2016.
What can be observed, however, are the types of junk news that can be characterised as
hyperpartisan, especially on 4chan/pol/. This appeared mostly through links to popular
tendentious and hyperpartisan blogs like The Post Online and De Dagelijkse Standaard, but also the
more clearly “fake” (in the sense of conspiratorial) NineForNews. This right-wing bias is expected
for 4chan/pol/ due to its infamy as a far-right hub; for Reddit it is more notable because we
took a politically diverse range of URLs and subreddits as a starting point. While labelling these
websites as “fake” is problematic, they do indicate a non-negligible presence of polarising
content. Indeed, section four showed that the most-engaged-with articles from these websites
often concern topics like migration and Islam, instead of other geopolitical events like Russian
interference.

144

Nonetheless, mainstream sources such as NOS.nl remain popular linked-to domains on both
Reddit and 4chan/pol/. This is somewhat counterintuitive since it has been argued the “fringe”
characterisation of these pseudonymous and anonymous spaces implies their users find
knowledge in different epistemological drawers. Despite these assumptions, the prevalence of
mainstream sources shows they have at least some authority within these online spaces. It is
important to note, however, that we have not considered how these mainstream websites are
discussed. Likely, domains like NOS.nl are considered on Reddit as a trustworthy source, while
on 4chan/pol/ it might be referenced purely to ridicule it or to portray it as “fake news” itself as is discussed elsewhere in this volume.
One should furthermore not be blinded by exclusively considering websites devoted to reporting
on current affairs as the sole source of news, as YouTube emerged as a particularly big “new”
player in relation to news consumption and circulation, especially on 4chan/pol/. On Reddit,
“News & Politics” videos on YouTube are estimated to form the second-largest source, while on
4chan/pol/, they are estimated to strongly outperform any other news source. From a brief
exploration of the YouTube channels posted on both platforms, it seems Dutch Reddit is largely
linking to fairly established sources, like PowNed, Zondag met Lubach, and NOS, while on
4chan/pol/, alternative, hyperpartisan, and problematic information channels emerged, like Mike
Cernovich and RT. As such, non-Dutch YouTube content might have a “radicalising” role on
Dutch users within certain Internet forums.
Since this report concerns the news sources linked to by actors on Reddit and 4chan, it does not
shed light on the grassroots production of alternative news or conspiracies within these spaces. As
Tuters et al. (2018) show in relation to the Pizzagate conspiracy, the wildest theories can be
cooked-up in these spaces through a short burst of a “butterfly effect” of “bullshit”,
unobservable when merely considering the prominence of URLs. A more holistic approach, also
taking into account text and images, could thus aid in further contextualising the current “fake
news” debate. Instead of identifying isolated issues of “fake news”, such broader approaches
could tackle the interwoven problematics surrounding the circulation of “junk news” (Venturini
2019) and “network propaganda” (Benkler et al. 2018), from the conspiracist mindset of
“virality-oriented subcultures” to the prevalence of polarising hyperpartisan content.

145

Appendices
Appendix I: Compiled list of Dutch Subreddits

ADODenHaag,AjaxAmsterdam,Alkmaar,Aluhoedjes,Amersfoort,Amster
dam,AmsterdamEnts,Appiememes,Arnhem,Aruba,Assen,avd,AZAlkmaar
,Bassie_en_Adriaan,BeermoneyNL,BeNeLux,Bier,Binnenhof,Bitcoin
NL,Boeken,Bonaire,BuurmanEnBuurman,CariceVanHouten,CelebsNL,C
irkeltrek,CreatieveKoppen,Curacao,de_thierry,de_thierry,DeCor
respondent,DeGraafschap,Delain,Delft,Depressie,DeSpeldOfNietD
eSpeld,DeStaat,DeStagiair,DeTandenborstel,DirkJan,Dordrecht,D
oucheGedachten,DoutzenKroes,Drenthe,Duindorp,Dumoulin,Dunglis
h,dutch,Dutch,DutchBoardgames,DutchComedy,DutchDesign,DutchEn
ts,DutchFIRE,DutchHipHop,DutchHouse,DutchKeto,DutchMusic,Dutc
hPoetry,DutchProblems,DutchSkincare,DutchTech,Eindhoven,Elfst
edentocht,Enschede,Epica,Eredivisie,Ethtradernl,FCGroningen,F
CTwente,FCUtrecht,Feyenoord,Formule1,Forum_Democratie,FreeDut
ch,Frisia,Frysk,GekkeJongens,Geldzaken,Geschiedenis,Gezellig,
Glitterplaatjes,groenlinks,Groningen,Haarlem,HanzeMemes,Heilz
ameMeems,Hulpdiensten,ik_ihe,JuridischAdvies,Kamerstukken,Kat
holiekeNederlanden,Kibbeling,KNVB,Koffie,Kut_Doen_Op_Tinder,K
utleven,Kutreclames,LearnDutch,LeCutInsideMan,Leiden,Leraren,
LimburgMan,Lowlands,Maastricht,MamaAppelsap,Marktplaats,MaxV,
Medejongeren,meerderheidnederland,Metal_NL,Motorfietsen,NACBr
eda,NEC,Nedercringe,Nederporn,NepParlement,netherlands,Nether
landsPics,NietDeSpeld,Nijmegen,NLvsFI,Nuenen,NuJijInActie,oek
raineukraine,ossem,otonde,Papgrappen,ParadoxPlaats,PECZwolle,
PodcastNed,PokemonGoNL,Poldersocialisme,Politiek,Politiekmeme
s,PSV,RijmenDichten,RMTK,RodaJC,RomeeStrijd,Rotterdam,Saba,SC
Cambuur,SCHeerenveen,ScoutsNL,SportNL,Spyker,StefanieJoosten,
Strips,Stroopwafels,StudyInTheNetherlands,SXM,SylvieMeis,Tene
nkrommend,The_Klaver,The_Wilders,TheHague,theNetherlands,theN
etherlandsFree,theNetherlandsNature,Tiesto,tokkiefeesboek,tok
kiefeesboek,TokkieFeesboek,Top2000,TUDelft,TuurlijkIsDatEenDi
ng,Twente,Utrecht,VeganNL,Veluwe,VitesseArnhem,Voetbalnieuws,
VraagDerNederlanden,VraagHetAanTonyQuark,Wetenschap,WIDM,With
inTemptation,XboxNederland,Zitkamer,ZonderContext,Zwolle
Table 4. Compiled list of Dutch subreddits.

146

Appendix II: Expert List of Dutch Junk News Domains
name
Opiniez
Stop de Bankiers,
t Pallieterke
E.J. Bron
Dagelijkse Standaard,
Climategate
De Staat van het klimaat
JDreport.com
tpook.nl
Nine for news
Daily Paper
Parra
Viraaltjes
about media
Martin Vrijland
The Loyalist
desportgek
Even Delen
nietbarkie.nl
hardewaarheid.nl

domain_name
opiniez.com
stopdebankiers.com
pallieterke.net
ejbron.wordpress.com
dagelijksestandaard.nl
climategate.nl
destaatvanhet-klimaat.nl
jdreport.com
tpook.nl
ninefornews.nl
dailypaper.org
parra.nu
viraaltjes.nl
aboutmedia.nl
martinvrijland.nl
loyalist.nl
desportgek.nl
evendelen.net
nietbarkie.nl
hardewaarheid.nl

The Post Online
tpo.nl
Saltmines.nl
saltmines.nl
eunmask.wordpress.com
eunmask.wordpress.com
novini.nl
novini.nl
niburu.nl
niburu.nl
React nieuws
reactnieuws.net
DMLplus
dlmplus.nl
martinvrijland.nl
martinvrijland.nl
world unity
worldunity.me
cultuur onder vuur
cultuurondervuur.nu
volks nieuws uit Amsterdam noir volksnieuwsuitamsterdamnoir.com
stop pas familie drama
stoppasfamiliedrama.blogspot.com
Obed Brinkman
obedbrinkman.noblogs.org
veren of lood
verenoflood.nu
De fouten van Rutte
defoutenvanvvdrutte.nl
Finding voices
finding-voices.blogspot.com

category
hyperpartisan
hyperpartisan
hyperpartisan
hyperpartisan
hyperpartisan
hyperpartisan
hyperpartisan
hyperpartisan
clickbait
conspiracy
hyperpartisan
clickbait
clickbait
clickbait
conspiracy
conspiracy
clickbait
clickbait
clickbait
clickbait
tendentioushyperpartisan
hyperpartisan
hyperpartisan
hyperpartisan
conspiracy
hyperpartisan
conspiracy
conspiracy
conspiracy
hyperpartisan
conspiracy
conspiracy
hyperpartisan
hyperpartisan
hyperpartisan
conspiracy

147

ik was haren
Piet Kei
bewiseman
Alternatieve Media Nederland
Apokalypsnu
Don Quijotte
Drimble
Fenixx
Hector Reban
Herstelde Republiek
Kremlin Troll
Magilando
Niburu
Absolute Duality
Stan van Houcke
Stelling
Tref
Want to know
Xandernieuws
APost
Best Gezond
Bewust Nieuws
Blik Op NOSjournaal
Bovendien
Brekend Nieuws
Dagelijks.nu
Dagelijkse Krant
De Stille Waarheid
Earth Matters
Ella’ster
Health Bytes
Healthwatch
gezondheidswaakhond
Leeshetnu
Lekkerwonen
LikeMag
Lijstverse
Live kijken
Nieuwsdump

ikwasinharen.nl
pietkei.nl
bewiseman.nl
alternatievemedianederland.com
apokalypsnu.nl
donquijotte.wordpress.com
drimble.nl
fenixx.org
hectorreban.wordpress.com
herstelderepubliek.wordpress.com
kremlintroll.nl
magilando.wordpress.com
niburu.co
nl.absoluteduality.com
stanvanhoucke.blogspot.com
stelling.nl
tref.eu
wanttoknow.nl
xandernieuws.punt.nl
apost.com
bestgezond.nl
bewustnieuws.nl
blikopnosjournaal.blogspot.nl
bovendien.com
brekendnieuws.nl
dagelijks.nu
dagelijksekrant.nl
destillewaarheid.nl
earth-matters.nl
ellaster.nl
healthbytes.me

hyperpartisan
conspiracy
hyperpartisan
hyperpartisan
conspiracy
conspiracy
hyperpartisan
hyperpartisan
hyperpartisan
hyperpartisan
hyperpartisan
conspiracy
conspiracy
conspiracy
hyperpartisan
conspiracy
hyperpartisan
conspiracy
hyperpartisan
clickbait
clickbait
conspiracy
hyperpartisan
conspiracy
conspiracy
clickbait
clickbait
hyperpartisan
conspiracy
conspiracy
conspiracy

healthwatch.nu
leeshetnu.nl
lekkerwonen.org
likemag.com
lijstverse.nl
livekijken.nl
nieuws-dump.nl

conspiracy
clickbait
clickbait
clickbait
clickbait
clickbait
clickbait

148

Not100
Ongelooflijke Verhalen
Prankster
Revolutionair Online
Snuggerd
Time 2 Wake Up
Tips & Weetjes
Tis Wat
TrendBuzz
Trendnieuws
Trendnova
United-Lightworkers
Vaccinatieraad
Viraalpunt
Viral Mundo
Viraaltje
Vrouwen Dingen
Vrijspreker

not100.nl
smullen-maar.nl
prankster.nl
revolutionaironline.com
snuggerd.nl
time2wakeup.me
tipsenweetjes.nl
tis-wat.nl
trendbuzz.nl
trendnieuws.nl
trendnova.nl
united-lightworkers.be
vaccinatieraad.nl
viraalpunt.nl
viralmundo.nl
Viraaltje.nl
vrouwendingen.com
vrijspreker.nl

The Post Online - Politiek
Erkenbrand
Das Kapital
Glop

politiek.tpo.nl
erkenbrand.eu
daskapital.nl
glop.nl

clickbait
clickbait
clickbait
hyperpartisan
clickbait
hyperpartisan
clickbait
clickbait
clickbait
clickbait
clickbait
conspiracy
conspiracy
clickbait
clickbait
clickbait
clickbait
hyperpartisan
tendentioushyperpartisan
hyperpartisan
hyperpartisan
hyperpartisan

Table 5. Junk news categorisation (expert list). Edited and enhanced list originating from Hoax-Wijzer.

149

Appendix III: Metrics on domains shared on Reddit and 4chan/pol/
Reddit
01-12-2015 to 01-062019

OPs

Category

Count

Percentage

Positive Negative Percentage

Mainstream

5255

89.9%

News

5959

27594

21.64%

Other

580

9.7%

Dutch news

5557

402

93.3%

Hyperpartisan

24

0.4%

Dutch junk
news

24

5935

0.4%

Disinformation

0

0%

Clickbait

0

0%

Conspiracy
0
Table 6. Metrics for the proportions of news, Dutch news, Dutch junk news, and categories in posts on Dutch
language Subreddits, 01-12-2015 to 01-06-2016.
Reddit
01-12-2015 to 01-062019

Count

0%

OPs

Category

Percentage

Positive Negative Percentage

Mainstream

10399

71.5%

News

14541

87301

16.6%

Other

1414

9.5%

Dutch news

3403

11138

23.41%

Hyperpartisan

2091

14.4%

Dutch junk
news

2809

11732

19.3%

Disinformation

241

1.7%

Clickbait

45

0.3%

Conspiracy

351

2.4%

Table 7. Metrics for the proportions of news, Dutch news, Dutch junk news, and categories in posts on
4chan/pol/ with a country flag from the Netherlands, 01-12-2015 to 01-06-2016.

150

Appendix IV: Most-posted URLs from posts containing links to RT.com and Sputnik on
4chan/pol/
Title

Stabbing death of 15yo schoolboy by
‘Arab migrant’ classmate in Sweden
sparks outrage
Sweden: Rape Capital of the West

Sweden charges 5 teenage refugees with
beating, gang-raping boy for over an
hour

Belgian prosecutor’s office denies
terrorist track in murder of guard at
nuclear center

Sex Slave Found Chained in Basement
of Immigrant Cafe in Sweden

URL

Amount of
appearances

https://www.rt.com/news/32924
3-sweden-migrant-stabbedteenager/

9

https://www.gatestoneinstitute.or
g/5195/sweden-rape

9

https://www.rt.com/news/36941
5-sweden-refugees-rape-afganboy/

9

https://www.rt.com/news/33727
6-belgium-nuclear-guard-killed/

8

http://speisa.com/modules/article 8
s/index.php/item.3584/sex-slavefound-chained-in-basement-ofimmigrant-cafe-in-sweden.html
(now offline)

Table 8. Most occuring URLs from posts containing links to RT.com and Sputnik by posts with a Dutch
country flag on 4chan/pol/. Derived with 4CAT.

151

FAKE NEWS AND THE DUTCH YOUTUBE
POLITICAL DEBATE SPACE

Marc Tuters

Fake News and the Dutch YouTube Political Debate Space
Marc Tuters
Introduction: YouTube as radicalising platform
On 1 February 2019, De Volkskrant and De Correspondent published a much anticipated report on
YouTube as a radicalization platform: “Leidt het algoritme van YouTube je naar extreme
content?” (Translated: Does the YouTube algorithm lead you to extreme content?) (Bahare et. al.
2019). Drawing on data analysis produced by some of the same authors of this current report, it
sought to investigate the extent to which YouTube functioned as an engine for online
“radicalization” (Tufekci 2018; Holt 2017). As these and other reports claimed, YouTube
appears to be playing a significant role in the development of a new antagonistic culture of
debate, in which an “alternative influence network” is said to have the capacity to shape public
opinion, especially amongst a demographic of young and politically rightward leaning men
(Lewis 2018). Amongst the figures who have risen to prominence through this YouTube debate
culture, is for example the now internationally well-known, Canadian academic psychologist
Jordan Peterson. Peterson is often viewed as a conservative political figure, even as a member of
the so-called “alt-right” (Lynskey 2018). This latter term, which stands for “alternative right”,
gained popularity in the aftermath of the 2016 US election as a means of describing a seemingly
new breed of conservative online activism that brought together a diverse array of actors united
against the perceived hegemony of “politically correct” liberal values, often through a jokey and
transgressive style (Hawley 2017; Heikkilä 2017; Nagle 2017). Whilst Peterson has refuted an
association with the alt-right, in consulting how the YouTube algorithm itself categorizes
Peterson it would appear that the platforms nevertheless still views him in this light. How exactly
this categorization works is inscrutable to all but the owners of the platform. And while it should
not be taken as definitive proof of what a given channel is about, we can nevertheless assume
that YouTube’s categorization does reflect some essential aspect of its bottom line, which is to
keep the most people watching for the longest time possible.
The present research report uses the same platform-centric categorization method as introduced
above, applying it to studying the space of Dutch parliamentary political debate on YouTube.
While initially motivated by the question of how this space engages with the issue of “fake
news”, the report however moves away from defining fake news as disinformation (which is to
say the deliberate manipulation of facts) towards conceiving of it in terms of a form of
“hyperpartisan” information as produced by “openly ideological web operations” (Herrman
2016). This latter conception of fake news is furthermore also resonant with the redefinition of
the term as it has begun to be appropriated by politicians around the world in order to describe
news organizations whose coverage they find “disagreeable” (Wardle and Derakshan 2017, 16)
— notably by Donald Trump who often refers to “establishment” media outlets such as CNN
and the New York Times as fake news (Weisman 2018). In the European context, where laws such
as the German Netz DG have been passed at the national level rendering platforms responsible
for policing this problem, such critics have framed the attempt at regulating fake news as a
“blueprint” for state censorship (Wardle and Derakshan, 2017: 71). In spite of these
controversies, the bewildering issue of fake news, entangled as it is together with broader
changes in political and media spheres at a variety of levels, remains relatively understudied
152

outside of the American context — the latter which is in many ways quite unique for a variety of
factors (Benkler et al., 2018: 381-387).
Whilst the precise mechanisms of YouTube’s algorithms are unknown, what is clear is that they
are designed to optimize “engagement,” defined in terms of “views” as well as the number of
“comments”, “likes”, and so forth (Covington et al., 2016). In recent years, YouTube’s algorithm
has been critiqued as creating a so-called “rabbit-hole effect” (Holt, 2017), whereby the
platform’s algorithms, as mentioned above, have been accused of recommending ever more
extreme content, in an effort to keep viewers engaged. It has thus been argued that this particular
environment has helped to draw audience from the mainstream towards the fringe. Along these
lines, it has indeed been argued that, on YouTube, “far-right ideologies such as ethnonationalism
and anti-globalism seem to be spreading into subcultural spaces in which they were previously
absent” (Marwick and Lewis, 2017: 45). Academic researchers exploring this phenomenon have,
for instance, found that YouTube’s “recommendation algorithm” has a history of suggesting
videos promoting bizarre conspiracy theories to channels with little or no political content
(Kaiser and Rauchfleisch, 2018). Beyond this current “radicalization” thesis, for some years new
media scholars have observed that YouTube appears to multiply extreme perspectives rather
than facilitating an exchange or dialogue between them — as for instance observed in an earlier
audience reception study of polemical documentary produced by the Dutch parliamentarian
Geert Wilders and published to YouTube (van Zoonen et al., 2011).
We may perhaps want to consider the growth of a new combative and conspiratorial culture of
debate on YouTube, as documented by these more recent YouTube studies, in the context of
broader global political shifts that have been picking up pace in the latter part of the 2010s, the
latter which may be referred to under the umbrella term of “national populism” (Eatwell and
Goodwin, 2018). Referred to as “thin ideology” (Mudde and Kaltwasser 2017), populism is
characterized by a suspicion of the “elite” as well as a purist notion of the “general will” of the
true people, the latter which is not necessarily equivalent to the democratic electorate (Muller,
2016). Recent new media scholarship has convincingly demonstrated how such populist anti-elite
sentiment translated readily into an embrace of alternative news media, particularly in the US
context in which the rise of an “alternative partisan news system” is said to have played a crucial
role in the last presidential election (Benkler et al., 2018). While there exists right and left variants
of the concept, right-wing populists tend to have an advantage in speaking to nationalist issues
(Goodwin and Eatwell, 2018). In the analysis of political scientists Matthew Goodwin and Roger
Eatwell, national populism can be characterized by four factors, that they call the “four D’s”.
These are a distrust in the liberal “establishment”, the destruction of long-held communal identity
owing to forces of globalization, the relative deprivation as “neoliberal” economics leads to a rise
in inequality and finally the political de-alignment from traditional political parties. Whatever the
political valence of national populism going forward, Goodwin and Eatwell conclude that these
four factors are destined to have “a powerful effect on the politics of many Western countries
for many years to come” (Goodwin and Eatwell, 2018).
Fakeness and hyperpartisanship
Thus far the problem of fake news has primarily been studied in the context of Anglo-American
national populism, specifically the political communication surrounding the Brexit referendum
153

and the insurgent Trump campaign and subsequent presidency. Furthermore, most current
studies of fake news have tended to focus on the US context, where institutional trust levels in
media and in the government are said to be at an all-time low (Edelman, 2018) and political
polarization stands at an all-time high (boyd, 2017). In that context, it has been noted that the
standard designation of “fakeness”, as a diagnosis to be remedied by “fact-checking”, fails to
acknowledge a much more profound epistemological problem. As has long been argued in the
literature on the sociology of scientific knowledge, “facts” are better understood as products of
negotiated settlements amongst domain experts (Latour and Woolgar, 1976). The atmosphere of
general suspicion towards expertise that underpins the rise of national populism thus poses a
fundamental epistemological problem. This same general atmosphere of suspicion furthermore
works to undermine trust in professional media institutions as the arbiters of facts. It is argued
that this particular context plays into an innate psychological tendency to seek out biasconfirming information.47
A leading scholar in the field recently posed the dilemma thusly: in the US, somewhere between
“25 and 30 percent of Americans willingly and intentionally pay attention to media outlets that
consistently tell that audience what it wants to hear, and what that audience wants to hear is often
untrue” (Benkler et al., 2018: 367, emphasis added). In the aforementioned context, such scholars
furthermore suggest that technocratic solutions designed to regulate and censor this fake news
would be “neither feasible nor normatively attractive as they would certainly generate heated
protest from a large spectrum of the populace” (367). Even in less politically polarized contexts
the problem of regulation is extremely challenging. It is not isolated cases of fake news that are at
issue but the larger problem of what these scholars refer to as “network propaganda”, which
constructs “materially misleading” narratives from a tissue of facts (102). Because it is extremely
difficult to establish “ground truth”, reliable technological solutions to the problem of fake news
are thus unlikely at present (377).
In light of the former diagnosis, the empirical study below reframes the issue of “fake news” in
the Dutch-language YouTube space by profiling the emergence of a network of channels
engaged in political debate and commentary. It conceptualises elements of this network as
hyperpartisan, in the sense that they are “openly ideological web operations” (Hermann, 2016).
Whilst marginal in comparison to mainstream Dutch news organizations these channels
nevertheless appear highly engaging, at least from the perspective of the YouTube algorithm. As
alternative news organizations almost all of these channels are unique to YouTube, making them
“natively digital objects” (Rogers, 2013: 1). The empirical research that follows is thus concerned

47 Indeed from the social psychology perspective, “fake news” would arguably represent a more “natural” human

preference than “facts”, insofar as the former more readily provides support that conforms to the “moral
foundations theory” of human values (see: Haidt 2012).

154

with understanding how these channels work, what their issues are, how they “do” Dutch
national politics, and how they differ from the mainstream.

Figure 1. Related channels on YouTube. Table where the top row displays the name of each Dutch political party
and the columns below each of these are the media organizations associated with each party’s YouTube channel.
29 March 2019.48
YouTube’s “related channels” and Dutch political space
Following the “digital methods” approach (Rogers 2013), the discussion that proceeds here can
be considered as an endeavour to “repurpose” YouTube as a research device by thinking along
those lines that the platform makes available the public. In particular the approach uses
YouTube’s “related channel” algorithm as the basis for an analytical method that takes a set of
Dutch alternative news channels as its primary site of study. As a forewarning, it is important to
recognize the contrived or “artificial conditions” with which the medium frames the object
(Rieder et al., 2016: 3). These conditions effectively make it impossible for the digital methods
researcher to identify where the medium ends and where in turn the social begins. Though we do
have a sense of how some of YouTube algorithms work from both the official corporate
statement (Press, 2019), as well as from attempts by scholars to “reverse engineer” or
“teardown” the platform (Bessi et al., 2016), the precise functioning is unknown and in any case
likely to change, thus frustrating the exact reproducibility of any of our findings. At any time,
YouTube may furthermore suddenly and unaccountably change its algorithms, which are in any
case invisible to all but certain engineers at YouTube. Needless to say, the capriciousness of platforms
renders the effective control of variables practically impossible. Whist the latter is axiomatic to
digital methods it should also be recognized as an inherent limitation of the methods as well. For
48 Note that the Dutch labour party visualized on the far right of the graph did not return any related channels.

155

these reasons the present report is thus best approached as “snapshots” of a milieu that is
constantly in flux.
The empirical research focuses primarily on repurposing YouTube’s “related channels” for the
purpose of analysis of the Dutch political space. In order to delineate what we are here calling
the Dutch “political debate space” in YouTube, we started from the channels corresponding to
the Dutch political parties. Since all 13 Dutch national political parties currently in the
parliament have a YouTube presence, we used these channels as a “seed list”, or set of starting
points, for the subsequent research. Starting then from this seed list the first technique compares
all of the channels that YouTube classified as related to those of the Dutch political parties. This
particular approach to categorization in all likelihood involved no human oversight; rather, it
should be understood as an artefact of how the algorithm “values” the object, in relation to the
aforementioned “engagement” metrics. Following the digital methods approach, the analytical
gambit here is that the channels that YouTube’s suggests may be treated as a measure of how the
platform views those parties.49
The most unusual finding is that the algorithm relates one particular channel to almost all parties
across the political spectrum: Forum voor Democratie (FvD). As a new “Eurosceptic” party
with a younger demographic than the established nationalist populist Partij Voor de Vrijheid
(PVV), FvD and its agenda seem to dominate discussion in political debate in a network of
“alternative” channels discussed below, several of which YouTube relates to the parties, most
notably “TheLvkrijger”. Before going on to discuss these alternative news channels in detail, the
next most striking finding here is how the algorithm seems to organize the political spectrum in
relation to different “establishment” news organizations. One cluster of parties is associated with
CNN, ABC, NOS50 and another around De Telegraaf, media organizations that may be considered
as relative liberal and conservative/populist, respectively. While it is not necessarily easy to
arrange the Dutch political parties on a left-right axis — as many smaller parties are more issuebased — it is worthy to note how the algorithm groups the new Groen Links and Denk parties
with center-right and right-of-center parties. In addition to De Telegraaf, the algorithm also relates
all of the parties in this latter cluster to alternative Dutch news organization: Omroep PowNed, a
public radio and TV broadcast renowned for its satirical news show, PowNews, which often
ridicules politicians with provocative questions. In what follows we will categorize Omroep
PowNed, along with GeenStijl a blog popular for its similarly abrasive style, as members of the
established anti-establishment alternative news organizations.

49 One should note here that social media use machine-learning for “predictive models of consumers”

(Kitchen/Dodge) in which “success” is a measure of how correctly the algorithm predicts what a user will engage
with. A well-known critique here is the notion of the “filter bubble” (Pariser 2011), which argues that algorithmic
categorization can have the effect of narrowing the range of alternate viewpoints that one is exposed to.
50 Note that we removed most US channels from Dutch media network visualization below.

156

Figure 2. TheLvkrijger post: Translated in English to: “He who is silent agrees! Don’t shut up anymore! This is
your country! Claim it”.
That the algorithm also relates the parties to a smattering of large Dutch commercial and public
media channels (WNL, RTL Nieuws, NPO Radio 1, Veronica Inside), is unsurprising as these
would be an expected part of an average Dutch media diet. What is likely surprising to those
unfamiliar with the Dutch political space in YouTube is the network of alternative or “alt”
channels that YouTube relates to the parties, notably the aforementioned “TheLvkrijger”, but
also “ARNews”, “Luekste YouTube fragmenten”, “Lissauer” and “Rafiek de Bruin.” With the
possible exception of ARNews, all of these channels could be categorized as “openly ideological
web operations”. As we will see, these Dutch political debate channels are “natively digital
objects… ‘born’ in the new medium” (Rogers, 2013: 19), as opposed, for example to Omroep
PowNed. While some of these channels, like TheLvkrijger, are transparently partisan, national
populist sentiments seem common in this space, as for example captured in a post by
TheLvkrijger encouraging viewers to vote in the upcoming elections, which featured the slogan
“He who is silent agrees! This is your country! Claim it”.

157

Figure 3. Related channels on YouTube. Panoramic graph of larger Dutch YouTube media sphere. This graph
was reproduced twice two months apart with identical outcomes on 29 March 2019 and again on 22 May 2019.
The Dutch YouTube media sphere
In an effort to create a panoramic graph of the larger Dutch YouTube media sphere that would
also remain connected to the Dutch political sphere on the platform we used YouTube’s related
channels algorithm to “snowball” out from the seed list of the 13 parties to 3 degrees of
relations. We subsequently visualised the related channel network with network analysis
software, where nodes represent channels and edges represent relations according to YouTube’s
algorithm. The relative size of the text represents a measure of their relative importance within
the network. Finally, relative similarity between channels determines their colouration, clusters
which we have then labelled as government, military, commercial, vlog, public, sport and finally our
specific object of study. The largest nodes in the graph are all “establishment” media organizations
with NOS Jeugdjournaal, RTL Nieuws and De Telegraaf at the center. Slightly outside of the
center another large node is the established, anti-establishment channel Omroep PowNed,
158

known once again for its “edgy” confrontational style of reportage. If one continues along this
same line one encounters the cluster labelled alternative media ecology at the center of which the
most connected node is FvD but which also includes a few government channels (for example
Eerste Kamer) as well as a number of the aforementioned “alt” political debate channels which
we encountered earlier (for example “Leukste YouTube Fragmenten”). In the next steps of the
analysis we will delve more deeply into these “alt” debate channels channels by performing some
qualitative analyses of their content.
In both the panoramic map as well as in the prior analysis (based on only a single degree of
relations to the seed list), we find the presence of a number of “natively digital” political debate
channels, such as Leukste YouTube Fragmenten and TheLvkrijger. In considering these
channels as a type of mini-genre, we can thus compare their style and how they “do” Dutch
politics. At the outset it should be noted that, by certain measures, some of these channels
appear quite marginal. TheLvkrijger, for example, which YouTube related to half the parties,
only has 6.5 thousand subscribers. CNN, which YouTube also related to half the parties, has 6.5
million subscribers. The Dutch political space on YouTube is not that large, however, and in any
case, despite differing by orders of magnitude, YouTube related channels algorithm places CNN
and TheLvkrijger on the same footing. One degree of relations gives us a collection of “alt”
political debate channels including 'TheLvkrijger’, ‘Leukste YouTube Fragmenten’, ‘Rafiek de
Bruin’, ‘LISSAUER.COM’, ‘Res Cogitans’, ’Omroep PowNed’, ‘ARNews’, to which we can add
a few more by exploring their relations including ‘GeenStijl’, ‘AllePolitiek’ and
‘Deweycheatumnhowe’. In analyzing their style we can observe that ‘TheLvkrijger’, ‘Leukste
YouTube Fragmenten’ , ‘Rafiek de Bruin’, ‘AllePolitiek’ and ‘Deweycheatumnhowe’ are all of a
sort, in that all post debate clips or interviews. Furthermore, sites as ‘ARnews’ and ‘LISSAUER’
use “meme” graphics — a style also employed, and in fact pioneered to an extent, by PowNed
and GeenStijl. Somewhat like Omroep PowNed in style, GeenStijl is famed for its provocative antiPC tone. Settled in the Dutch media landscape (and with PowNed receiving structural funds
from the government), they can thus fairly be labelled as “established anti-establishment”. Using
clickbait tactics to attract attention, with the notable exception of AllePolitiek, the aim of these
channels seems to be to amplify dissensus in the Dutch political space. Whilst this of course
stands in marked contrast to the country’s long history of consensus politics, where one
normatively stands on this depends on one’s democratic political theory. Furthermore, whilst
several channels are transparently partisan, what it remarkable is that the majority of the most
viewed videos in most of the channels focus on figures from the FvD and PVV.
To provide a synoptic view of the natively digital debate channels’ issues one can look at the
most commonly used words in the titles of all of the channels in the form of “word clouds” with
words colour-coded and sized by frequency. Those appearing in black are issues such as
referendum, climate agreement, dividend tax and Brexit, whist those in colour are the names of
parties and their spokespeople. At first glance, what one notices is that ARNews and AllePoeliek
appear primarily issue driven, whilst the other channels seem more engaged with Dutch political
personalities. One can also observe the relative similarity between ResCogitans and Leukste
YouTube Fragmenten, as channels that both appear partisan towards FvD — on closer
inspection this is indeed the case (and in fact they even appear to be run by the same person).
Similarly, TheLvkrijger appears to be partisan towards the PVV, which is also the case on closer
159

inspection. As with the thumbnails, discussed above, the names of the figures from both these
parties commonly appear in all these channels video titles. Further scrutiny reveals all of these
channels to be at least somewhat sensationalistic, with ARnews, often using terms like heated
debate (“verhit debat”) in order to describe content. The more partisan of the channels follow an
antagonistic logic when commenting on parliamentary debates, identifying the winner or loser of
a given debate, at times resembling a debate genre familiar on YouTube, for example in videos
featuring Jordan Peterson, often labelled in the style: Jordan Peterson DESTROYS so and so.

Figure 4. Thumbnail diagram of the ‘fringe channels’’ top ten most popular videos.
Alongside the related channels findings, the fact that official Dutch parliament channels, along
with Forum voor Democratie (but not the other Dutch political parties), seemed to clustered
alongside these “alt” debate channels seems peculiar. Given the aforementioned capriciousness
of platforms, might these findings be attributable to an excited algorithm in the aftermath of FvD’s
surprising success in the senate elections? If so, then one would expect for these findings to
differ when reproduced at another point in time, either revealing an underlying stable state of
network composition or else another excited state. With this question in mind we reproduced
these first two methods, that were initially explore at the time of the senate election, at the time
of the EU election. Remarkably, we found no substantial difference in either the channels that
YouTube considered as related to the parties (see Appendix 1). Moreover, the panoramic graph
remained identical,51 suggesting that it may thus reflect an underlying stable state of how the
algorithm currently categorizes the larger Dutch YouTube media sphere in YouTube (see Figure
3). Because the EU elections did involve several other parties, we did however identify the
presence of two new clusters in the panoramic graph: one of which, associated with the new
pan-European Volt party, floats on its own completely disconnected from the overall network;
and another, associated with Dutch Pirate Party, which is connected to the larger network via a
51 YouTube disabled the related video feature shortly after we completed this analysis (YouTube, 2019).

160

channel ‘talking-head chat show’ called ‘Cafe Weltschmertz’. In close proximity to the alternative
media ecosystem discussed above, Cafe Weltschmertz seems to frame its political debates in a
tendentious style similar to some of the channels profiled above — referring to its approach, for
example, as “politically incorrect”. In this same cluster we also however find leftist investigative
journalism channels including ‘Follow the Money’ and De Correspondent as well as the expected
channels focused on the issue of privacy, ‘Bits of Freedom’, ‘Privacy First’ and finally a debate
channel called ‘Potkaars Podcast’ featuring a video on its front page, entitled “Potkaars prat met
iedereen”. In light of our subsequent discussion of fake news as a topic of debate, the video’s
description is worth quoting at length: “If you want real news, you have to cut through the
smoke -smokes & mirrors- to get to information and demand a controllable government. Putting
information away as ‘fake news’ is easy. But what do you replace it with?”

Figure 5. Screenshot from the “About” page on Cafe Weltschmertz’s YouTube channel which includes a sarcastic
“trigger warning” for viewers whom might be angered by its frank approach to political debate, as well as cryptonormative espousal of “democratic hygiene processes”.52

On fake news as issue
The final analysis concerns how it is that channels in the Dutch political space “do” the one issue
in particular: fake news. We begin with a video from TheLvkrijger of PVV representative Martin
Bosma confronting the government minister of Internal Affairs, Kajsa Ollongren in a Tweede
Kamer debate on the fake news that became central to her portfolio. In the video Bosma accuses
Ollongren of “playing a strange game” with “what is truthful and what is not”.
Bosma points to a fundamental lack of consensus of what’s at issue in the fake news controversy

52 Without offering any analysis of this particular unique term, for reasons of brevity and focus, it is nevertheless

worth noting here that one of the signature accomplishments of some of the American alternative partisan news
system, especially those on the far-right, has been to introduce new terminology in the hopes of normalizing certain
formerly radical conceptual frames (Hatewatch Staff, 2015; Benkler et al., 2018: 128-132). In political punditry this
technique is sometimes called “opening the Overton window” (Marwick and Lewis, 2017: 11)

161

more generally as well as alleging that Ollongren has seemingly tended to change her own
definition of what constitutes fake news in order to suit her political purposes. When examining
the comment section below this video we see commenters echoing Bosma’s sentiments and
questioning Ollongren’s integrity, expressing the need for a concrete definition of fake news (45
likes). Commenters furthermore speak of Dutch public broadcasting as fake new that does
‘nothing but mislead citizens’ (78 likes).53 This latter use of the concept of fake news echoes
Trump’s use of the term as means of attacking the establishment media.
Another video of interest, also published by TheLvkrijger, features a PVV-organized populisttype debate with pundits on the topic of fake news (‘nepniews’) and the European Union. Similar
to the aforementioned Trumpian framing of fake news, the debate discusses the supposedly leftwing bias in the establishment media, as represented in one participant’s statement that “media
serve the ideology of the establishment”. Again we see positive reception on the comment
section where a commenter writes about the Dutch public broadcaster “NOS = FAKE NEWS”,
and advocates viewers to seek their news form alternative sources on YT.54
In another video on the topic, this time published by GeenStijl, a reporter asks politicians leaving
the Tweede Kamer about the issue of ‘fake news’. This time the reporter’s questioning revolves
around proposed European legislation, rather than Ollongren’s engagement with the issue. As
per the channel’s provocative style, the video does not hide its partisan stance on the issue, titling
the video: “Brussels is censoring free speech”. Again representative Bosma appears, this time
with an attack on liberal political correctness emanating from the liberal technocrats in Brussels,
stating “everything that is not politically correct will be tackled”.55 By contrast other politicians
interviewed by the journalist see the necessity of government action in response to the ‘crisis’ of
fake news. In the comments section multiple commenters again reiterate the theme of the Dutch
Government itself being “fake news”.
A video published on Leukste YouTube Fragmenten features a Tweede Kamer debate fragment,
once again on the concept of freedom of speech, this time by FvD leader Thierry Baudet. In this
clip Baudet makes a sophisticated conceptual point on the alethiological (the study of truth).
Using logic, Baudet tries to refute Ollongren’s concept of fake news as fallacious. He argues that
if for an atheist god is not true, then that would not make preaching a form of disinformation.
Based on this argument he then claims that Ollongren would “accuse the teachings of
Catholicism of being untrue” and thus “a form of disinformation”. After his sophistry, Baudet
then goes on to make the point that state actors should not be allowed to decide what is true and
what is not true. “You cannot trust the state”, he says, what “we need”, he argues is “free press”.
In the comments section commenters state that all politicians, besides Baudet, define fake news

53 The number of likes on a comment can be treated here as a measure of agreement with these sentiments

expressed therein.

54 This theme of framing of “NOS is fake news” and “NOS is left-wing propaganda” came up in multiple

comments of multiple videos.
55 Political correctness is a very popular straw man amongst “dark intellectual web” figures like Jordan Peterson on
the right (Weiss 2018), but also left-wing figures such as Slavoj Zizek.

162

subjectively in particular falling back on the Russian “evil actor” narrative, which a commenter
characterized as “Orwellian”.

Figure 6. Word Clouds of the titles of all the videos from the political commentary channels.

163

Figure 7. Screenshot of a comment under the video of ‘Leukste YT Fragmenten’, referring to a ‘hopeless debate’
and the lack of consensus on the definition of ‘nepnieuws’.
Although our analysis in the report did not include any left-of-center Dutch political
commentators, this is not to say that they do not exist on YouTube, merely that the methods we
used did not bring them to the fore. Indeed, alongside the “alt” channels profiled above we can
in fact find a video of Arjen Lubach’s Zondag met Lubach, the VPRO broadcast in which the
commentator, as with the one on the Green Style video, critiques the Russian “evil actor”
narrative. In Lubach’s opinion the real threat is in fact an alt-right conspiracy theory, in the style
of Pizzagate, which Russian actors merely amplify.
Conclusions: Left-leaning bias?
Academics are often accused by conservatives of having a left-leaning bias; indeed, apparently
evidence reflects these allegations (Abrams, 2016). This narrative of “liberal bias” has been one
of the central themes of the American new right, extending from contemporary “neoreactionaries” (Malice 2019), to 1990’s “culture warriors” (Nagle 2017), and back to the 1960’s
“messengers of the right”, who pioneered new media formats in order to disseminate their
message (Hemmer 2016). And whilst accusations of such perceived liberal bias may be offered
against this report, the fact remains that we came by the data underlying our findings by merely
following the platform and the way that it categorized the Dutch political parties. In doing so we
identified a series of “alternative” debate channels many of which appeared hyperpartisan —
following Hermann’s initial definition of the concept as “openly ideological web operations”. If
we were to locate the political bias of these “alternative” political debate channels in relation to
“establishment” media organizations in the Netherlands, then many would seem to be roughly
aligned with the conservative and populist tone of De Telgraaf. Closer still to the antagonistic
debate style that we observed in many of these channels is the transgressive style of reportage
pioneered by the “established anti-establishment” of GeenStijl and Omroep PowNed.
The Netherlands is also well known for having innovated new new media formats, notably reality
TV. Additionally, one might also say that the Netherlands has been innovative in developing new
positions and issues on the right — notably the issue of homo-nationalism (Aydemir 2011).
What we may however also be seeing in this research is the possible emergence of US-style rightwing punditry in the Dutch sphere. While it still seems marginal in the current “alternative”
debate space on YouTube space, exemplary here is the channel of ‘Paul Nielsen’ (24,531
subscriptions), an English language Dutch “alt-lite” channel which features such titles as: “NOS
is the Dutch CNN | Biased News in Holland” and “How Marxists took over the Netherlands”.
164

The site claims to be endorsed by Prof. Dr. Paul Cliteur, defendant at Wilders trial and PhD
supervisor to Thierry Baudet. This channel may be a bridging node to the figures in what has
been called YouTube’s “dark intellectual web” (Weiss 2018) or its “alternative influence
network” (Lewis 2018), such as for example Stephan Molyneux who features a video with the
title: “The Truth About Immigration and Crime in the NL”. At the same time, in scrutinizing a
network one should be careful of the guilt by association fallacy. The point is rather to
acknowledge the proximity to an active and controversial area of debate within the platform.
While the possible intervention of “Russian trolls” as a factor in 2016 US elections has been
convincingly made (Jamieson 2018), the Dutch case is different. In addition to the absence of an
Anglo-American “first past post” electoral system there is a very different media ecosystem in
the Netherlands, which, for example still has a much higher trust in the general “establishment”
than in the US (Edelman 2018). Furthermore, as opposed to the “neutrality” axiom that has
characterize 20th-century US news media, Dutch news media have always been partisan. This
having been said what we see in YouTube suggests the emergence of a hyperpartisan Dutch new
media political space. Currently this is mostly dominated by one party, but other parties may take
this as a challenge. Insofar as YouTube represents a media source in the Netherlands, especially
for youth, the Dutch YouTube “alt” political debate space may represent a re-politicization of
youth, which runs counter to neoliberalism’s historical project of pre-emptive depoliticization
(Foucault 2008). If political pluralism advocates peaceful coexistence of different interests the
combative and anti-politically correct tone of much of political debate on YouTube may militate
against this. Can the long tradition of consensus in Dutch culture be brought to bear on this new
debate culture or is the Netherlands on the path to Americanized Trump-style polarization? In
terms of final takeaways, we can say that an inquiry into fake news, which defines the latter as the
deliberate manipulations of facts, must also consider the inherently problematic aspects of this
very conception as well. For this reason, regulating disinformation can be portrayed as Orwellian
“thought control” which in turn resonates with populists’ anti-establishment, conspiratorial
frameworks.

165

Appendix 1
Related channels on YouTube, 22 May 2019. Table where the top row displays the name of each
Dutch political party who ran candidates in the EU election. As with figure 1, the columns below
each of these are the media organizations associated with each party’s YouTube channel. The
related channels for the parties are identical to figure 1 apart from a few minor differences and
the fact that D66 now no longer returns any related channels, as with PvdA. Note also that of
the two EU parties that return channels are categorized quite differently than the other national
Dutch political parties.

166

CONCLUSIONS: MAINSTREAM UNDER FIRE

Richard Rogers and Sabine Niederer

Conclusions: Mainstream under fire
Richard Rogers and Sabine Niederer
Separating disinformation and fake news and developing other notions further
Disinformation and fake news are contemporary phenomena with rich histories. Disinformation,
or the willful introduction of false information for the purposes of causing harm, recalls
infamous foreign interference operations in national media systems, such as the Russian
campaign ‘Operation Infektion’ that in the early 1980s effectively publicly linked the HIV virus
with a supposed, secret US bioweapons lab. Outcries over fake news, or dubious stories that
have the trappings of news, have occurred repeatedly with the introduction of new media
technologies that disrupt the publication, distribution and consumption of news – from the socalled rumour-mongering broadsheets centuries ago to the blogosphere more recently. Social
media are only the most recent ‘truthless’ media. Designating a news organization as fake, or
calling it der Lügenpresse, however, has a darker history, associated with authoritarian regimes or
populist bombast diminishing the reputation of ‘elite media’ and the value of inconvenient truths
more generally.
These days social media platforms have been implicated in both the practice of disinformation as
well as the rise of these two varieties of fake news. As discussed in the theoretical and empirical
scholarship to date, social media have enabled the penetration of foreign disinformation
operations, the widespread publication and spread of dubious content as well as extreme
commentators with considerable followings attacking mainstream media as fake.
Worldwide, disinformation and fake news are increasingly under study together, but the
argument could be made to separate them. Indeed, in the Netherlands evidence of foreign
disinformation campaigning is scant; cases of domestic actors employing the ‘Russian playbook’
of disinformation tactics are also seldom documented. Unlike in the case of the US, to which
much of the scholarship is dedicated, in the Netherlands one has not witnessed the rise of
imposter news organisations or the formation of advocacy groups purporting to represent social
groups or causes. Indeed, when employing narrow definitions of disinformation and fake news,
there is hardly any to be found in the Netherlands.
But definitions of fake and its next-of-kin ‘junk’ news often extend to clickbait, conspiracy,
hyperpartisan and tendentious sources as well as artificially amplified social media content and
accounts. As a case in point, when Buzzfeed News famously reported in 2016 that ‘fake news’ was
outperforming mainstream news on Facebook, included in its definition were clickbait and
hyperpartisan sources, such as Breitbart News. Expanding the definition in such a manner would
have consequences in that the Netherlands has all of them in relative abundance.
Initial studies have found that the Dutch are great consumers of clickbait and ‘pulp’ content;
there is a well engaged-with set of tendentious and highly partisan news-like organisations
especially on the right of the political spectrum, and the artificial amplification of social media
accounts, including those of certain politicians and musicians, has been well documented. Their
sway varies. Clickbait is said to be consumed more often than mainstream news, though there is
167

also more of it. Conspiracy is perhaps the least clicked on, at least according to our findings per
platform, discussed below. In political spaces online, news furnished by commercial and public
broadcasting are still referenced, liked or shared in greater quantities than tendentious and
hyperpartisan sources, though the latter has been present in the most engaged-with lists of
sources around election issues. Artificial amplification both burnishes one’s image but also has
led to mini-scandals when fake followers are revealed through new online detection tools and
news reporting.
Whether any of them is particularly persuasive is a question increasingly posed. The first wave of
scholarship on the production and spread of disinformation has yielded to the next wave on its
effects. Given people’s hardened attitudes the question concerns whether the influence of
disinformation and fake news is ever more than ‘minimal’.
In that regard, the rise of extreme content (including extreme clickbait), circulated on social
media platforms, is one source of continuing consternation and measurement, leading to calls for
platform regulation and prompting social media companies to hire more content reviewers and
work on automated detection. Another source of concern is the mainstreaming of doubt and
trust in public institutions and media, concomitant with the rise of both ‘alternative facts’ and
‘alternative fact infrastructures’. The post-truth condition, as it is termed, is discussed as both
first-order ‘fact fights’ as well as second-order competitions between ‘sectarian knowledge’
regimes and competing media ecologies. Is the authority of mainstream news and knowledge
institutions declining for increasing segments of society that consume the alternatives? One
finding often related is that older consumers are ‘available audiences’ for fringe media and are
relatively ‘heavy users’.
The consuming and sharing of fake news have been the subject of media literacy initiatives,
including quizzes, serious games and public service campaigns. Through heightened awareness,
especially around the time of elections, the impact on consumers of any disinformation and
dubious content may be mitigated and the institutions made resilient, it has been argued.
Voluntary and professional fact-checking are also discussed in this regard, as are (automated)
content flagging, together with the need for human review. The question regularly posed is
whether the sheer amount of platform junk will overwhelm the capacity to review it, together
with the related issue of who will review the reviewers.
Finally, there is widespread scholarly concern about the restrictions to public access of social
media data, otherwise known as the issue of ‘locked platforms’. Future research should address
the extent to which disinformation and fake news (in all its definitions) continue to thrive online,
and whether there are monitoring capacities in place so that its overall consumption and
persuasive capacity may be measured and the wider societal implications may be studied and
acted upon.
Empirical findings concerning junk news around the Dutch elections of 2019
The present study consists of a series of empirical case studies concerning the engagement with
fake/junk news, together with hyperpartisan and tendentious sources, in Dutch political spaces
in social media in the run up to two elections in 2019. These spaces were demarcated using
168

queries of politicians’ and political party names as well as social issues, some related to the
elections (such as climate and EU) and others more controversial (such as Zwarte Piet). Here the
findings are summarised, and subsequently put into perspective in a discussion of their
implications, also for policy.
The overall research question driving the study is as follows. To what extent do disinformation
and so-called fake or junk news resonate in political spaces online within social media (and
search engine returns) around the 2019 provincial elections and the European parliamentary
elections in the Netherlands?
Here the findings are provided in brief.
1) We found neither foreign disinformation (campaigning) nor fake advocacy groups operating
around the Dutch provincial and European parliamentary elections of 2019.
2) Mainstream news is consumed and engaged-with more than junk news, but not for all
platform in all periods under study (8 February - 25 March 2019, 26 April - 24 May 2019 or
longer durations). The issue spaces around Zwarte Piet and MH17 have proportionately higher
quantities of junks news than election issues and are also ‘year-round’ issues, so to speak.
3) With respect to social media manipulation, troll-like users are active across Dutch political
issues spaces (on Twitter). We also suspect there is artificial amplification taking place (fake
followers on Instagram).
4) There is the emergence of a hyperpartisan/tendentious right-wing (separate) media space,
competing with the mainstream news and also mainstreaming, in the sense that these sources are
spread by regular (non-suspect) social media users as well as troll-like users.
5) Proportionately, Facebook has the greatest amount of fake/junk news compared to other
platforms, followed by Twitter. YouTube is a significant right-wing news space generally, and
debate space for ‘fake news’ as issue.
6) Dutch 4chan and Reddit circulate far more Dutch mainstream than junk news, with 4chan
users likely commenting upon rather than taking over its narratives. 4chan is an incubator of farright activity in the Netherlands as seen through the types of YouTube videos referenced.
Facebook: Fertile ground for junk news
The method behind the research presented here derives from data journalism, particularly that of
Buzzfeed News, and later the NRC Handelsblad, which both ascertained the most engaged-with
stories on Facebook in the run-up to national elections. Whereas Buzzfeed News found that
‘fake news’ on Facebook outperformed mainstream news in the months preceding the US
federal elections of 2016, leading in part to the overall ‘fake news crisis’, the NRC Handelsblad,
deploying a far stricter definition, found scant presence of such material prior to the 2017 Dutch
national elections. Our study found that Facebook is a fertile ground not for disinformation and
fake news in the Netherlands but rather for junk news, a roomier definition than fake news as
169

discussed above. Whilst it is not outperforming mainstream news, it is far from marginal, and in
a few periods under study its presence in the top stories on Facebook (judged from engagement
measures) equals or outperforms the mainstream. Engagement, measured by such interaction as
shares and likes, requires further study, however, for it should not be equated in each case with
support or agreement.
While Facebook contains a significant stream of junk news, including hyperpartisan, tendentious
and conspiracy sources, foreign disinformation and fake news – such as organisations pretending
to be news sources or advocacy groups – were found to be absent, at least for the most engagedwith stories related to the elections gathered through keyword queries of political leaders, parties
and social issues (or in longer longitudinal studies of certain issue spaces on Twitter as well as in
Dutch spaces on Reddit and 4chan). Disinformation and fake news may have not been
encountered, but junk news is a factor in Dutch political spaces on social media and its impact
should be critically studied.
Google Web Search: vernacular search queries result in junk news
The Google Web Search study involved locating fake/junk news within the top twenty results in
Google.nl for queries of Dutch political parties and significant social issues prior to the Dutch
provincial and European parliamentary elections of 2019. The queries were formulated by
combining the names of the political parties with social issue keywords. The keywords derive
both from the ‘official’ issue language collected from the party platforms as well as vernacular
terms distilled from the comments on political party Facebook pages. Of the junk news found
nearly all originate from hyperpartisan and tendentious sources rather than disinformation,
conspiracy theory or clickbait. For particular groups of issue queries, up to 25% of the results
were hyperpartisan. As on the other platforms under study, in the ‘top content’ no foreign
disinformation, fake news organisations or fake advocacy groups were present during the
election periods.
The social issue keyword queries in combination with right-of-center political parties resulted in
junk news sources in greater quantities than that of left-of-center party names. It was also found
that the presence of junk news is not stable over time. Prior to the provincial elections the
quantities rose, only to decline the day of the election and in its immediate aftermath, as
witnessed by the issue of migration. The inverse was witnessed during the European
parliamentary elections. When comparing the two types of search queries, the official and the
vernacular, the latter results in a higher percentage of junk news in the results.
Twitter: junk news and troll activity around polarising subject matters
The Twitter study examined the presence of junk news, tendentious news as well as troll-like
activity during the campaign periods around the Dutch provincial and European parliamentary
elections of 2019. There was troll-like activity encountered around the provincial elections
around political terms such as the tag for the elections themselves (‘PS2019’), certain party
leaders as well as potentially polarising issues such as MH17, Zwarte Piet and the Utrecht tram
shooting of March 2019. Troll-like activity refers to a series of behavioural indicators, including
targeting politicians with unusually high bursts of tweets in short period of time or through a set
of accounts created at about the same time. The analysis found fourteen troll-like Twitter users
170

were active around all issues studied and twenty-nine around most of them. Four of these
profiles remained active (or became active again) around the European parliamentary elections.
They all spread mainly hyperpartisan and tendentious sources, followed in quantity by conspiracy
websites.
In answering the question of the extent to which junk news is present in the Dutch political
Twittersphere, we ascertained the most shared sources, finding a steady resonance of junk news,
paling in comparison however to mainstream sources. One finding of note is that during the
Provincial elections both Zwarte Piet and MH17 witnessed proportionately high amounts of
activity, in spite of the fact that the Santa Klaus holiday (where Zwarte Piets make their
appearance) does not take place until December and there was no particular MH17 news, for
example concerning the investigations into the cause of the crash of the airliner. Troll-like users
shared mostly hyperpartisan and tendentious sources, followed by conspiracy websites putting
forward theories concerning MH17 and the Utrecht tram shooting. The pro-Russian site,
novini.nl, which on a story level oscillates between hyperpartisan and conspiracy, also was
circulated by troll-like users for all social issues under study, but only rarely in relation to political
leaders. During the European Parliamentary elections junk news occasionally resonated more
than mainstream news around such polarising issues as Zwarte Piet and MH17. It outperformed
mainstream news largely owing to the lack of news coverage of these issues during the periods
under study, when junk news remains steady.
Based on the findings, it appears that the Dutch Twittersphere does not have a disinformation
problem; no professional or large-scale disinformation or fake advocacy campaigns were
encountered. Troll-like users, whether in the form of bots or semi-automated users autoretweeting and posting original content, do lend some symbolic power to divisive points of view
around several social issues. Polarising issue activity, fuelled by reference to hyperpartisan and
tendentious news, remain rather stable (albeit marginal) throughout both periods of study,
suggesting that these issues do not resonate at expected times of the year only, but rather
throughout.
Instagram: a separate far-right media ecology and signs of artificial amplification
The Instagram study inquired into the presence (and absence) of disinformation and fake/junk
news in three ways: on a post-level, a source-level and that of followers. On a post-level, we
examine the circulation of fake/junk news in political spaces, on a source-level we compare the
audiences of fake/junk news and political leaders and parties, and finally we study the follower
bases of the political entities, searching for signs of inauthentic behaviour. In all we found a
relatively healthy Dutch political arena on Instagram with only small amounts of junk news and
fake followers. The vast majority of liked content in Dutch political Instagram, demarcated
through political keyword queries, is not junk news, though around certain right-wing political
leaders and divisive issues small amounts of tendentious and hyperpartisan news appear.
Mainstream news was more prominent than junk news in the posts related to political parties
and leaders in both periods under study. The most active users of the platform in the Dutch
political Instagram arena are seemingly authentic with little sign artificial manipulation.

171

Within this relatively healthy political space online the only suspicious activity encountered was
on the far right of the political spectrum, where the circulation of junk news takes place and
where those who follow the far-right parties and leaders also follow the junk news sources. Here
there are also indications of artificial manipulation. Certain right-wing leaders (as well as the
personal account of the Prime Minister) show signs of a significant fake follower base. The
artificial activity found is in line with the 2015 fake follower incident when Twitter announced
the deletion of fake followers that affected certain Dutch politicians (and celebrities)
disproportionately.
Reddit and 4chan: YouTube videos as news source contribute to polarisation
Despite their characterisations as alternative spaces on the web, the ‘Dutch’ Reddit and 4chan,
following from our findings, do not appear to spread alternative news sources, but rather refer
more often (even overwhelmingly) to mainstream news sources. Apart from witnessing two
examples of circulating the pro-Russian site, novini.nl, and the activity of one particularly suspect
Reddit account, there does not appear to be any coordinated disinformation or fake news
campaigning. There is certainly junk news to be found but compared to the overall spreading of
sources the proportion is marginal. Particularly Reddit seems to be ‘resistant’ to disinformation.
In 4chan the research did find different types of junk news, especially of the hyperpartisan
variety, particularly in 4chan/pol/.
The research found the presence of junk news, especially of the hyperpartisan variety,
particularly in 4chan/pol/. These are largely links to tendentious and hyperpartisan sources such
as The Post Online and De Dagelijkse Standaard, but also to the conspiracy site, NineForNews. A
right-wing orientation was to be expected in 4chan/pol/, given its reputation as a hub of the
extreme right, but it was perhaps less likely for Reddit, as the ‘Dutch’ Reddit that we took as a
starting point contains a number of politically diverse subreddits. The articles that have gained
the greatest salience concern migration and Islam rather than such geopolitical content as the
Russian involvement in MH17. Whilst it may be problematic to label these sources as ‘fake’, they
could be characterised as polarising.
Nevertheless, the vast majority of the links to news sources are directed at mainstream outlets
such as NOS.nl, both on 4chan/pol as well as Reddit. These findings are counter-intuitive in that
the platforms are often described as alternative, as was said, and the anonymous and
pseudonymous users point to marginal or alternative knowledge sources such as alternatives to
Wikipedia. Our findings dispute such a characterisation, for mainstream sources enjoy some
authority on these platforms, but we did not as of yet research how they are discussed, e.g., as
the starting point for a discussion or ridicule. On Reddit it could be that the mainstream NOS.nl
is considered a reliable source and on 4chan/pol/ “fake news”.
Finally, it important not to regard mainstream and junk news as the only sources of news on the
web. In both the Reddit and 4chan research but also in the YouTube study, we found that
YouTube has emerged as a major news source. That can be said particularly for 4chan/pol/ but
also for Reddit, where “News & Politics” videos are a significant source. On 4chan/pol/ they
may be the most significant, quantitatively leaving other sources well behind. From a small
explorative study of the YouTube channels posted on Reddit we found established sources
172

referenced such as PowNed, Zondag met Lubach and NOS, while on 4chan/pol/ alternative,
foreign and political (hyperpartisan) sources are pointed to, such as Mike Cernovich and Russia
Today. Should such linking and engagement continue, such polarising content could have a
polarising effect in the Dutch political space.
From findings to implications: Mainstream under fire
There is a small, but growing literature concerning how fake news could be considered a moral
panic (Morozov, 2017; Hirst, 2017). The term refers to recurring episodes in history when “rightthinking people” (defined seminally by Stanley Cohen as “editors, bishops and politicians”) spot
a condition that is supposedly prompting a decline in societal standards and values (1972). When
fake news is viewed through that lens, the concern is about how traditional journalism as a pillar
or ‘4th estate’ of democracy is being hollowed out by social media and replaced by low-quality
clickbait as well as openly ideological commentary, both formatted in manners that drives their
consumption not so unlike sugary junk food. The overall health of media as social fabric is said
to be at stake, for citizens using social media as source for political information are
disadvantaged in their capacity to form judgements about social issues and politics more broadly
(Carlson, 2018). There is a second set of literature describing how the media coverage of fake or
junk news, and especially its relationship to the growth of a right-wing media ecology, gives it
‘oxygen’ (Phillips, 2018). More poignantly, it has been argued that journalistic coverage should
turn its attention to the victims, rather than to the fascinating subcultural milieu online where the
far right cultivates itself. There are also cases of politicians’ forwarding extremist and divisive
content, which also gives it oxygen in the sense that it contributes to its spread and perhaps to its
normalcy. Along all these lines, the recommendations concern identifying and acting upon
threats to the mainstream, be they from social media platforms or from within the professions
and practices of journalism, online content creation and political leadership.
As we have found there are particular platforms and subject matters where the threats to the
mainstream appear more acute. Whilst not a space where Dutch junk news sources are spread on
a massive scale, the Dutch 4chan is an incubator of extremist sentiment, especially with respect
to anti-semitism and anti-immigration. Other platforms are problematic for different reasons.
Dutch political spaces in Facebook and Twitter, demarcated through politician, party and issue
queries, have the largest quantities of junk news that is engaged with, though they are still smaller
than mainstream news consumption overall in those same spaces. Among the junk news,
hyperpartisan sources (rather than disinformation or conspiracy) are amongst the more popular,
and for divisive subject matters such as climate change, MH17 and Zwarte Piet their stories
occasionally outperform those in the mainstream press. On Twitter during the European
parliamentary election campaign period, for example, a pro-Nexit story in the hyperpartisan
newspaper, De Dagelijkse Standaard, about the Netherlands leaving the EU outperformed a
counterpart article in the mainstream NRC Handelsblad. A more general polarised media ecology
is also in evidence. On YouTube an alternative (right-wing) media sphere has formed, where
extreme YouTubers, or micro-celebrities, hold sway. Instagram also has a right-wing media
space, analytically detected through shared followers of right-wing politicians and hyperpartisan
media organisations. These are largely ‘alt lite’, meaning anti-establishment and anti-political
correctness, with content that also could be considered anti-Islam. There are no discernable leftwing equivalents. Rather, these spaces compete with more mainstream ones.
173

In contrast to the situation in other countries during the European Parliamentary elections, in
our study we did not find foreign disinformation but rather so-called junk news, especially
around particular issues, such as Zwarte Piet, MH17, climate and the European Union (Peel,
2019). We also found it around the topic of ‘fake news’, studied in this instance as a social issue.
Although decent quantities of junk news were in evidence, mainstream news largely
outperformed it. The largest quantities of junk news circulated not so much around political
parties and leaders (with some exceptions), but around specific polarising issues. Junk news
activity around these issues sometimes appeared during the election periods, but for other issues
there was year-round activity, even for such seasonal issues as Zwarte Piet. Thus, the question is
not only whether there is junk news around election time, but also more generally when it
manifests itself, and with which intensity and duration.
The following policy implications of our work are directed specifically at the phenomenon of
junk news, rather than at foreign disinformation and fake news from organisations feigning to be
news organisations or fake advocacy groups, of which we found none, at least in the top or most
engaged-with content related to Dutch politics across the web and social media platforms. Our
recommendations concern the recognition and monitoring of the polarisation of the media
landscape, the devitalisation or disengagement with extreme content, a national conversation
about issues that appear frequently in junk news (such as Zwarte Piet) rather than one about
disinformation or fake news generally, training for professionals that produce online content,
and enabling access to the (increasingly inaccessible) data on social media platforms for research
and media monitoring.

Policy themes in brief
1) The monitoring of the polarisation of the media landscape, and the mainstreaming of
polarising media with extreme content on social media platforms.
Social media platforms rely on software, their users as well as content reviewers to detect
extreme content. More and more of it is subsequently removed. But historically the attention
paid by social media companies to extreme content has been uneven, and definitions unstable. It
thereby remains desirable to institute independent monitoring. Such work could be taken up by
academic researchers, non-governmental organisations, governmental agencies specialised in
extremism and polarisation as well as media watchdogs.

2) Media training for professional content makers – from journalists to digital media
producers – concerning online source criticism as well as amplification or ‘oxygen-giving’
of extreme speech actors in society.
The Netherlands has existing media literacy training programs, designed for example for senior
citizens as well as primary and secondary school students. This recommendation is made
specifically for professional content-makers such as journalists and editors. It could be made a
part of existing or new media literacy programs dedicated to online source criticism and dealing
174

with polarising content (see also point three below). Such a training program is also of use to
lecturers in higher education, policymakers and civil society.
3) No oxygen-giving to extreme actors and their (online) content.
In our study we found that tendentious news stories circulate well during the election campaign
periods and beyond. The articles are shared and liked by troll-like users but also by regular news
consumers, which we found for example on Twitter during the Provincial elections. In the same
spaces we also found users sharing and liking discriminatory, anti-Semitic, misogynist and
xenophobic content, albeit it to a lesser degree.
It is important not to equate tendentious and extremist media, even when they appear to share
standpoints without using the same words. Similarly, that tendentious media is on the rise and
mainstreaming does not mean that similar weight should be given to extreme media, particularly
right-wing extremist media. The recommendation is that no oxygen should be given to extreme
media sources and their content, meaning no sharing, liking, reacting, commenting, retweeting or
YouTube-debating. Any form of engagement with such content increases the attention and the
metrics and contributes to its spread, ranking and normalisation. Such a recommendation goes
for public broadcasting and commercial media organisations, but also for the tendentious media.
Instead of journalists’ writing about far-right subcultures, attention could be spent on their
victims (Philips, 2018).
4) Recognition of polarising issues such as Zwarte Piet and the facilitation of national
conversations.
The research found that attention to polarising issues such as Zwarte Piet is year-round rather
than seasonal. Such recognition of increasing polarisation in society should lead to discussions
about how common ground may be found. The Netherlands has a tradition of collective
discussion concerning major societal issues through such mechanisms as the Brede
Maatschappelijke Debat (society-wide debate) and interactive policy making. There are other
contemporary forms of citizen participation and discussion that could be instrumental in dealing
with polarising issues and cultural contestation. Institutions experienced in organising societal
discussion and debate should be called upon and supported to do so, and bottom-up initiatives
should be facilitated.
5) Advocacy for social media data access for researchers, journalists and watchdogs, and
creation of research archives of deleted content.
The current issue of ‘locked platforms’ concerns the extent to which social media companies are
making their data inaccessible to researchers, journalists and non-governmental organisations. As
an answer to governmental concern about ‘dark political posts’ (political ads directed only at a
segment of users in their newsfeeds) and other political ads without clear provenance, Facebook
has launched a political ad archive tool and API. But at the same time Facebook has removed in
part or in whole access to services such as the Pages API and Graph Search, which had been in
widespread use by researchers. Social media companies should take up the task of making
175

available the data that researchers, journalists and non-governmental organisations would like to
use for the purposes of research, monitoring and archiving. Governmental agencies, in
consultation with the users and use types mentioned above, have a facilitative as well as a
regulatory role to play here.

176

References
Abbruzzese, Jason (2018) ‘Facebook became your news diet. Now, it’s going to serve you junk’,
Mashable, 17 January.
Abrams, Sam (2016) ‘Professors Moved Left Since 1990s, Rest of Country Did Not’, Heterodox
Academy, January 9. https://heterodoxacademy.org/2016/01/09/professors-moved-left-butcountry-did-not/
Access Now, Civil Liberties Union For Europe, and European Digital Rights (2018) Informing the
“disinformation” debate, Report, Published online on October 18.
https://edri.org/files/online_disinformation.pdf
Albright, Jonathan (2017) ‘Itemized Posts and Historical Engagement - 6 Now-Closed FB Pages’,
Data set, Tableau Public, 5 October,
https://public.tableau.com/profile/d1gi#!/vizhome/FB4/TotalReachbyPage
_____ (2016) ‘The #Election2016 Micro-Propaganda Machine’, Medium, 18 November.
Alexander, Julia (2017) ‘Reddit’s New Policy Won’t Affect Some of Its Most Notorious, Hate-Filled
Subreddits’, Polygon (blog). 2 November.
https://www.polygon.com/2017/11/2/16591508/reddit-content-policy-update-subreddit-banthe-donald-kia
Allcott, Hunt and Matthew Gentzkow (2017) ‘Social Media and Fake News in the 2016 Election’,
Journal of Economic Perspectives, 31(2): 211–236. https://doi.org/10.1257/jep.31.2.211
Alvarez, R. Michael, Ines Levin, Alexander H. Trechsel and Kristjan Vassil (2014) ‘Voting Advice
Applications: How Useful and for Whom?’, Journal of Information Technology & Politics, 11(1): 82101, DOI: 10.1080/19331681.2013.873361
Angwin, Julia, Madeleine Varner and Ariana Tobin (2017) ‘Facebook Enabled Advertisers to Reach
“Jew Haters”’, ProPublica, 14 September. https://www.propublica.org/article/facebook-enabledadvertisers-to-reach-jew-haters
Aydemir, Murat (2011) ‘Dutch Homonationalism and Intersectionality’, ARC-GS Lecture.
https://arcgs.uva.nl/videos/video-artikelen/dutch-homonationalism-andintersectionality.html?1556011447190
Bahara, Hassan, Annieke Kranenberg and Dimitri Tokmetzis (2019) ‘Leidt het Algoritme van
Youtube je Naar Extreme Content?’, De Volkskrant, February 11.
https://www.volkskrant.nl/nieuws-achtergrond/leidt-het-algoritme-van-youtube-je-naarextreme-content~bea101e3/
Barbrook, Richard (1998) ‘The Hi-Tech Gift Economy’, First Monday, 3(12).
https://ojphi.org/ojs/index.php/fm/article/view/631
Baumgartner, Jason (2018) ‘Pushshift API (version 1.0)’, API Documentation,
Pushshift. https://pushshift.io/api-parameters/
Beiler, Markus and Johanna Kiesler (2018) ‘“Lügenpresse! Lying press!” Is the Press Lying?’, in Otto,
Kim and Andreas Köhler (eds.) Trust in Media and Journalism, Wiesbaden: Springer, 155-179.
Belk, Russell (2007) ‘Why Not Share Rather Than Own?’, The ANNALS of the American Academy of
Political and Social Science, 611: 126-140.
Bell, Emily and Taylor Owen (2017) ‘The Platform Press’, Report, Columbia University: Tow Center
for Journalism.
Bellingcat Investigation Team (2016) ‘Behind the Dutch Terror Threat Video: The St. Petersburg
“Troll Factory” Connection’, Bellingcat, 3 April.
177

Benkler, Yochai, Robert Faris, Hal Roberts, and Ethan Zuckerman (2017) ‘Study: Breitbart-Led
Right-Wing Media Ecosystem Altered Broader Media Agenda’, Columbia Journalism Review, 3
March.
Benkler, Yochai, Robert Faris and Hal Roberts (2018) Network Propaganda: Manipulation,
Disinformation, and Radicalization in American Politics, Oxford: Oxford University Press.
Berg, Kati Tusinski (2017). ‘Social Media, Hashtag Hijacking, and the Evolution of an Activist
Group Strategy.’ in Lucinda Austin and Yan Jin (eds.), Social media and crisis communication,
London: Routledge, pp. 141-156
Bergstrom, Kelly. 2011. ‘‘Don’t Feed the Troll’: Shutting down Debate about Community
Expectations on Reddit.Com’, First Monday, 16(8). https://doi.org/10.5210/fm.v16i8.3498
Bessi, Alessandro and Emilio Ferrara (2016) ‘Social bots distort the 2016 US presidential election
online discussion’, First Monday, 21(11). https://firstmonday.org/article/view/7090/5653
Bessi, Alessandro, et al. (2016) ‘Users Polarization on Facebook and Youtube’, PLoS ONE, 11(8): 1–
13. https://doi.org/10.1371/journal.pone.0159641
Boczkowski, Pablo (2016) ‘Fake news and the future of journalism’, NiemanLab, December.
https://www.niemanlab.org/2016/12/fake-news-and-the-future-of-journalism/
Boghardt, Thomas (2009) ‘Soviet Bloc Intelligence and Its AIDS Disinformation Campaign’, Studies
in Intelligence, 53(4): 1-24.
Borra, Erik (2013). ‘DMI Tools’, wiki. https://wiki.digitalmethods.net/Dmi/ToolDatabase.
Borra, Erik, & Bernhard Rieder (2014) ‘Programmed method: developing a toolset for capturing and
analyzing tweets’, Aslib Journal of Information Management, 66(3): 262-278.
Borra, Erik, Sabine Niederer, Johannes Preuß, and Esther Weltevrede (2017) ‘Mapping troll-like
practices on Twitter’, in Liliana Bounegru, Jonathan Gray, Tomasso Venturini & Michele Mauri
(Eds.), A Field Guide to ‘Fake News’ and Other Information Disorders: A collection of recipes for those who
love to cook with digital methods, Amsterdam: Public Data Lab, pp. 161-196.
Bos, Menno van den (2018) ‘Nee, Jíj Bent Een Cultuurrelativist! Hoe Academische Begrippen
Strijdwapens Werden’, Vrij Nederland, 12 December. https://www.vn.nl/identiteitspolitiekcultuurmarxisme-intersectionaliteit/.
Bounegru, Liliana, Jonathan Gray, Tommaso Venturini, Michele Mauri (2018) A Field Guide to “Fake
News” and Other Information Disorders: A Collection of Recipes for Those Who Love to Cook with Digital
Methods, Amsterdam: Public Data Lab.
Bowden, Ian (2016) ‘How Google is tackling fake news, and why it should not do it alone,’ Search
Engine Land, 30 November, https://searchengineland.com/google-tackling-fake-news-not-alone264058.
boyd, danah (2017) ‘Did Media Literacy Backfire?’, Points blog post, Data & Society, 5 January,
https://points.datasociety.net/did-media-literacy-backfire-7418c084d88d.
boyd, danah, Scott Golder and Gilad Lotan (2010) ‘Tweet, tweet, retweet: Conversational aspects of
retweeting on Twitter’, in 43rd Hawaii International Conference on System Sciences, Honolulu, HI:
IEEE, January, pp. 1-10, DOI: 10.1109/HICSS.2010.412.
Boyd, Ryan, Alexander Spangher, Adam Fourney, Besmira Nushi, Gireeja Ranade, James
Pennebaker and Eric Horvitz (2018) ‘Characterizing the Internet Research Agency’s Social Media
Operations During the 2016 U.S. Presidential Election Using Linguistic Analyses’, DataCite,
DOI:10.31234/osf.io/ajh2q.

178

Bradshaw, Samantha (2018) ‘Responding to Fake News through regulation and automation’, in Fake
News, Authentic Views, Report, London: Carter- Ruck. https://www.carterruck.com/images/uploads/documents/RESPONDING_TO_FAKE_NEWS.pdf
Bradshaw, Samantha and Phillip N. Howard (2018) ‘Challenging Truth and Trust: A Global
Inventory of Organized Social Media Manipulation’, Computational Propaganda Data Memo,
Oxford: Oxford Internet Institute.
Brattberg, Erik and Tim Mauer (2018) ‘Russian Election Interference: Europe’s Counter to Fake
News and Cyber Attacks’, White Paper, Washington, DC: Carnegie Endowment for
International Peace, May.
Brennen, Bonnie (2017) ‘Making Sense of Lies, Deceptive Propaganda, and Fake News’, Journal of
Media Ethics, 32(3): 179-181.
Broderick, Ryan and Jules Darmanin (2018) ‘The “Yellow Vest” Riots In France Are What Happens
When Facebook Gets Involved With Local News’, Buzzfeed News, 6 December.
Bruns, Axel, Anja Bechmann, Jean Burgess et al. (2018) ‘Facebook shuts the gate after the horse has
bolted, and hurts real research in the process’, Internet Policy Review, 25 April.
Bump, Philip (2017) ‘The investigation goes digital: Did someone point Russia to specific online
targets?’ Washington Post, 12 July.
Burger, Peter (2016) ‘Moslims vernielen kerstmarkt in Litouwen? Nee: rellen in de VS’, De Gestolen
Grootmoeder blog, 14 December. http://www.gestolengrootmoeder.nl/wordpress/moslimsvernielen-kerstmarkt-in-litouwen-nee-rellen-in-de-vs/
Burger, Peter, Soeradj Kanhai, Alexander Pleijter, and Suzan Verberne (2019) ‘The Reach of
Commercially Motivated Junk News on Facebook’. ArXiv:1901.07290 [Cs], 22 January.
http://arxiv.org/abs/1901.07290.
Butac, Maicolengel (2018) ‘The Reuters Institute for the Study of Journalism vs fake news’, BUTAC
blog, 8 February.
Cadwalladr, Carole and Emma Graham-Harrison (2018) ‘Revealed: 50 million Facebook profiles
harvested for Cambridge Analytica in major data breach’, The Guardian, 17 March.
Cadwalladr, Carole (2016) ‘Google, democracy and the truth about internet search’, The Guardian, 4
December.
Canadian Security Intelligence Service (2018) Who said what? The Security Challenges of Modern
Disinformation, World Watch: Expert Notes series publication (No. 2016-12-05), Canada:
Canadian Security Intelligence Service.
Carlson, Matt (2018) ‘Fake news as an informational moral panic: the symbolic deviancy of social
media during the 2016 US presidential election’, Information, Communication & Society, DOI:
10.1080/1369118X.2018.1505934, published online on August 1.
Cary, Stephen G. (1955) Speak Truth to Power: A Quaker Search for an Alternative to Violence,
Philadelphia: American Friends Service Committee.
Chadwick, Andrew (2013) The Hybrid Media System: Politics and Power. Oxford: Oxford University
Press.
Chen, Adrian (2015) ‘The Agency’, The New York Times, 2 June.
Claussen, Victor (2018) ‘Fighting hate speech and fake news. The Network Enforcement Act
(NetzDG) in Germany in the context of European legislation’, Media Laws 3, published online on
October 14.
Cohen, Stanley (1972) Folk Devils and Moral Panics. London: Routledge.
Colbert Report (2005) ‘The WØRD “Truthiness”, Comedy Central, 17 October.
179

Collins, Ben, and John Russell (2018) ‘Russians Used Reddit and Tumblr to Troll the 2016 Election’,
The Daily Beast, 2 March. https://www.thedailybeast.com/russians-used-reddit-and-tumblr-totroll-the-2016-election.
Collins, Keith Sheera Frenkel (2018) ‘Can you spot the deceptive Facebook post?’ New York Times, 4
September.
Confessore, Nicholas and Daisuke Wakabayashi (2017)’How Russia Harvested American Rage to
Reshape U.S. Politics’, New York Times, 9 October.
Covington, Paul, Jay Adams, and Emre Sargin (2016) ‘Deep Neural Networks for YouTube
Recommendations’, Proceedings of the 10th ACM Conference on Recommender Systems (RecSys '16) New
York, NY: ACM, pp.191-198. DOI: https://doi.org/10.1145/2959100.2959190
Dahlgren, Kristen and Daniel Arkin (2017) ‘11-Year-Old Texas Boy Invents Device to Prevent Hot
Car Deaths’, NBC News, 29 June.
Dan, Ovidiu and Brian D. Davison (2016) ‘Measuring and Predicting Search Engine Users’
Satisfaction’, ACM Computing Surveys, 49(1), art. 18.
Daniels, Jessie (2018) ‘The algorithmic rise of the alt-right’, Contexts, 17(1): 60-65.
Darnton, Robert (2010) Poetry and the Police: Communication Networks in Eighteenth-Century Paris.
Cambridge, MA: Harvard University Press.
_____ (2017) ‘The True History of Fake News’, New York Review of Books, 13 February.
Day, Amber and Ethan Thompson (2012) ‘Live from New York, it’s the fake news! Saturday night
live and the (non)politics of parody,’ Popular Communication 10(1-2): 170-182.
DeAmicis, Carmel (2014) ‘Facebook shifts its algorithm to fight clickbait. Will it kill off Upworthy
and Buzzfeed?’, GIGAOM, Austin: Giga Omni Media.
De Hoax-Wijzer (n.d.) ‘Valse Nieuwssites’. https://sites.google.com/site/dehoaxwijzer/valsenieuwssites.
Digital Methods Initiative (2015) ‘Instagram scraper’.
https://wiki.digitalmethods.net/Dmi/ToolInstagramScraper
DiResta, Renee, Kris Shaffer, Becky Ruppel, et al. (2018) ‘The Tactics & Tropes of the Internet
Research Agency’, Report, New Knowledge.
https://disinformationreport.blob.core.windows.net/disinformation-report/NewKnowledgeDisinformation-Report-Whitepaper.pdf.
Drewe, Nick (2016) ‘The Hilarious List Of Hashtags Instagram Won’t Let You Search’, The Data
Pack, 10 May. http://thedatapack.com/banned-instagram-hashtags-update/
Drog (2018) ‘Slecht Nieuws: Serious game over propaganda’, Website,
https://www.slechtnieuws.nl/.
Duke Reporters Lab (2019) ‘Fact Checking News’, Website. https://reporterslab.org/factchecking/
Dumont, P. and Kies, R. (2012) ‘Smartvote.lu: Usage and impact of the first VAA in Luxembourg’,
International Journal of Electronic Governance 5(3/4): 388–410.
Dupuy, Lisa (2018) ‘GeenStijl Staat Niet Meer Op EU-Lijst Nepnieuws’, NRC Handelsblad, 29
January. https://www.nrc.nl/nieuws/2018/01/29/geenstijl-staat-niet-meer-op-eu-lijstnepnieuws-a1590216.
Edelman (2018) ‘2018 Edelman Trust Barometer’, Report.
https://www.edelman.com/sites/g/files/aatuss191/files/201810/Edelman_Trust_Barometer_Employee_Experience_2018_0.pdf
Ellick, Adam B. and Adam Westbrook (2018) ‘Operation Infektion’, New York Times, 12 November.
180

EU HLEG (2018) A multi-dimensional approach to disinformation, Report of the independent High level
Group on fake news and online disinformation, Luxembourg: Publications Office of the
European Union.
EU vs Disinfo. 2017. ‘Sputnik’s Short-Lived Presence in the Slovak Press Agency’. 2017. EU vs
Disinfo (blog). 1 April. https://euvsdisinfo.eu/sputniks-short-lived-presence-in-the-slovak-pressagency/.
Fandos, Nicholas (2017) ‘White House Pushes ‘Alternative Facts.’ Here Are the Real Ones’, New
York Times, 22 January.
Faris, Robert M., Hal Roberts, Bruce Etling, Nikki Bourassa, Ethan Zuckerman, and Yochai Benkler
(2017) ‘Partisanship, Propaganda, and Disinformation: Online Media and the 2016 U.S.
Presidential Election’, Berkman Klein Center for Internet & Society Research Paper.
Farkas, Johan and Marco Bastos (2018) ‘IRA Propaganda on Twitter: Stoking Antagonism and
Tweeting Local News’, SMSociety ‘18, Copenhagen.
Feldman, Brian (2017) ‘In Russia, You Can Buy Instagram Likes From a Vending Machine’, New
York Magazine, 8 June.
Feuz, Martin, Matthew Fuller, and Felix Stalder (2011) ‘Personal Web Searching in the Age of
Semantic Capitalism: Diagnosing the Mechanisms of Personalisation’, First Monday, 16(2), 7
February. https://doi.org/10.5210/fm.v16i2.3344.
Fink, Christina (2018) ‘Dangerous Speech, Anti-Muslim Violence, and Facebook in Myanmar’,
Journal of International Affairs 71(1.5): 43–52.
Finkel, Jacob, Steven Jiang, Luo Mufan, Rebecca Mears, Danaë Metaxa-Kakavouli, Camille Peeples,
Brendan Sasso, Arjun Shenoy, Vincent Sheu, and Nicolás Torres-Echeverry (2017) Fake News
and Misinformation: The Roles of the Nation’s Digital Newsstands, Facebook, Google, Twitter and Reddit,
Stanford, CA: Stanford Law School. https://www-cdn.law.stanford.edu/wpcontent/uploads/2017/10/Fake-News-Misinformation-FINAL-PDF.pdf
FireEye (2018) ‘Suspected Iranian Influence Operation’, Report, Milpitas, CA: FireEye.
Fitts, Alexis Sobel (2017) ‘Welcome to the Wikipedia of the Alt-right’, Wired, 21 June.
Fletcher, Richard, Alessio Cornia, Lucas Graves, and Rasmus Kleis Nielsen (2018) ‘Measuring the
reach of “fake news” and online disinformation in Europe’, Reuters Institute for the Study of
Journalism, Oxford University, February.
Flynn, Kerry (2018) ‘Facebook will elevate “trusted” news outlets after surveying U.S. users’,
Mashable, 19 January.
Foucault, Michel (2008) The Birth of Biopolitics: Lectures at the Collège De France, 1978-1979, New York,
NY: Palgrave Macmillan.
Frenkel, Sheera, Kate Conger and Kevin Roose (2019) ‘Russia’s Playbook for Social Media
Disinformation Has Gone Global’, New York Times, 31 January.
Frenkel, Sheera, Nicholas Casey and Paul Mozur (2018) ‘In Some Countries, Facebook’s Fiddling
Has Magnified Fake News’, New York Times, 14 January.
Fubini, Federico (2018) ‘Tweet populisti dalla Russia sulla politica italiana. Come negli Usa’, Corriere
della Sera, 1 August.
Gartenberg, Chaim (2018) ‘Facebook is going to start surveying users to determine trustworthy news
sources’, The Verge, 19 January.
Garzia, Diego and Marschall, Stefan (2012) ‘Voting Advice Applications Under Review: The State of
Research’, International Journal of Electronic Governance, 5: 203-22.

181

German Federal Foreign Office (2018) ‘Fake news, bots and provocative statements –
disinformation on the internet’, Auswaertiges Amt website, published online on August 7.
Gessen, Masha (2018) ‘Why the Russian Influence Campaign Remains So Hard to Understand’, The
New Yorker, 18 December.
Gitelman, Lisa (2006) Always Already New: Media, History, and the Data of Culture. Cambridge, MA:
MIT Press.
Glowacki, Monika, Vidya Narayanan, Sam Maynard, Gustavo Hirsch, Bence Kollanyi, Lisa-Maria
Neudert, Phil Howard, Thomas Lederer, and Vlad Barash (2018) ‘News and Political
Information Consumption in Mexico: Mapping the 2018 Mexical Presidential Election on
Twitter and Facebook,’ 2018.2, COMPROP Data Memo. Oxford: Oxford Internet Institute.
Goodwin, Matthew and Roger Eatwell (2018) National Populism: The Revolt Against Liberal Democracy,
London: Penguin Books.
Gottfried, Jeffrey and Elisa Shearer (2016) ‘News Use Across Social Media Platforms 2016’,
Washington, DC: Pew Research Center.
Government Offices of Sweden (2017) A practical approach on how to cope with disinformation,
Government Offices of Sweden website, published online on October 6.
Gramer, Robbie (2017) ‘Denmark Creates the World’s First Ever Digital Ambassador’, Foreign Policy,
published online on January 27.
Graves, Lucas (2016) Deciding What’s True: The Rise of Political Fact-Checking in American Journalism. New
York: Columbia University Press.
Graves, Lucas and Federica Cherubini (2016) The Rise of Fact-checking Sites in Europe. Oxford:
University of Oxford, Reuters Institute for the Study of Journalism.
Grinberg, Nir, Kenneth Joseph, Lisa Friedland, Briony Swire-Thompson and David Lazer (2019)
‘Fake news on Twitter during the 2016 U.S. presidential election’, Science 363(6425): 374-378.
Groll, Elias (2018) ‘Battling the Bots’, Foreign Policy, 12 November.
Groot, Tim, Sophie Minihold, Jessica Robinson, Manuel Schneider, Joanna Sleigh and Dydimus
Zengenene (2019). ‘Russia, Twitter & Authenticity: Establishing Credibility Metrics’, Digital
Methods Initiative, Winter School 2019.
https://wiki.digitalmethods.net/Dmi/WinterSchool2019CredibilityMetrics
Guess, Andrew, Brendan Nyhan and Jason Reifler (2018) ‘Selective Exposure to Misinformation:
Evidence from the consumption of fake news during the 2016 U.S. presidential campaign’,
Report, Hannover, NH: Dartmouth College, https://www.dartmouth.edu/~nyhan/fake-news2016.pdf.
Haciyakupoglu, Gulizar, Jennifer Yang Hui, V. S. Suguna, Dymples Leong, and Muhammad Faizal
Bin Abdul Rahman (2018) Countering Fake News: A Survey Of Recent Global Initiatives, Singapore:
Nanyang Technological University.
Hagen, Sal, Daniël de Zeeuw, Stijn Peeters, Emilija Jokubauskaitė, and Ángeles Briones (2019).
‘Understanding Normiefication: A Cross-Platform Analysis of the QAnon Conspiracy Theory’,
Digital Methods Initiative, 21 February.
https://wiki.digitalmethods.net/Dmi/WinterSchool2019Normiefication.
Haidt, Jonathan (2012) The Righteous Mind: Why Good People Are Divided by Politics and Religion. New
York: Pantheon Books.
Haim, Mario, Andreas Graefe and Hans-Bernd Brosius (2018) ‘Burst of the Filter Bubble? Effects of
personalization on the diversity of Google News’, Digital Journalism, 6(3):1-14.

182

Halavais, Alexander M. Campbell (2017) Search Engine Society, Second edition, Cambridge, UK: Polity
Press.
Hall, Jim (2001) Online journalism: a critical primer. London: Pluto Press.
Hameleers, Michael and Toni van der Meer (2019) ‘Fact-checks helpen tegen desinformatie! Maar
dan moeten ze wel gezien worden’, De Nieuwe Reporter, 29 January.
Hankes, Keegan, and Alex Amend (2018) ‘The Alt-Right Is Killing People’, Southern Poverty Law
Center, 5 February. https://www.splcenter.org/20180205/alt-right-killing-people.
Haque, Md Mahfuzul, Mohammad Yousuf, Zahedur Arman, Md Main Uddin Rony, Ahmed Shatil
Alam, Kazi Mehedi Hasan, Md Khadimul Islam, and Naeemul Hassan (2018) ‘Fact-Checking
Initiatives in Bangladesh, India, and Nepal: A Study of User Engagement and Challenges’,
ArXiv:1811.01806 [Cs], November. http://arxiv.org/abs/1811.01806.
Harsin, Jayson (2018) ‘A Critical Guide to Fake News: From Comedy to Tragedy’, Pouvoirs,
2018/1:164, 99-119.
Harvey, Del and Yoel Roth (2018) ‘An update on our elections integrity work’, Twitter blog, 1
October.
Hatewatch Staff (2015) ‘Getting Cucky: A Brief Primer On The Radical Right's Newest
'cuckservative' Meme’, Southern Poverty Law Center, 7 August.
https://www.splcenter.org/hatewatch/2015/08/07/getting-cucky-brief-primer-radical-rightsnewest-cuckservative-meme
Hawley, George (2017) Making Sense of the Alt-Right, New York: Colombia University Press.
Haynes, Gavin (2016) ‘Can you spot the ‘real’ fake news story?’ The Guardian, 28 December.
Heck, Wilmer (2018) ‘Nederlandse media dagen EU voor rechter na beschuldigingen desinformatie’,
NRC Handelsblad, 20 February.
Heck, Wilmer (2017) ‘Rusland Beïnvloedt Ons Vooral Online’, NRC Handelsblad, 8 January.
Hedman, Freja, Fabian Sivnert, Lisa-Maria Neudert, Bence Kollanyi, Philip N. Howard and Vidya
Narayanan (2018) ‘News and Political Information Consumption in Sweden: Mapping the 2018
Swedish General Election on Twitter’, Computational Propaganda Data Memo, Oxford: Oxford
Internet Institute, 6 September.
Hedrick, Ashley, Dave Karpf and Daniel Kreiss (2018) ‘The Earnest Internet vs. the Ambivalent
Internet’, International Journal of Communication, 12:1057-1064.
Heikkilä, Niko (2017) ‘Online Antagonism of the Alt-Right in the 2016 Election,’ European Journal of
American Studies 12(2): 1-23, DOI:10.4000/ejas.12140.
Helmond, Anne (2015) ‘The Platformization of the Web: Making Web Data Platform Ready’, Social
Media + Society 1(2):1-11. https://doi.org/10.1177/2056305115603080.
Hemmer, Nicole (2016) Messengers of the Right: Conservative Media and the Transformation of American
Politics, Philadelphia: University of Pennsylvania Press.
Hendricks, Vincent F. and Mads Vestergaard (2019) Reality Lost: Markets of Attention, Misinformation
and Manipulation, Cham: Springer.
Herderscheê, Gijs, and Remco Meijer (2019) ‘Kabinet krijgt bitter voorproefje van regeren zonder
meerderheid in Eerste Kamer’, De Volkskrant. January 31. https://www.volkskrant.nl/gsb378d420.
Hern, Alex (2017) ‘Google acts against fake news on search engine’, The Guardian, 25 April.

183

Hern, Alex and Jim Waterson (2018) ‘Facebook cracks down on ‘dark ads’ by British political
groups’, The Guardian, 16 October.
Herrman, John (2016) ‘Inside Facebook’s (Totally Insane, Unintentionally Gigantic, Hyperpartisan)
Political-Media Machine’, The New York Times, 28 August.
https://www.nytimes.com/2016/08/28/magazine/inside-facebooks-totally-insaneunintentionally-gigantic-hyperpartisan-political-media-machine.html.
Hindman, Matthew Scott (2008) The Myth of Digital Democracy, Princeton: Princeton University Press.
Hirst, Martin (2017) ‘Towards a political economy of fake news’, The Political Economy of Communication
5(2), 82–94.
Hirzalla, Fadi and Liesbet van Zoonen (2015) ‘Online voting applications. Foci, findings and future
of an emerging research field’ in Coleman, S. & Freelon, D. (eds), Handbook of Digital Politics,
Northampton, MA: Edward Elgar: 87-103.
Holt, Jared (2017) ‘White Supremacy Figured Out How to Become YouTube Famous | Right Wing
Watch,’ October. http://www.rightwingwatch.org/report/white-supremacy-figured-out-how-tobecome-youtube-famous/.
Holzman, Michael (2008) James Jesus Angleton, the CIA and the Craft of Counterintelligence, Amherst:
University of Massachusetts Press.
Honeycutt, Courtenay and Susan C. Herring (2009) ‘Beyond microblogging. Conversation and
collaboration’, 42nd Hawaii International Conference on System Sciences. Los Alamitos, CA: IEEE
Press.
House of Commons (2019) ‘Disinformation and “Fake News”: Final Report Eighth Report of
Session 2017–19’, Digital, Culture, Media and Sport Committee, London: UK Parliament.
Howard, Philip N., Gillian Bolsover, Bence Kollanyi, Samantha Bradshaw, and Lisa-Maria Neudert
(2017) ‘Junk news and bots during the U.S. election: What were Michigan voters sharing over
Twitter?’ Computational Propaganda Data Memo, Oxford: Oxford Internet Institute.
Howard, Philip N., Bharath Ganesh, Dimitra Liotsiou, John Kelly and Camille François (2018) ‘The
IRA, Social Media and Political Polarization in the United States, 2012-2018’, Report,
Computational Propaganda Research Project, Oxford: Oxford Internet Institute.
Hughes, Taylor, Jeff Smith, and Alex Leavitt (2018) ‘Helping People Better Assess the Stories They
See in News Feed with the Context Button | Facebook Newsroom’, Facebook Newsroom (blog).
April. https://newsroom.fb.com/news/2018/04/news-feed-fyi-more-context/
Instaloader (2019). Instaloader software, version 4.2.5, GitHub project,
https://instaloader.github.io/
Isaac, Mike (2016) ‘Facebook, in Cross Hairs After Election, Is Said to Question Its Influence’, New
York Times, 12 November.
Jack, Caroline (2017) Lexicon of Lies: Terms for Problematic Information New York: Data & Society
Research Institute.
Jamieson, Kathleen Hall. 2018. Cyberwar: How Russian Hackers and Trolls Helped Elect a
President. Oxford: Oxford University Press.
Jankowski, Nicholas W. (2018) ‘Researching Fake News: A Selective Examination of Empirical
Studies’, Javnost - The Public, 25(1-2): 248-255.

184

Jansen, Bernard J. and Amanda Spink (2003) ‘An Analysis of Web Documents Retrieved and
Viewed’, 4th International Conference on Internet Computing, Las Vegas, Nevada, 23-26 June,
pp. 65-69.
Java, Akshay, Xiaodan Song, Tim Finin and Belle Tseng (2007) ‘Why we twitter: understanding
microblogging usage and communities’. In Proceedings of the 9th WebKDD and 1st SNA-KDD 2007
workshop on Web mining and social network analysis, New York, NY: ACM, pp. 56-65.
Jeangène Vilmer, Jean-Baptiste, Alexandre Escorcia, Marine Guillaume, Janaina Herrera (2018)
Information Manipulation: A Challenge for Our Democracies, report by the Policy Planning Staff (CAPS)
of the Ministry for Europe and Foreign Affairs and the Institute for Strategic Research (IRSEM)
of the Ministry for the Armed Forces, Paris, August.
Johnson, Brett G., and Kimberly Kelling (2018) ‘Placing Facebook’ Journalism Practice 12(7): 817–33.
https://doi.org/10.1080/17512786.2017.1349546
Jonker, Jorn (2017) ‘Nepnieuws gevaar voor Nederland’, De Telegraaf, 14 November.
Kaiser, Jonas, and Adrian Rauchfleisch (2018) ‘Unite the Right? How YouTube’s Recommendation
Algorithm Connects the U.S. Far-Right.’ Data & Society Media Manipulation, April 11.
doi:10.1093/acprof:oso/9780199641260.001.0001/acprof-9780199641260.
Kanne, Peter and Milan Driessen (2017) Desinformatie leidt tot verwarring bij nieuwsconsument, Amsterdam:
I&O Research.
Kasteleijn, Nando (2017) ‘Nepnieuws groot gevaar voor Nederland? Dat lijkt dus wel mee te vallen’,
NOS, 28 August.
van Keulen, Ira, Iris Korthagen, Paul Diederen en Pieter van Boheemen (2018) ‘Digitalisering van
het nieuws: Online nieuwsgedrag, desinformatie en personalisatie in Nederland’, Den Haag:
Rathenau Instituut.
King, Gary and Nathaniel Persily (2018) ‘A New Model for Industry-Academic Partnerships’,
Working Paper, 9 April. http://j.mp/2q1IQpH
Kist, Reinier (2019) ‘Factchecken Facebook loopt stuk op aansprakelijkheid’, NRC Handelsblad, 26
February.
Kist, Reinier and Rik Wassens (2017) ‘Ook Nederlandse media werden misleid door Russische
trollen’, NRC Handelsblad, 8 December.
Kist, Reinier and Rik Wassens (2018a) ‘Russische trollen actief in Nederland’, NRC Handelsblad, 15
July.
Kist, Reinier and Rik Wassens (2018b) ‘Russisch trollenleger ook actief in Nederland’, NRC
Handelsblad, 15 July.
Kist, Reinier and Peter Zantingh (2017) ‘Geen grote rol nepnieuws in aanloop naar verkiezingen’,
NRC Handelsblad, 6 March. https://www.nrc.nl/nieuws/2017/03/06/fake-news-nee-zo-erg-ishet-hier-niet-7144615-a1549050
Kitta, Andrea (2018) ‘Alternative Websites and Fake News: Taking a Stab at Definition, Genre and
Belief’, The Journal of American Folklore 131(522): 405-412.
Komok, Anna (2018) ‘How to Check Instagram Account for Fake Followers’, HypeAuditor, 6 July.
https://hypeauditor.com/blog/how-to-check-instagram-account-for-fake-followers/
Kouwenhoven, Andreas and Hugo Logtenberg (2017) ‘Hoe Denk met ‘trollen’ politieke
tegenstanders monddood probeert te maken’, NRC Handelsblad, 10 February.
Kranenberg, Annieke (2017) Wie weet nog wat er waar is?, De Volkskrant, 23 December.

185

Kranenberg, Annieke, and Hassan Bahara (2018) ‘Hoe alt-right online Jodenhaat verspreidt’, De
Volkskrant, 9 November. https://www.volkskrant.nl/gs-b1714693
Kwak, Haewoon, Changhyun Lee, Hosung Park and Sue Moon (2010). ‘What is Twitter? A social
network or a news media?’, in Proceedings of the 19th International Conference on World Wide Web, New
York: ACM, April, pp. 591-600.
Lagorio-Chafkin, Christine (2018) ‘Reddit Confirms New Russian Meddling Efforts’, Inc., 4 October.
https://www.inc.com/christine-lagorio/reddit-finds-new-russian-interference-campaign.html.
Laquintano, Timothy and Annette Vee (2017) ‘How Automated Writing Systems Affect the
Circulation of Political Information Online’, Literacy in Composition Studies, 5(2): 43-62.
Latour, Bruno (2008) What is the style of matters of concern? Assen: Van Gorcum.
Latour, Bruno, and Steve Woolgar (1979) Laboratory Life: The Social Construction of Scientific Facts.
Princeton, NJ: Princeton University Press.
Lazarsfeld, Paul F., Bernard R. Berelson and Hazel Gaudet (1948) The People’s Choice: How the Voter
Makes Up His Mind in a Presidential Campaign, New York, NY: Columbia University Press.
Lazer, David M.J., Matthew A. Baum, Yochai Benkler, Adam J. Berinsky, Kelly M. Greenhill, Filippo
Menczer, Miriam J. Metzger, Brendan Nyhan, Gordon Pennycook, David Rothschild, Michael
Schudson, Steven A. Sloman, Cass R. Sunstein, Emily A. Thorson, Duncan J. Watts and
Jonathan L. Zittrain (2018) ‘The science of fake news’, Science, 359(6380):1094-1096.
Lee, Alexis (2019) ‘What Is a ‘Facebook Engagement’?’, BuzzSumo Knowledge Base.
https://help.buzzsumo.com/faqs-and-troubleshooting/product-faqs/what-is-a-facebookengagement.
Lewis, Rebecca (2018) ‘Alternative Influence: Broadcasting the Reactionary Right on YouTube’, Data
& Society. https://datasociety.net/wp-content/uploads/2018/09/DS_Alternative_Influence.pdf.
Liotsiou, Dimitra, Bence Kollanyi, and Philip N. Howard (2019) ‘The Junk News Aggregator:
Examining Junk News Posted on Facebook, Starting with the 2018 US Midterm Elections’,
ArXiv:1901.07920 [Cs], January. http://arxiv.org/abs/1901.07920.
Lippmann, Walter (1922) Public Opinion, New York, NY: Harcourt, Brace and Co.
Lomas, Natasha (2018) ‘Google is surfacing Texas shooter misinformation in search results —
thanks also to Twitter’, TechCrunch, 11 June.
Lorenz, Taylor (2014) ‘Instagram Rapture Claims Millions Of Celebrity Instagram Followers’,
Business Insider, 18 December.
Lorenz, Taylor (2018) ‘Instagram’s Christmas Crackdown. No meme account is safe—not even
@God’, The Atlantic, 27 December.
Louwerse, Tom and Martin Rosema (2014) ‘The design effects of voting applications: Comparing
methods of calculating matches’, Acta Politica 49(3):286-312.
Lüders, Marika, Lin Prøitz and Terje Rasmussen (2010) ‘Emerging personal media genres’,
New Media & Society, 12(6): 947–963.
Lynskey, Dorian (2018) ‘How Dangerous Is Jordan B Peterson, the Rightwing Professor Who 'Hit a
Hornets' Nest'?’ The Guardian, February 7.
https://www.theguardian.com/science/2018/feb/07/how-dangerous-is-jordan-b-peterson-therightwing-professor-who-hit-a-hornets-nest.
MacFarquhar, Neil (2018) ‘A Powerful Russian Weapon: The Spread of False Stories’, The New York
Times, 20 January. https://www.nytimes.com/2016/08/29/world/europe/russia-swedendisinformation.html.

186

Maheshwari, Sapna (2018) ‘Uncovering Instagram Bots With a New Kind of Detective Work’,
The New York Times, 12 March.
Malice, Michael (2019) The New Right: A Journey to the Fringe of American Politics, New York, NY: All
Points Books.
Marchal, Nahema, Lisa-Maria Neudert, Bence Kollanyi, and Philip N. Howard (2018) ‘Polarization,
Partisanship and Junk News Consumption on Social Media During the 2018 US Midterm
Elections’ 2018.5, COMPROP Data Memo, Oxford: Oxford Internet Institute.
Margolis, Joseph (1995) ‘Beyond postmodernism: Logic as rhetoric’, Argumentation, 9(1): 21-31.
Marres, Noortje (2018) ‘Why We Can’t Have Our Facts Back’, Engaging Science, Technology, and Society,
4: 423-443.
Marres, Noortje, and Esther Weltevrede (2013), ‘Scraping the Social?’ Journal of Cultural Economy, 6(3):
313–35, DOI:10.1080/17530350.2013.772070.
Martens, Bertin, Luis Aguiar, Estrella Gomez-Herrera and Frank Mueller-Langer (2018) ‘The Digital
Transformation of News Media and the Rise of Disinformation and Fake News,’ SSRN Electronic
Journal, DOI:10.2139/ssrn.3164170.
Marvin, Carolyn (1988) When Old Technologies were New. New York, NY: Oxford University Press.
Marwick, Alice E. (2018) ‘Why Do People Share Fake News? A Sociotechnical Model of Media
Effects’, Georgetown Law Technology Review, 2(2): 474-512.
Marwick, Alice, and Rebecca Lewis (2017) ‘Media Manipulation and Disinformation Online’, Data &
Society, New York.
Matsakis, Louise (2017) 'This Russian Vending Machine Will Sell You Fake Instagram Likes',
Motherboard Vice, 7 June.
Mauri, Michele, Tommaso Elli, Giorgio Caviglia, Giorgio Uboldi, and Matteo Azzi (2017)
‘RAWGraphs: A Visualisation Platform to Create Open Outputs’, in Proceedings of the 12th
Biannual Conference on Italian SIGCHI Chapter, 28:1–28:5. CHItaly ’17, New York, NY: ACM.
https://doi.org/10.1145/3125571.3125585.
McGarry, Caitlin (2013) ‘Twitter bots, fake retweets rake in big bucks’, PCWorld, 10 April.
McGonagle, Tarlach (2017) ‘“Fake news”: False fears or real concerns?’, Netherlands Quarterly of
Human Rights, 35(4): 203–209.
McKernon, Edward (1925) ‘Fake News and the Public’, The Harper’s Monthly, October, pp. 528-536.
McKernon, Edward (1928) ‘News Fakers’, The Outlook, 149(4): 130-141.
McNeil, Maureen (2013) ‘Between a Rock and a Hard Place: The Deficit Model, the Diffusion
Model and Publics in STS’, Science as Culture, 22(4): 589-608.
McQueen, Sharon (2018) ‘From Yellow Journalism to Tabloids to Clickbait: The Origins of Fake
News in the United States’, in Agosto, Denise E. (ed.) Information Literacies and Libraries in the Age
of Fake News, Santa Barbara, CA: Libraries Unlimited, pp. 12-35.
Merrill, Jeremy B. and Ariana Tobin (2019) ‘Facebook Moves to Block Ad Transparency Tools —
Including Ours’, ProPublica, 29 January.
Milneil, Christian (2018) ‘Data: Read the tweets from alleged Russian troll accounts targeting Maine
politicians’, Portland Press Herald, 2 August.
Mina, An Xiao (2019). Memes to Movements: How the World’s Most Viral Media Is Changing Social Protest
and Power, Boston, MA: Beacon Press.
Ministerie van Binnenlandse Zaken en Koninkrijksrelaties (2019) ‘Campagne nepnieuws vandaag van
start - Nieuwsbericht’, Rijksoverheid.nl. March 11.

187

https://www.rijksoverheid.nl/actueel/nieuws/2019/03/11/campagne-nepnieuws-vandaag-vanstart.
Misérus, Mark and Robert van der Noordaa (2018a) ‘Het trollenleger van popartiest Dotan’, De
Volkskrant, 14 April.
Misérus, Mark and Robert van der Noordaa (2018b) ‘Dotan erkent aanmaken nepfans: “Ik was heel
naïef, veel te ambitieus en onzeker”’, De Volkskrant, 16 April.
Monbiot, George (2019) ‘Dark money is pushing for a no-deal Brexit. Who is behind it?’, The
Guardian, 13 February.
Morozov, Evgeny (2017) ‘Moral panic over fake news hides the real enemy – the digital giants’, The
Guardian, 8 January.
Mosseri, Adam (2017a) ‘Working to Stop Misinformation and False News | Facebook Newsroom’,
Facebook Newsroom, blog post, April. https://newsroom.fb.com/news/2017/04/working-to-stopmisinformation-and-false-news/.
———. 2017b. “Showing More Informative Links in News Feed | Facebook Newsroom.” Facebook
Newsroom, blog post, June 30. https://newsroom.fb.com/news/2017/06/news-feed-fyi-showingmore-informative-links-in-news-feed/
Mudde, Cas, and Cristóbal Rovira Kaltwasser (2017) Populism: a Very Short Introduction, Oxford:
Oxford University Press.
Muirhead, Russell, and Nancy L. Rosenblum (2019) A Lot of People Are Saying: The New Conspiracism
and the Assault on Democracy, Princeton, NJ: Princeton University Press.
Muller, Jan-Werner (2016) What Is Populism? Philadelphia, PA: University of Pennsylvania Press.
Nagle, Angela (2017) Kill All Normies: Online Culture Wars From 4Chan and Tumblr to Trump and the AltRight. London: Zero Books.
Neudert, Lisa-Maria N. (2017) ‘Computational Propaganda in Germany: A Cautionary Tale’,
Computational Propaganda Working Paper, 2017.7, COMPROP Data Memo, Oxford: Oxford
Internet Institute.
Neudert, Lisa-Maria, Bence Kollanyi, and Philip N. Howard (2017) ‘Junk News and Bots during the
German Parliamentary Election: What Are German Voters Sharing over Twitter?’, 2017.7,
COMPROP Data Memo, Oxford: Oxford Internet Institute.
New Knowledge (2018) ‘The Tactics & Tropes of the Internet Research Agency’, White Paper,
Austin, TX: New Knowledge.
Newman, Michelle C. (2010) ‘The Daily Show and Meta-Coverage: How Mock News Covers the
Political Communications System’, The Elon Journal of Undergraduate Research in Communications,
1(2).
NewsGuard (2019), ‘Restoring Trust and Accountability’, webpage,
https://www.newsguardtech.com.
Niederer, Sabine (2017) ‘The Study of Networked Content: Five Considerations for Digital Research
in the Humanities,’ in Giovanni Schiuma and Daniela Carlucci (eds.), Big Data in the Arts and
Humanities: Theory and Practice, Boca Raton, FL: CRC Press, pp. 89-100.
Nijmeijer, Bert (2018) ‘Zelf nepnieuws maken om daarna de echte trollen te kunnen herkennen’,
NRC Handelsblad, 22 July.

188

Noble, Safiya Umoja (2018) Algorithms of Oppression: How Search Engines Reinforce Racism. New York,
NY: New York University Press.
van der Noorda, Robert van der en Coen van de Ven (2019) ‘Het MH17 Complot’, De Groene
Amsterdammer, 29 May.
van der Noorda, Robert and Coen van de Ven (2018a) ‘Hoe Russische trollen inspelen op westerse
angsten’, De Groene Amsterdammer, 28 Augustus.
van der Noordaa, Robert and Coen van de Ven (2018b) ‘3 Miljoen “trollentweets” onderzocht: hoe
Rusland via sociale media ook in België verdeeldheid zaait’, Knack, 27 November.
NOS (2017a) ‘Baby van 20 kilo heeft de primeur: eerste nepnieuws in Nederland’, NOS, 8 June.
NOS (2017b) ‘Ollongren: Russische desinformatie bij Oekraïne-referendum’, NOS, 15 November.
NOS (2018) ‘Twitters grote schoonmaak: Wilders en Denk-politici verliezen volgers’, NOS, 13 July.
Nuzzi, Olivia (2017) ‘Kellyanne Conway Is a Star’, New York Magazine, March.
Nyhan, Brendan, Ethan Porter, Jason Reifler and Thomas J. Wood (2019) ‘Taking Fact-Checks
Literally But Not Seriously? The Effects of Journalistic Fact-Checking on Factual Beliefs and
Candidate Favorability’, Political Behavior, published online 21 January.
O’Brien, Chris (2019) ‘Sheryl Sandberg says Facebook is now blocking 1 million fake accounts every
day’, Venture Beat, 21 January.
O’Donovan, Caroline (2014) ‘What is clickbait?’, Niemanlab, 25 August.
Omnicore. (2019) ‘Twitter by the numbers’, Omnicore
Agency. https://www.omnicoreagency.com/twitter-statistics/
Opper, F. (1894) ‘The fin de siècle newspaper proprietor’, Puck, 35(887).
Oudenampsen, Merijn (2013) ‘Met de Tjoeki Tjoeki Naar Takki Takki’. De Groene Amsterdammer, 3
July. https://www.groene.nl/artikel/met-de-tjoeki-tjoeki-naar-takki-takki.
Panagopoulos, Costas (2016) ‘All about that base: Changing campaign strategies in US presidential
elections’ Party Politics, 22(2): 179–90.
Pariser, Eli (2011) The Filter Bubble: What the Internet Is Hiding From You. New York, NY: Penguin.
Parlapiano, Alicia and Jasmine C. Lee (2018) ‘The Propaganda Tools Used by Russians to Influence
the 2016 Election’, New York Times, 16 February.
Peck, Reece (2019) Fox Populism: Branding Conservatism as Working Class, Cambridge: Cambridge
University Press.
Pedersen, David B. and Vincent F. Hendricks (2014) ‘Science Bubbles’, Philosophy and Technology,
27(4): 503-518.
Peek, Simone (2018) ‘NOS-Hoofdredacteur Over Bericht Soros: “Zo Had Het Niet Gemoeten”’,
NRC Handelsblad, 24 October.
Peel, Michael (2019) ‘EU election suffered Russian disinformation, Brussels finds’, Financial Times, 14
June.
Persily, Nathaniel (2017) ‘The 2016 U.S. Election: Can Democracy Survive the Internet?’. Journal of
Democracy, 28(2): 63-76
Phillips, Whitney (2015) This Is Why We Can’t Have Nice Things: Mapping the Relationship Between Online
Trolling and Mainstream Culture. Cambridge, MA: MIT Press.
Phillips, Whitney (2018) ‘The Oxygen of Amplification: Better Practices for Reporting on
Extremists, Antagonists and Manipulators Online’, Report, New York: Data & Society Research
Institute.
Phillips, Whitney and Ryan Milner (2018) The Ambivalent Internet: Mischief, Oddity, and Antagonism
Online, Cambridge: Polity Press.
189

Pleijter, Alexander (2017) ‘De nepnieuwslawine zonder nepnieuws’, Villamedia, 17 November.
Pohjonen, Matti and Sahana Udupa (2017) ‘Extreme Speech Online: An Anthropological Critique of
Hate Speech Debates’, International Journal of Communication, 11: 1173-1191.
Posetti, Julie and Alice Matthews (2018) A short guide to the history of ‘fake news’ and disinformation. A
learning module for journalists and journalism educator, Washington, DC: International Center for
Journalists.
Poynter (2019) A guide to anti-misinformation actions around the world, St. Petersburg, FL: The Poynter
Institute.
Puschmann, Cornelius (2018) ‘Beyond the Bubble: Assessing the Diversity of Political Search
Results’ Digital Journalism, published online 28 November.
https://doi.org/10.1080/21670811.2018.1539626
Puschmann, Cornelius (2017) ‘How significant is algorithmic personalization in searches for political
parties and candidates?’, Algorithmed Public Spheres blog, https://aps.hans-bredowinstitut.de/personalization-google/
Ratkiewicz, Jacob, Michael D. Conover, Mark Meiss, Bruno Goncalves, Alessandro Flammini,
Filippo Menczer Menczer (2011) ‘Detecting and Tracking Political Abuse in Social Media’,
Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media, Barcelona, Spain,
July.
Reid, John, Bastiaan Geleijnse and Jean-Marc van Tol (2018) ‘Fokke en Sukke hebben zitting in de
taskforce nepnieuws’, Cartoon, 17 January.
Renner, Nausicaa (2017) ‘Memes trump articles on Breitbart’s Facebook page’, Columbia Journalism
Review, 30 January.
Rieder, Bernhard (2018) “Facebook’s app review and how independent research just got a lot
harder”. Politics of Systems blog, 11 August.
_____ (2012). The refraction chamber: Twitter as sphere and network. First Monday, 17(11).
_____, Ariadna Matamoros-Fernández and Òscar Coromina (2018). “From ranking algorithms to
‘ranking cultures’: Investigating the modulation of visibility in YouTube search results.”
Convergence: The International Journal of Research into New Media Technologies, 24(1): 50–68.
_____, Rasha Abdulla, Thomas Poell, Robbert Woltering, Liesbeth Zack (2016) ‘Data Critique and
Analytical Opportunities for Very Large Facebook Pages. Lessons Learned from Exploring “We
Are All Khaled Said”’, Big Data & Society, 2.2: 1-22
RNTC (2019). ‘Challenging fake news’, webpage, Hilversum: RNTC.
Https://rntc.com/blog/challenging-fake-news.
Roberts, Sarah T. (2016) ‘Commercial Content Moderation: Digital Laborers’ Dirty Work’, in Safiya
Umoja Noble and Brendesha M. Tynes (eds.) The Intersectional Internet, New York, NY: Peter
Lang, pp. 147-160.
Robertson, Adi (2017) ‘After its 4chan slip-up, is it time for Google to drop Top Stories?’, The Verge,
3 October.
RoBhat Labs (2017) ‘Identifying Propaganda Bots on Twitter’, Medium, 31 October.
Roeder, Oliver (2018) ‘We Gave You 3 Million Russian Troll Tweets. Here’s What You’ve Found So
Far’, FiveThirtyEight, 8 August.
Roermund, Jannes van (2017) ‘Wie Zijn de Mensen Achter Het Nepnieuws in Nederland?’ Nieuwe
Revu, 26 April.
Roig-Franzia, Manuel (2019) ‘Inside the Spectacular Fall of the Granddaddy of Right-Wing
Conspiracy Sites’, The Washington Post, 2 April.
190

https://www.washingtonpost.com/lifestyle/style/inside-the-spectacular-fall-of-the-granddaddyof-right-wing-conspiracy-sites/2019/04/02/6ac53122-3ba6-11e9-a06c3ec8ed509d15_story.html?utm_term=.7032f27e6939.
Rogers, Richard (2019) Doing Digital Methods, London: Sage.
______ (2018a) ‘Otherwise Engaged: Social Media from Vanity Metrics to Critical Analytics’,
International Journal of Communication, 12: 450-472.
______ (2018b) ‘Digital Methods for Cross-Platform Analysis’, in Jean Burgess, Alice Marwick and
Thomas Poell (eds.), SAGE Handbook of Social Media. London: Sage, pp. 91-110.
______(2017) ‘Foundations of digital methods: Query design’ in Mirko Tobias Schäfer and Karin
Van Es (eds.) The Datafied Society. Studying Culture through Data, Amsterdam: Amsterdam University
Press, pp. 75-94.
______ (2013) Digital Methods, Cambridge, MA: MIT Press.
______(2013). Debanalizing Twitter: The transformation of an object of study, in Proceedings of the 5th
Annual ACM Web Science Conference, New York, NY: ACM, pp. 356-365.
______ (2005) ‘Poignancy in the US political blogosphere’, Aslib Proceedings: New Information
Perspectives, 57(4): 356-368.
Romero, Jessica (2019) ‘Preventing Inauthentic Behavior on Instagram’, Facebook Newsroom, 25 April.
https://newsroom.fb.com/news/2019/04/preventing-inauthentic-behavior-on-instagram/
Rone, Julia (2019) ‘Why talking about “disinformation” misses the point when considering radical
right “alternative” media’, LSE Media Policy project blog, 9 January.
Rony, Md Main Uddin, Naeemul Hassan and Mohammad Yousuf (2017) “BaitBuster: Destined to
Save You Some Clicks”. Proceedings of Computation+Journalism Symposium, Northwestern University,
October.
Roozenbeek, Jon and Sander van der Linden (2018) ‘The fake news game: actively inoculating
against the risk of misinformation’, Journal of Risk Research, DOI:
10.1080/13669877.2018.1443491.
RTL Nieuws (2018) ‘Alles Over Complottheorie QAnon - #TrumpUpdate 77’, RTL Nieuws, 15
September. https://www.rtlnieuws.nl/nieuws/laatste-videos-nieuws/video/4417801/alles-overcomplottheorie-qanon-trumpupdate-77.
Ruusuvirta, Outi (2010) ‘Much Ado About Nothing? Online Voting Advice Applications in Finland’,
in Cedroni, Lorella and Diego Garzia (eds.), Voting Advice Applications in Europe. The State of the
Art, Napoli: ScriptaWeb, pp. 47-77.
Sängerlaub, Alexander, Miriam Meier and Wolf-Dieter Rühl (2017) ‘Fakten statt Fakes: Das
Phänomen “Fake News”’, Berlin: Stiftung Neue Verantwortung.
Shane, Scott (2018) ‘How Unwitting Americans Encountered Russian Operatives Online’, New York
Times, 18 February.
Shao, Chengcheng, Giovanni Luca Ciampaglia, Onur Varol, Kai-Cheng Yang, Alessandro Flammini
and Filippo Menczer (2018) ‘The spread of low-credibility content by social bots’, Nature
Communications, (9)4787, 20 November.
Shaw, Eugene (1979) ‘Agenda-Setting and Mass Communication Theory’, International Communication
Gazette, 25(2): 96–105.
Shead, Sam (2019) ‘Facebook Reacts to Live-Streamed Footage of the Deadly New Zealand Mass
Shooting That Was Posted on Its Platform’, Business Insider Australia, March 15.

191

https://www.thisisinsider.com/facebook-responds-to-alleged-live-stream-of-christchurchshooting-2019-3.
Shieber, Jason (2017) ‘How Reports from 4chan on the Las Vegas Shooting Showed Up on Google
Top Stories’, TechCrunch (blog), 2 October. http://social.techcrunch.com/2017/10/02/howreports-from-4chan-on-the-las-vegas-shooting-showed-up-on-google-top-stories/
Silverman, Craig, J. Lester Feder, Saska Cvetkovska and Aubrey Belford (2018) ‘Macedonia’s ProTrump Fake News Industry Had American Links, And Is Under Investigation For Possible
Russia Ties’, Buzzfeed News, 18 July.
Silverman, Craig (2016) ‘This Analysis Shows How Viral Fake Election News Stories Outperformed
Real News On Facebook’, Buzzfeed News, 16 November.
Silverman, Craig and Lawrence Alexander (2016) ‘How Teens In The Balkans Are Duping Trump
Supporters With Fake News’, Buzzfeed News, 3 November.
Silverman, Craig and Jeremy Singer-Vine (2016) ‘The True Story Behind The Biggest Fake News Hit
Of The Election’, Buzzfeed News, 16 December.
Silverman, Craig, Lauren Strapagiel, Hamza Shaban, Ellie Hall and Jeremy Singer-Vine (2016)
‘Hyperpartisan Facebook Pages Are Publishing False And Misleading Information At An
Alarming Rate’, Buzzfeed News, 20 October.
Sismondo, Sergio (2017) ‘Post-truth?’, Social Studies of Science, 47(1): 3–6.
Snyder, Timothy (2018) The Road to Unfreedom: Russia, Europe, America, New York, NY: Tim Duggan
Books.
Solon, Olivia and Sam Levin (2016) ‘How Google’s search algorithm spreads false information with
a rightwing bias’, The Guardian, 16 December.
Sommer, Will (2018) ‘Instagram Is the Alt-Right’s New Favorite Haven’, The Daily Beast, 30 October.
Southern Poverty Law Center (n.d.), ‘WorldNetDaily’, blog, Southern Poverty Law Center.
https://www.splcenter.org/fighting-hate/extremist-files/group/worldnetdaily.
Srnicek, Nick (2017) Platform Capitalism. Cambridge: Polity Press.
Statista (2019) ‘Most famous social network sites worldwide as of January 2019, ranked by number
of active users (in millions)’, Statista. https://www.statista.com/statistics/272014/global-socialnetworks-ranked-by-number-of-users/
Still, Keith (2017) ‘Inauguration Crowd Size’, Crowd Safety and Risk Analysis blog, Manchester:
Manchester Metropolitan University. http://www.gkstill.com/CV/Projects/Counting.html.
Swedish Civil Contingencies Agency (2018) Countering information influence activities: A handbook for
communicators, Karlstad: Swedish Civil Contingencies Agency (MSB).
Swedish Media Council (2019) Källkritik — en utmaning, webpage,
https://statensmedierad.se/larommedier/kallkritikvemvadvarfor/kallkritikenutmaning.422.html,
5 March.
Tan, Yvette (2017) ‘There's a vending machine selling fake Instagram likes, because this is what
we've become’, Mashable, 7 June.
Tandoc Jr., Edson C., Zheng Wei Lim and Richard Ling (2018) ‘Defining “Fake News”: A typology
of scholarly definitions’, Digital Journalism, 2: 137-153.
Thomas, David R. (n.d.) ‘A General Inductive Approach for Analyzing Qualitative Evaluation Data’,
American Journal of Evaluation 27 (2): 238–46. https://doi.org/10.1177/1098214005283748.
Timberg, Craig (2017) “Russian propaganda may have been shared hundreds of millions of times,
new research says”. Washington Post, 5 October.
192

Tripodi, Francesca (2018) ‘Alternative Facts, Alternative Truths’, Data & Society: Points blog, 23
February.
Tucker, Joshua A., Andrew Guess, Pablo Barberá, Cristian Vaccari, Alexandra Siegel, Sergey
Sanovich, Denis Stukal, and Brendan Nyhan (2018) ‘Social Media, Political Polarization, and
Political Disinformation: A Review of the Scientific Literature’, Report, Menlo Park, CA: William
and Flora Hewlett Foundation.
Tufekci, Zeynep (2018) ‘YouTube, the Great Radicalizer’, The New York Times, 10 March.
https://www.nytimes.com/2018/03/10/opinion/sunday/youtube-politics-radical.html.
Turner, Stephen (2001) ‘What is the problem with experts?’, Social Studies of Science, 31(1): 123–149.
Tuters, Marc, and Daniël De Zeeuw (2019) ‘Teh Internet Is Serious Business: On the Deep
Vernacular Web Imaginary’, ANSOC Working Paper.
Tuters, Marc, Emilija Jokubauskaitė, and Daniel Bach (2018) ‘Post-Truth Protest: How 4chan
Cooked Up the Pizzagate Bullshit’, M/C Journal 21(3). http://journal.mediaculture.org.au/index.php/mcjournal/article/view/1422.
Tuters, Marc, Emilija Jokubauskaitė and Daniel Bach (2018) ‘Post-Truth Protest: How 4chan
Cooked Up the Pizzagate Bullshit’, M/C: A Journal of Media and Culture, 21(3).
Vaidhyanathan, Siva (2018) ‘Why Facebook Will Never Be Free of Fakes’, New York Times, 5
September.
Vaidhyanathan, Siva (2017) ‘Facebook Wins, Democracy Loses’, New York Times, 8 September.
Vassil, Kristjan (2011) Voting smarter? The impact of voting advice applications on political behavior, PhD
dissertation, European University Institute, Tartu, Estonia.
Venturini, Tommaso (2019) ‘From Fake to Junk News, the Data Politics of Online Virality’, in
Didier Bigo, Engin Isin, and Evelyn Ruppert (eds), Data Politics: Worlds, Subjects, Rights. London:
Routledge. https://hal.archives-ouvertes.fr/hal-02003893.
Volkova, Svitlana, Kyle Shaffer, Jin Yea Jang and Nathan Hodas (2017) ‘Separating facts from
fiction: Linguistic models to classify suspicious and trusted news posts on Twitter’, in Proceedings
of the 55th Annual Meeting of the Association for Computational Linguistics, 2: 647-653.
Volpicelli, Gina (2019) ‘This is how Facebook’s news feed fact-checking will work in the UK,’ Wired,
11 January.
Vosoughi, Soroush, Deb Roy and Sinan Aral (2018) ‘The spread of true and false news online’,
Science, 359(6380): 1146-1151.
Walgrave, Stefaan, Michiel Nuytemans and Koen Pepermans (2009) ‘Voting Aid Applications and
the Effect of Statement Selection’, West European Politics, 32(6): 1161-1180.
Wang, Rong, Wenlin Liu, and Shuyang Gao (2016). ‘Hashtags and information virality in networked
social movement: Examining hashtag co-occurrence patterns’, Online Information Review, 40.7: 850866.
Wardle, Claire (2018) Information Disorder: The Essential Glossary, Shorenstein Center, Cambridge, MA:
Harvard Kennedy School.
Wardle, Claire and Hossein Derakhshan (2017) Information Disorder: Toward an interdisciplinary framework
for research and policy making, Strasbourg: Council of Europe, September 27.
Waters, Richard (2017) ‘Facebook and Google help showcase Las Vegas fake news’, Financial Times, 3
October.
Weisman, Jonathan (2018) (((Semitism))): Being Jewish in America in the Age of Trump, New York, NY: St.
Martin's Press.

193

Weiss, Ben (2018) ‘Opinion | Meet the Renegades of the Intellectual Dark Web’, New York Times, 8
May. https://www.nytimes.com/2018/05/08/opinion/intellectual-dark-web.html.
Wendling, Mike (2018a) ‘The (almost) complete history of ‘fake news’’, BBC News, 22 January.
_____ (2018b). Alt-Right: From 4chan to the White House. London: Pluto Press.
Wieringa, Maranke, Tim de Winkel and Callum Lewis (2017) ‘Wie is de waakhond op sociale
media?’, report, Utrecht: Utrecht Data School.
Woolley, Samuel and Philip N. Howard (2016) ‘Social media, revolution, and the rise of the political
Bot’, in Piers Robinson, Philip Seib, Romy Fröhlich (eds.), Handbook of Media, Conflict and Security.
New York, NY: Routledge, 282-292.
Wynne, Brian (1991) ‘Knowledge in context’, Science, Technology & Human Values, 16(1): 111-121.
YouTube (2019) ‘Related channels discontinued May 2019’.
https://support.google.com/youtube/answer/7216516?hl=en
_____ (2018) ‘Press - YouTube’ https://www.youtube.com/intl/en-GB/yt/about/press/.
Zannettou, Savvas, Tristan Caulfield, Emiliano De Cristofaro, Nicolas Kourtelris, Ilias Leontiadis,
Michael Sirivianos, Gianluca Stringhini, and Jeremy Blackburn (2017) ‘The Web Centipede:
Understanding How Web Communities Influence Each Other through the Lens of Mainstream
and Alternative News Sources’, 17th ACM Internet Measurement Conference, 405–17.
van Zijl, Frank and Huib Modderkolk (2017) ‘Generaal: Nederland kan digitale dreiging niet aan’, De
Volkskrant, 29 August.
van Zoonen, Liesbet, et al. (2011) ‘YouTube Interactions between Agonism, Antagonism and
Dialogue: Video Responses to the Anti-Islam Film Fitna’, New Media & Society, 13(8): 1283–300.
DOI:10.1177/1461444811405020.

194