# Whose Agenda Is It Anyway? The Effect of Disinformation on COVID-19 Vaccination Hesitancy in the Netherlands

Natalia I. Kadenko, J. Boon, J. van Der Kaaij, W. J. Kobes, A. T. Mulder, J.

J. Sonneveld

► To cite this version:

Natalia I. Kadenko, J. Boon, J. van Der Kaaij, W. J. Kobes, A. T. Mulder, et al. Whose Agenda Is It Anyway? The Effect of Disinformation on COVID-19 Vaccination Hesitancy in the Netherlands. 13th International Conference on Electronic Participation (ePart), Sep 2021, Granada, Spain. pp.55-65, 10.1007/978-3-030-82824-0_5 . hal-04014065

HAL Id: hal-04014065

https://inria.hal.science/hal-04014065

Submitted on 3 Mar 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Whose Agenda Is It Anyway? The Effect of Disinformation on COVID-19 Vaccination Hesitancy in the Netherlands

Natalia I. Kadenko¹ [0000-0001-8831-6744], J.M. van der Boon¹ [0000-0003-2467-9508],
J. van der Kaaij¹,² [0000-0001-6078-7421], W.J. Kobes¹,² [0000-0002-9861-4806], A.T. Mulder¹ [0000-0002-8776-7508], J.J. Sonneveld¹,² [0000-0001-8509-3135]

¹ Delft University of Technology, Delft, The Netherlands

² University of Twente, Enschede, The Netherlands

n.i.kadenko@tudelft.nl

**Abstract.** With the problem of disinformation becoming more apparent, one of the current topics for disinformation campaigns is the COVID-19 vaccine, which has broad implications for public health. This research was conducted to investigate a possible connection between the amount of vaccination-related disinformation and the willingness among the Dutch population to get vaccinated. The contribution of this research is 1) developing a tool-supported approach to identify words and bigrams used in alternative news outlet, 2) classifying disinformation-related vocabulary, 3) applying the approach that relates disinformation and vaccination willingness in the context of the COVID pandemic, highlighting its strengths and limitations. We conceptualised vaccination disinformation, expressed it in certain 'trigger terms' and plotted the popularity of those terms amongst Dutch Internet users over time, using Google Trends and Twitter data. Using a linear regression model, we combined this with vaccination willingness studies of June through December of 2020 to investigate a possible correlation. Our results, while not statistically significant, did point towards a negative relationship between disinformation spread and willingness to vaccinate. Further research, utilizing similar approach and additional available information on vaccination willingness, may provide more insight on disinformation spread and vaccination willingness across the world.

**Keywords:** Disinformation, Social Media, Vaccine Hesitancy

## 1 Introduction

With the rise of social media and ease of website hosting, almost every message can get amplified and reach a large audience. This has enabled diverse actors to spread disinformation and influence a broad range of issues, from democratic processes to public health. Spreading disinformation can plant the ‘seed of doubt’ leading to the vaccination hesitancy [1]. In case of COVID-19 vaccination, the existing narratives of vaccine scepticism have been enhanced by obscure web of relationships between vaccines, 5G, microchips, Great Reset, and Bill Gates. Narratives of a secret government
entity which uses or has fabricated the virus to get control of the population [2], or the QAnon movement [3] have been spread predominantly via social media.

The number of adults that indicate to be willing to vaccinate themselves against COVID has steadily decreased in the Netherlands in the last few months of 2020 [4]. Understanding the causes of vaccination hesitancy can help address this issue. This paper sets out to investigate whether there is a correlation between the amount of disinformation related to the COVID-19 vaccines in the Netherlands and the population's willingness to vaccinate themselves against this virus. We focus on the Netherlands, as, to the best of our knowledge, no similar research has been conducted for this country. Furthermore, while the country has known relatively low vaccine hesitancy, the data from 2018 and 2020 shows decrease in confidence in the safety and effectiveness of vaccines in general, and the reasons for this are not clear [5]. This paper will attempt to answer the following research question:

**RQ: To what extent does disinformation spread affect the willingness to vaccinate against COVID-19 among the Dutch population?**

We conceptualised vaccination disinformation, expressed it in certain 'trigger terms' and plotted the popularity of those terms among the Dutch Internet users over time, using Google Trends and Twitter data. Using a linear regression model, we combined this with vaccination willingness studies of June through December of 2020 to investigate a possible correlation. We used a mixed method approach to explore how the discourse changed over time, and subsequently related this evolution to the estimated vaccination willingness of the Dutch population. First, we used an automated approach to isolate commonly used terms (words or bigrams) in both mainstream and alternative media, where a bigram is a sequence of two adjacent elements from a string of words. This list of bigrams was then manually classified into categories using a qualitative approach. Subsequently we used a second automated tool to measure how the usage of those terms on Twitter and Google Trends evolved. Finally, we used survey data to quantify vaccination willingness and to compare it to the evolution of the usage of those terms.

The key contributions of this paper are: 1) developing a tool-supported approach to identify words and bigrams used in alternative news outlet, 2) classifying disinformation-related vocabulary, 3) applying the approach that relates disinformation and vaccination willingness in the context of the COVID pandemic, highlighting its strengths and limitations.

All the supporting materials referenced in the paper are available in the research data repository<fnref n="1" />.

<footnote n="1">¹ https://doi.org/10.4121/14714031 - dataset in the 4TU.Research Data repository, containing supporting material, (non-copyrighted) data and code used for this study, with its documentation (alternative link: https://github.com/Spirited666/PublicDiscourseMiner-COVID)</footnote>
## 2 Literature review

According to UNESCO handbook [6], misinformation and disinformation mean explic-
itly false information. Misinformation may result from honest mistakes, negligence, or
unconscious biases [7] including possibly outdated or incomplete information that
could still be misleading [8]. Disinformation “entails the distribution, assertion, or dis-
semination of false, mistaken, or misleading information in an intentional, deliberate,
or purposeful effort to mislead, deceive, or confuse” [9, p. 228]. It may hold the same
properties as misinformation, adding an intent to deceive. Studies on the effect of dis-
information on individuals’ attitudes and actions indicate that we need better under-
standing on the relationship between social media usage, disinformation spread, and
polarization [10]. A recent study suggests that even short exposure to fake news could
significantly modify the unconscious behaviour of individuals [11]. Finally, Pizzagate
and QAnon conspiracies have resulted in real-life violence.

The interaction between vaccination hesitancy and disinformation has been studied before COVID-19 [12]. The studies from Italy and Denmark identified negative influence of a landmark event (a court ruling confirming a link between a specific vaccine and autism in Italy [13] and a documentary about the complications related to HPV vaccine in Denmark [14]) on the vaccination uptake. Other studies focused on information diffusion concerning the debate on vaccines in Brazil [15,16] relating disinformation websites and their low Google search result ranking. A study [17] found that the vaccine discourse on the social media (namely, Twitter) has become increasingly polarized in the period between 2011-2019: the percentage of both negative and positive tweets has increased, while the percentage of the neutral ones decreased. The study used hybrid models combining lexicon-based and supervised machine-learning approaches to study the tweets containing vaccination-related keywords. Loomba et al. [18] measured the impact of COVID-19 misinformation on vaccination intent in the UK and USA by conducting a survey on the vaccination readiness before and after exposing the respondents to factual information and misinformation. In both countries they identified a misinformation-induced decline in intent to vaccinate. Kurten and Beullens [19] examine the COVID-related public discourse on Twitter by looking at the change of number of tweets with time and in relation to the landmark events, as well as at the changes in the (emotional) content. They computed the network of bi-grams and showed that landmark events correlated with immediate increase in related tweeting.

In light of this related research, we 1) assume that vaccination-related disinformation can potentially have real-life consequences; 2) apply innovative combination of methods in underexplored setting to investigate the possible connection between the amount of disinformation and the vaccination willingness against COVID-19 among Dutch population; 3) expect to observe a negative relationship.
## 3 Methodology

<figure><img src="image_3.png" /><figcaption>Fig. 1. Methodology overview</figcaption></figure>

This approach is designed to relate the influence of alternative media on the Dutch population with their willingness to vaccinate against COVID-19. We consider that the influence of alternative media grows if the Dutch population uses terms that are specific to alternative media in the Google search engine (the most popular search engine in the Netherlands [20]) or rely on such vocabulary in messages shared on Twitter (2.8 million Dutch users).

We start by selecting news outlet relevant to our research goal. Our list includes four of the most popular newspapers in the Netherlands (NOS, NRC, FD, NU) and eight alternative media sites (corona-nuchterheid, oervaccin, xanderniews, dagelijks-estandaard, transitieweb, stichtingvaccinvrij, viruswaarheid, and staopvoorbrijheid). For selecting the alternative media, we used snowballing technique, starting with the media mentioned in the mainstream news (e.g., due to organized protests) and compiling a list of the other alternative media referenced by these media until the point of saturation has been reached. In all cases, accessibility of the articles was a key criterion, as well as permission from the website's maintainers where appropriate.

From all those media outlets, we isolate all articles mentioning “CORONA” and covering the studied period (January to December 2020), from which we remove
HTML tags, and stop-words (step 1), and store the output in the database, relating the text, the source, and its classification (“mainstream”/”alternative”). From this database, we extract words and combination of words (bigrams) from each source to identify which words/bigrams are specific to mainstream media, alternative media, and those present in both (step 2). This allows us to isolate words/bigrams specific to alternative media, which were manually analyzed and classified by four independent coders in iterative process (step 3). In these steps, we found 7 categories of “alternative” information: people referred to in conspiracy theories (e.g “Bill Gates”), people involved in alternative media, alternative media’s name, other diseases (e.g. “Ebola”), vaccine discouraging terms (e.g. “DNA damage”), accusatory terms for corona’s protagonists (e.g. “Big Pharma”), other (partly) complot theories (e.g. “Big brother”). Subsequently we use those categories, and their associated keywords, to identify usage of terms from alternative media in Dutch messages on Twitter, and queries on Google (step 4). We obtain, for the given time frame, the number of tweets and trends of searches containing keywords.

The next step was to collect data regarding COVID vaccination willingness in the Netherlands (step 5) from the reports by the two research institutes (IPSOS and I&O Research). We selected five studies conducted in June, July, August, September, November and December 2020. Four studies were conducted by research institute I&O Research [4, 21–23] and the fifth one by research institute IPSOS [24]. For comparability in our analysis, the results were recoded into a dichotomous variable. The option “Don’t know” from the IPSOS questionnaire was regarded as a “no, as doubt could also lead to vaccination refusal. The availability of information regarding vaccination willingness limited the time frame of the study – reducing it to June to December 2020. As the final step (step 6), we use linear regression analysis to relate the usage of alternative media terms on Twitter and Google (dependent variables), indicating appropriation of the vocabulary, with vaccination willingness over time (independent variable).

All this data was in ratio scale, allowing us to create such a model.

$$y_i = a_0 + a_1 x_{1i} + \dots + a_n x_{ni}$$

The formula above represents a linear regression model, where x are the categories of disinformation, y is the willingness and i are the months.

# 4 Results and Analysis

## 4.1 Distinguishing disinformation from regular information

A total of four mainstream media and eight alternative media websites were scraped for their articles. The repository lists the sites and the number of articles which were scraped per site. Notably, more articles from the mainstream media (17301) have been used, compared to the alternative media (2652). The difference can be explained by the
mainstream media producing considerably more content related to coronavirus from the different categories. After extracting the words and bigrams which are significantly more popular among alternative media, we had a list of over 1300 terms. These trigger words and bigrams were coded as mentioned in the Methodology section. The final list of coding categories and their description is available in our repository. Category 1, “person referred to in conspiracy theories” was populated not only by the internationally renowned names like Bill Gates and Klaus Schwab, but included several Dutch politicians who hadn’t been closely associated with conspiracy theories before. Categories 2 and 3, “Person involved in alternative media production” and “Name of alternative media” mostly included specific Dutch content. Category 4, “Other disease” was used to try and nuance the severity of COVID-19, or was mentioned as a part of government immunisation programs (i.e., capitalizing on the pre-existing vaccine hesitancy narratives) – similarly to category 5, “Vaccine discourage term”, which included both old and new terms. Category 6 “Accusatory term for corona protagonists” was created to include a broad range of terms, from Big Pharm, tyranny and elite to slave mentality and risk obsession. An interesting find was that category 8, “Other (partly) conspiracy theories” included 5G and microchips narratives, but the relation between COVID-19 and QAnon-related conspiracy theories did not come forward from the data. A possible explanation could be that, despite their growing popularity, QAnon theories have been less popular on Dutch media.

## 4.2 Measuring the amount of disinformation over time

<figure><img src="image_4.png" /><figcaption>Fig. 2. Publication date of articles</figcaption></figure>
Figure 2 shows the number of alternative and mainstream media articles over the stud-
ied period. We can see that the mainstream (blue) and alternative (red) media numbers
follow the same curve and do not show considerably different numbers. However, the
number of mainstream media articles demonstrates a peak in March and April, when
the lockdown was introduced, and a lower valley in July and August, when the number
of cases was lower, while the alternative media show an increase in content, possibly
due to the protests and lawsuits (more on that below).

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Tweet count over time per category</figcaption></figure>

The number of tweets (7719 in total) that included words and/or bigrams from a cate-
gory is visualized in Figure 3. We observe a peak in vaccine discourage terms in July.
One possible explanation is that we are seeing an example of increased inauthentic ac-
tivity [25], with bots tweeting to discourage vaccination willingness. Publications from
alternative news sources could lead to such a peak in tweets, such as the reports about
a mass protest against corona measures that took place at the end of June. Another
explanation could be news publication, such as the reports on the lawsuit started by the
action group “Viruswaarheid” against the Dutch state with the demand to lift all corona
measures, which the group lost in July. Simultaneously, there is a small peak for cate-
gory 8 (“Other (partly) conspiracy theories”). After the peak in July, the “vaccine dis-
courage term” category has an upwards trend, which could be related to the news about
vaccine development and approval starting the public debate. Category 3 has a peak
starting in August, characterized by more protest activity, and ending around November, when the above-mentioned group lost another lawsuit against the Dutch State.

<figure><img src="image_6.png" /><figcaption>Fig. 4. Google Trends</figcaption></figure>

Figure 4 shows the number of searches done per category between June and December, relative to our normalisation factor that was used to combine all the terms. A value above 100 indicates that the term was more popular than our normalisation factor. What is clear is that most categories have a more or less constant search value with no clear peaks for any category. There are, however, several categories which demonstrate an increase in searches, such as category 2 (“Person involved in alternative media production”), categories 3 (“Name of alternative media”) and 4 (“Other (partly) conspiracy theories”). This might indicate the interest in both the content and the concept of alternative news outlets and their main public figures. Notably, category 5 (“Vaccine discourage term”) does not see an increase in any period. This does not correspond with the clear Twitter peak in July 2020, and the increase after that period. One could argue that the rising of a general interest in a certain category should be visible in both Twitter and Google. This lack of alignment, which would be indicative of the increase in interest, might be accounted for by another factor that could have caused the peak – such as inauthentic behaviour or coordinated activity.
4.3 Vaccination willingness

As described in the Methodology section, we applied recoding to the vaccination will-
ingness from the research reports to make the different reports comparable. The result-
ing measurement of vaccination willingness are presented in Figure 5.

<figure><img src="image_7.png" /><figcaption>Fig. 5. Recorded vaccination willingness per month</figcaption></figure>

The main observation is that we see a steady decrease of vaccination willingness over the period June till November, from about 75 percent to 60 percent. In December a sudden increase in vaccination willingness is observed, but due to the fact that we only had partial data for December at the time research was conducted, it was not included in the analysis. Another observation is that the fraction of the respondents indicating to not be willing to vaccinate is approximately equal to the fraction of respondents still unsure about their stance against the vaccine. Both these fractions fluctuate between 15 and 25 percent.

With the vaccination willingness operationalised, we are now able to execute the
analysis where vaccination willingness is used as the dependent variable.
## 4.4 Analysis

The model created during this research set out to predict the willingness of the Dutch public to get vaccinated. Unfortunately, we were not able to create a model that provided significant results for both the Twitter data(F(3.1)=0.53, p=0.73, R²adjusted=-0.54) and the Google trends data (F(3,1)=104.2, p=0.072, R²adjusted=0.987). The biggest obstacle in this quantitative analysis was the low number of predictor values. Only five points were available to use at the time of creation. Due to this the degrees of freedom were significantly restrained and it was not even possible to fit a model using all categories at the same time. The significance of the factors could also not be found due to the high standard error.

<table><caption>**Table 1.** Correlation factors of the categories with the willingness to vaccinate per model</caption><thead><tr><th></th><th>Twitter</th><th>Google</th></tr></thead><tbody><tr><td>1) Person (referred to in conspiracy theories)</td><td>0.119</td><td>0.849</td></tr><tr><td>2) Person involved in alternative media production</td><td>-0.387</td><td>-0.753</td></tr><tr><td>3) Name of alternative media</td><td>-0.145</td><td>-0.540</td></tr><tr><td>4) Other disease</td><td>-0.395</td><td>-0.790</td></tr><tr><td>5) Vaccine discourage term</td><td>-0.686</td><td>0.194</td></tr><tr><td>6) Accusatory term for corona protagonist</td><td>-0.146</td><td>-0.120</td></tr><tr><td>8) Other (partly) conspiracy theories</td><td>-0.447</td><td>-0.630</td></tr></tbody></table>

The models did not prove a significant relationship with the willingness to get vaccinated. However, in the process of building the models the correlation of the categories with the willingness has been calculated. We can make an interesting observation: as can be seen in Table 1, in accordance with our expectations, the direction of the relationship is correct and for some categories a high correlation is present. This may indicate that there might be a relationship between the categories and the willingness to get vaccinated, but our dataset is insufficient to draw conclusion.<fnref n="2" />

<footnote n="2">² Recently, the Dutch government released a dataset on vaccination willingness, which is not compatible with the data set initially used in this study, as the questions asked to members of the public differed, and the time coverage was not suitable for our research: https://coronad dashboard.government.nl/verantwoording#willingness-to-be-vaccinated</footnote>
# 5 Discussion

This research investigated the possible effect of disinformation spread on the vaccination willingness against COVID-19 in the Netherlands and attempts to bring into picture broader context of disinformation spread and its implications for the real world. By distinguishing different categories of disinformation terms identified in newspapers, we have been able to nuance both the spread of disinformation (in Google searches and social media) and its possible real-life effect (vaccination willingness). Using the calculated correlations between the different disinformation categories and vaccination willingness, we observed negative correlations in line with our expectations. While there is no statistically significant relationship between willingness to vaccinate and search for or use of the category “derogatory terms for actors in the pandemic management”, we observed a slight correlation between searches for the category “people involved in alternative media production” and lack of willingness to vaccinate. We attribute the lack of significance to the very low number of available points for predictor values, of which more may be available in the future.

Our results are to be a stepping stone for the similar research, and an illustration of what could be achieved with our method combination. Taking into consideration the lack of statistical significance, we would still like to argue that disinformation has had a negative effect on the vaccination willingness against COVID-19 among Dutch population for the observed period of time. We found some indication for several categories of disinformation negatively influencing vaccination readiness, with relatively high correlation present. This may indicate a relationship between the certain disinformation categories and the willingness to get vaccinated, which could be proved by further research using our disinformation categorization approach.

The study contributes to the developing body of literature on the possible relationship between the disinformation spread and the real-word effects in general, and the possible effect of the vaccination disinformation in particular. The contribution of this research is 1) developing a tool-supported approach to identify words and bigrams used in alternative news outlet, 2) classifying disinformation-related vocabulary in Dutch, 3) applying the approach that relates disinformation and vaccination willingness in the context of the COVID pandemic, highlighting its strengths and limitations. Further work needs to extend this exploratory study to more media outlets. A more detailed qualitative analysis would be beneficial to investigate the possible links between the changes in usage patterns and the diverse worldwide and national current events. Finally, the tweet count used was objective, i.e., the tweets could have originated both from the supporters or opponents of a certain viewpoint or movement. All these limitations can and should be addressed by future research.
*Acknowledgements*. The authors would like to thank Nicolas Dintzner for his as-
sistance with the data management, proofreading, and general support, Digital Compe-
tence Centre of TU Delft for their help with making code F.A.I.R., and the reviewers
for their valuable comments. This work has been supported by the European Commis-
sion through the H2020 program in project CyberSec4Europe (Grant No. #830929).
Any opinions, findings, and conclusions or recommendations in this paper are those of
the authors and do not necessarily reflect the views of their host institutions or the Eu-
ropean Commission.

References

1. Steffens, M.S., Dunn, A. G., Wiley, K. E., and Leask, J.: How organisations promoting vaccination respond to misinformation on social media: a qualitative investigation. BMC Public Health, vol. 19 (1), 1–12 (2019)

2. Reuters: Fact check: Video makes multiple false claims about COVID-19 pandemic, https://www.reuters.com/article/uk-factcheck-pandemic-video/ fact-check-video-makes-multiple-false-claims-about-covid-19-pandemic-idUSKBN27J2HM. Last accessed 18/3/2021

3. BBC News: QAnon: What is it and where did it come from, https://www.bbc.com/news/53498434. Last accessed 18/3/2021

4. Kanne, P. and Driessen, M.: Coronabeleid: bereidheid tot vaccineren. I&O Research, Amsterdam, Tech. Rep. (2020)

5. De Figueiredo, A., Karafillakis, E. and Larson H. J.: State of Vaccine Confidence in the EU and the UK A Report for the European Commission (2020)

6. Cherilyn, I. and Julie, P.: Journalism, fake news & disinformation: handbook for journalism education and training. UNESCO Publishing (2018)

7. Fallis, D.: What is disinformation? Library Trends, vol. 63, no. 3, 401–426 (2015)

8. Karlova, A. and Fisher, K. E. : A social diffusion model of misinformation and disinformation for understanding human information behaviour, Information Research, vol. 18, no. 1, 1–17 (2013)

9. Fetzer J. H.: Information: Does it Have To Be True? Minds and Machines, vol. 14, no. 2, 223–229 (2004)

10. Tucker, J. A., Guess, A., Barberá, P., Vaccari, C., Siegel, A., Sanovich, S., Stukal, D. and Nyhan, B.: Social media, political polarization, and political disinformation: A review of the scientific literature, (2018). https://dx-doi-org.tudelft.idm.oclc.org/10.2139/ssrn.3144139 Last accessed 1/6/2021

11. Bastick, Z.: Would you notice if fake news changed your behavior? An experiment on the unconscious effects of disinformation. Computers in human behavior, 116, 106633 (2021).

12. Gostin, L. O.: Global polio eradication: Espionage, disinformation, and the politics of vaccination, Milbank Quarterly, vol. 92, no. 3, 413–417 (2014)
13. Carrieri V., Madio, L. and Principe, F.: Vaccine hesitancy and (fake) news: Quasi-experimental evidence from Italy, Health Economics (United Kingdom), vol. 28, no. 11, 1377–1382 (2019)

14. Suppli, C. H., Hansen, N. D., Rasmussen, M., Valentiner-Branth, P., Krause, T. G., & Mølbak, K.: Decline in HPV-vaccination uptake in Denmark—the association between HPV-related media coverage and HPV-vaccination. BMC Public Health, 18(1), 1-8 (2018)

15. Massarani, L., Leal, T., and Waltz, I.: The debate on vaccines in social networks: An exploratory analysis of links with the heaviest traffic, Cadernos de Saude Publica, vol. 36 (2020)

16. Arif, N., Al-Jefri, M., Bizzi, I. H., Perano, G. B., Goldman, M., Haq, I., Chua, K. L., Men gozzi, M., Neunez, M., Smith, H., and Ghezzi, P.: Fake news or weak science? Visibility and characterization of antivaccine webpages returned by google in different languages and countries, Frontiers in Immunology, vol. 9, no. JUN, 1–12 (2018)

17. Piedrahita-Valdés. H., Piedrahita-Castillo, D., Bermejo-Higuera, J., Guillem-Saiz, P., Bermejo-Higuera, J.R., Guillem-Saiz, J., Sicilia-Montalvo, J.A., Machío-Regidor. F.: Vaccine Hesitancy on Social Media: Sentiment Analysis from June 2011 to April 2019. Vaccines, 9(1), 28 (2021), https://doi.org/10.3390/vaccines9010028

18. Loomba, S., de Figueiredo, A., Piatek, S. J., de Graaf, K., & Larson, H. J.: Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA. Nature human behaviour, 1-12 (2021)

19. Kurten, S., Beullens, K.: #Coronavirus: Monitoring the Belgian Twitter Discourse on the Severe Acute Respiratory Syndrome Coronavirus 2 Pandemic Behavior, and Social Networking. Cyberpsychology 117-122 (2021) http://doi.org.tudelft.idm.oclc.org/10.1089/cyber.2020.0341

20. Marktaandelen zoekmachines Q2 2020, https://www.pure-im.nl/blog/marktaandelen-zoekmachines-q2-2020/ Last accessed 18/3/2021

21. Kanne, P. and Driessen, M.: Coronabeleid: draagvlak september 2020, I&O Research, Amsterdam, Tech. Rep., (2020)

22. Bereidheid tot vaccineren (onder publiek en zorgpersoneel), I&O Research, Amsterdam, Tech. Rep. December, (2020)

23. Kanne. P.: De coronamaatregelen: draagvlak en gedrag, I&O Research, Amsterdam, Tech. Rep., (2020).

24. IPSOS: Bereidheid tot vaccinatie tegen corona (2020)

25. Allyn, B: Researchers: Nearly Half Of Accounts Tweeting About Coronavirus Are Likely Bots, https://www.npr.org/sections/coronavirus-live-updates/2020/05/20/859814085/researchers-nearly-half-of-accounts-tweeting-about-coronavirus-are-likelybots?t=1616098707209, Last accessed 18/3/2021