User-generated video platforms like YouTube have exploded in popularity over the course of the last decade [4] . For many users, it has also become one of the most important information sources for news, world events, and various topics [13, 45] . Alas, platforms like YouTube are often fertile ground for the spread of misleading and potentially harmful information like conspiracy theories and health-related disinformation [12, 19] .
YouTube (and other social media platforms) have struggled with mitigating the harm from this type of content, in part because of the sheer scale and also because of the deployment of recommendation algorithms [63] . Pure machine learning moderation tools have thus far been insufficient to moderate content, and human moderators had to be brought back into the loop [61] . Additionally, the machine learning algorithms that YouTube relies on to recommend content to users also recommend potentially harmful content [63, 46] , and their opaque nature makes them difficult to audit. For certain types of content, e.g., health-related topics, harmful videos can have devastating effects on society, especially during crises like the COVID-19 pandemic [56] . For instance, since the beginning of the pandemic, we witnessed an explosion in the spread of pseudoscientific conspiracy theories and disinformation, e.g., theories suggesting that COVID-19 is caused by 5G [41] or Bill Gates [62] , or the notorious "Plandemic" conspiracy theory documentary [2] . Unlike the scientific process, where experts develop testable hypotheses and perform experiments to provide evidence for or against the hypothesis, conspiracy theories are built up from tenuous connections between various events, with little to no actual evidence to support them. On user-generated video platforms like YouTube, these hypotheses are often presented as facts, regardless of whether or not they have been tested, whether any evidence exists, and whether or not they have been widely debunked.
Motivated by the pressing need to mitigate the spread of pseudoscientific content, in this paper, we focus on detecting and characterizing pseudoscientific and conspiratorial content on YouTube. We aim to assess how likely it is for users with different watch histories to come across pseudoscientific content on YouTube, as well as how YouTube's recommendation algorithm contributes to the discovery of pseudoscientific content.
Research Questions. More precisely, we set out to answer the following research questions: RQ1 Can we effectively detect and characterize pseudoscientific content on YouTube? RQ2 What is the proportion of pseudoscientific content on the homepage of a YouTube user and how is this affected by the user's watch history?
RQ3 What is the proportion of pseudoscientific content in search results on YouTube? How are they affected by watch history? RQ4 What is the proportion of pseudoscientific content being suggested to users when they just randomly browse YouTube?
Methodology. To answer these questions, we look into 4 pseudoscientific topics related to: 1) COVID-19, 2) flat earth theory, 3) anti-vaccination, and 3) anti-mask movement. We collect 6.6K unique videos and use crowdsourcing to label them as one of three categories: 1) science; 2) pseudoscience; or 3) irrelevant. We then train a deep learning classifier to detect pseudoscientific content across multiple topics on YouTube. Our experimental evaluation shows that the classifier outperforms SVM, Random Forest, and a BERT [18] -based classifier, reaching 76.1% accuracy (RQ1).
The classifier allows us to design and perform experiments to address RQ2-RQ4. More specifically, we use three carefully crafted user profiles, each one with a different watch history, while all other account information remains the same. We also perform experiments using a browser without a Google Account to simulate non-logged-in users, and using exclusively the YouTube Data API. To build the watch history of the three user profiles, we devise a methodology to identify the minimum amount of videos that must be watched by a user before YouTube's recommendation algorithm starts generating more personalized recommendations. We build three different profiles: 1) a user interested in scientific content; 2) a user interested in pseudoscientific content; and 3) a user interested in both scientific and pseudoscientific content. Using these profiles, we perform three experiments to quantify the user's exposure to pseudoscientific content on various parts of the platform and how this exposure changes based on a user's watch history.
Findings. Overall, our study leads to the following findings:
1. The watch history of the user substantially affects search results and related video recommendations. 2. Pseudoscientific videos are more likely to appear in search results than in the video recommendations section or the homepage of a user. 3. In traditional pseudoscience topics (e.g., flat earth), there is a higher rate of recommended pseudoscientific content than in more recent topics like COVID-19, antivaccination, and anti-mask. For COVID-19, we also find an even smaller amount of pseudoscientific content being suggested, which may indicate that YouTube took partly effective measures to mitigate pseudoscientific misinformation related to the COVID-19 pandemic.
Contributions. We present the first study focusing on multiple pseudoscientific topics on YouTube while accounting for the effect of a user's watch history. To do so, we build YouTube user profiles that are representative of users viewing pseudoscientific or scientific content. Our methodology can be re-used for other studies focusing on other topics of interest. We will also publish, along with the final version of the paper, our ground truth dataset, the classifier, and the source code/crawlers used in our experiments; we are confident that Pseudoscientific Topic #Seed #Recommended   COVID-19  378  1,645  Flat Earth  200  1,211  Anti-vaccination  346  1,759  Anti-mask  199  912 Total 1,123 5,527 this will enable the research community to shed additional light on YouTube's recommendation algorithm and its potential influence on users' consumption patterns.
In this section, we present our data collection and crowdsourced annotation methodology. We collect a set of YouTube videos related to scientific topics and then use crowdsourcing to create a ground truth dataset of videos that are pseudoscientific or not.
Since we aim to automatically detect video content that is pseudoscientific, we collect a set of YouTube videos related to several scientific and pseudoscientific topics. To do this, we first create a list of four topics whose popularity has increased over the last years: 1) COVID-19 [22] , 2) the anti-vaccine movement [11] , 3) the anti-mask movement [51] , and 4) the flat earth theory [55] .
Next, we use the YouTube Data API [17] , which provides metadata of videos uploaded on YouTube, and we perform a search query for each selected topic obtaining the first 20 videos as returned by YouTube's Data API search functionality. We refer to those videos as the "seed" videos of our data collection methodology. Additionally, for each seed video, we collect the top 10 recommended videos associated with it, as returned by the YouTube Data API. We perform our data collection between August 1, 2020 and August 10, 2020, collecting 6.6K unique videos in total (1.1K seed videos and 5.5K videos that are recommended from the seed videos). Table 1 summarizes  our dataset. For each video in our dataset, we collect the following: 1) the transcript of the video; 2) video snippet, which is the concatenation of the video title and description; 3) a set of tags defined by the uploader; 4) video statistics such as the number of views, likes, etc.; and 5) the 200 top comments, defined by YouTube's relevance metric, without their replies. 
To create a ground truth dataset of scientific and pseudoscientific videos, we use the Appen [9] platform to get crowdsourced annotations for all the collected videos. Each video is presented to three annotators that inspect its content and metadata to assign one of three labels:
• Science. A video falls under the "Science" category when it contains content that is related to any scientific field that systematically studies the structure and behavior of the natural world or humanity's artifacts (e.g., Chemistry, Biology, Mathematics, Computer Science, etc.). Videos that debunk science-related conspiracy theories (e.g., explaining why the 5G technology is not harmful) also fall under this category. For example, a COVID-19 video with an expert estimating the total number of cases or excess deaths falls under this category if the estimation is based on the scientific consensus and official data.
• Pseudoscience. A video falls under the "Pseudoscience" category when it contains content that meets at least one of the following criteria: (a) holds a view of the world that goes against the scientific consensus (e.g., anti-vaccing movement); (b) consists of statements or a belief that is self-fulfilling or unfalsifiable (e.g., Meditation [58] , Reiki healing [65] , etc.); (c) develops hypotheses that are not evaluated following the scientific method (e.g., Astrology); or (d) explains events as secret plots by powerful forces rather than as overt activities or accidents (e.g., the 5G-coronavirus conspiracy theory).
• Irrelevant. We consider a video "Irrelevant" when it contains content that is not relevant to any scientific field and does not fall under the Pseudoscience category. For example, movie trailers, music videos, and cartoon videos are considered irrelevant. Conspiracy theory debunking videos that are not relevant to a scientific field are considered irrelevant (e.g., a video debunking the Pizzagate conspiracy theory).
Annotation. The annotators were given instructions on what constitutes scientific and pseudoscientific content using appropriate descriptions and several examples and were offered $0.03 per annotation. Each video is annotated by three annotators. To ease the annotation process, we provide a clear description of the annotation task, our labels, as well as all the video information that an annotator needs to inspect and correctly annotate a video. Screenshots of the instructions are available, anonymously, at [8] .
The platform provides no demographic information about the annotators, other than an assurance that they are experienced annotators with high accuracy in other tasks. To assess the quality of the annotators, before allowing them to submit annotations we ask them to annotate 5 test videos randomly selected from a set of 54 test videos (20 science, 21 pseudoscience, and 13 irrelevant) annotated by the first author of this paper. An annotator can submit annotations only when at least 3 out of the 5 test videos are annotated correctly.
We also calculate the Fleiss' Kappa Score (k) [25] to assess the agreement of annotators. In the end, we get k = 0.14, which is considered "slight" agreement [36] . This relatively low agreement score is not surprising due to the subjective nature of the problem. For each video, we assign one of the labels according to the majority agreement of all the annotators, except a small percentage (13.8%) where all annotators disagreed among each other, which we exclude from our ground truth dataset. The final ground truth dataset includes 1,197 science, 1,325 pseudoscience, and 3,212 irrelevant videos. Table 2 shows the number of videos from each class for each of the 4 topics considered.
We only collect publicly available data, we make no attempt to de-anonymize users, and overall follow standard ethical guidelines [6, 20, 53] . We also note that we obtained advice and ethics approval by the first author's national ethics committee to ensure that our crowdsourced annotation process does not pose risks to the annotators despite the occasionally harmful nature of the misinformative material.
To train and test a classifier that detects pseudoscientific videos we use our ground truth dataset of 5,734 videos. Since our aim is to train a classifier able to discern pseudoscientific videos from science and irrelevant videos, we decide to collapse our three labels into two, by combining the science with the irrelevant videos into one "Other" category resulting in a ground truth dataset that contains 1,325 pseudoscience and 4,409 "Other" videos.
Below we provide a description of the input features, as well as the architecture of our proposed classifier. We perform an experimental evaluation to assess the performance of the classifier and an ablation study to understand which of the input features contribute the most to the classification task. In the end, the outputs of all four branches are concatenated to form a five-layer, fully-connected neural network that merges their output and drives the final classification.
The classifier uses fastText [23] , a library implemented by Facebook for efficient learning of word and document-level vector representations, as well as for sentence classification. We use fastText to generate vector representations (embeddings) for all the available video metadata in text. Specifically, for each input feature, we use the pre-trained fastText models released in [43] and we fine-tune them using each of our corresponding input features. This allows us to extract a 300dimensional vector representation for each of the following input features of our dataset:
Snippet. The snippet is the concatenation of the title and the description of a video.
Tags. Tags are words defined by the uploader of a video to describe the content of the video.
Transcript. We consider the transcript of the video, which comprises the subtitles uploaded by the creator of the video or auto-generated by YouTube. The transcript is one of the most important features since it describes the content of the video. For the transcript, the classifier uses the fine-tuned model to learn a vector representation of the concatenated text of the transcript. Comments. We consider the top 200 comments of the video as returned by the YouTube Data API, without their replies. We first concatenate the comments of each video and use them to fine-tune the fastText model and extract vector representations. The second part of the classifier (the "Fusing Network" in Figure 1 ) is essentially a four-layer, fully-connected, dense neural network. At first, we use a Flatten utility layer to merge the outputs of the four branches of the first part of the classifier, creating a 1200-dimensional vector. This vector is processed by the four subsequent layers comprising 256, 128, 64, and 32 units, respectively, with ReLU activation. To avoid overfitting, we regularize using the Dropout technique [57] . More specifically, at each one of the four fully-connected layers, we apply a Dropout level of d = 0.5, which means that during each iteration of training half of the units of each layer do not update their parameters. Finally, the output of the Fusing Network is fed to the last dense-layer neural network of two units with softmax activation, which essentially yields the probabilities that a particular video is pseudoscientific or not.
We implement the deep learning classifier using Keras [15] with Tensorflow as the back-end [1] . We use ten-fold stratified cross-validation [10] , training and testing the classifier for binary classification using all the aforementioned input features. To deal with data imbalance, we use the Synthetic Minority Over-sampling Technique (SMOTE) [14] and oversample only the training set at each fold. For stochastic optimization, we use the Adam algorithm with an initial learning rate of 1e−3, and = 1e−8. We then compare the performance of the classifier, in terms of accuracy, precision, recall, F1 score, and area under the ROC curve (AUC), against the following three baselines: 1) a Support Vector Machine (SVM) classifier with parameters γ = 0.1 and C = 10.0, 2) a Random Forest classifier with an entropy criterion and number of minimum samples leaf equal to 2, and 3) a Deep Neural Network with the same architecture as our proposed classifier but using Google's BERT method [18] and more specifically a pre-trained BERT model [60] to learn document-level representations from all the available input features (BERT-based). For hyper-parameter tuning of baselines (1) and (2), we use the grid search strategy, while for (3) we use the same hyper-parameters as the proposed classifier. For a fair comparison, all evaluated models use all available input features. Table 3 reports the performance of all classifiers, while Figure 2 plots their ROC curves. We observe that our classifier outperforms all baseline models across all performance metrics. Specifically, compared to Random Forest, which has the best overall performance among the baselines, we improve accuracy, precision, recall, F1 score, and AUC by 4.0%, 3.3%, 4.0%, 3.1%, and 5.4%, respectively.
Ablation Study. To understand which of the input features contribute the most to the classification of pseudoscientific videos we perform an ablation study. That is, we systematically remove each of the four input feature types (as well as their associated branch in the proposed classifier's architecture), and retrain the classifier. Again, we use ten-fold cross-validation and the oversampling technique to deal with data imbalance. Table 4 shows the performance metrics of all the classifiers for each possible combination of inputs. For the single input classifiers, we observe that the comments and the transcript of the video yield the best performance, indicating that they are the more informative input features. For the classifiers trained with combinations of three input features, we observe similar performance. However, by using all the available input features we achieve better performance, which indicates that all four input features are important for the classification task.
Remarks. Although the proposed classifier outperforms all the baselines, its accuracy points to the subjective nature of scientific vs. pseudoscientific content on YouTube. Also, the low agreement score of our crowdsourced annotation brings out the Table 4 : Performance of the proposed classifier trained with all the possible combinations of the four input feature types.
difficulty in identifying whether a video is pseudoscientific, and it is also evidence of the hurdles in devising models that automatically discover pseudoscientific content. However, we argue that our classifier can detect pseudoscientific content with acceptable performance and can provide a meaningful signal of the behavior of YouTube's recommendation algorithm with regards to recommending pseudoscientific content (RQ1).
In this section, we analyze the prominence of pseudoscientific videos on various parts of the platform (i.e., homepage, search results, and video recommendations) using a variety of experiments.
We focus our analysis on three parts of the platform: 1) the homepage; 2) the search results page; and 3) the video recom-mendations section (recommendations when watching videos). Figure 3 shows an example of each part. In our experiments, we aim to simulate the behavior of users with varying interests that watch videos on YouTube, and measure how the watch history affects the recommendation of pseudoscientific content.
To do so, we create three different Google accounts, each one with a different watch history, while all other account information is the same to avoid confounding effects caused by profile differences. Additionally, we perform experiments without a Google Account to simulate not logged-in users, as well as using the YouTube Data API (if the API provides the required functionality) to investigate the differences between YouTube as an application and the API. User Profile Creation. All three Google accounts were manually created and phone verification was performed. According to Hussein et al. [30] , once a user forms a watch history, user profile attributes (e.g., demographics, geolocation) affect future video recommendations. Hence, since we are only interested in the watch history, each account has the same profile: 30 years old, female. To minimize the likelihood of Google automatically detecting our user profiles, we carefully crafted each one assigning them a unique name and surname, while we read all introductory emails and performed standard phone verification. To the best of our knowledge, none of the created user profiles were banned or flagged by Google during or after our experiments. Watch History. Next, we build the watch history of each profile aiming to create the following three profiles: 1) a user interested in legitimate science videos ("Science Profile"); 2) a user interested in pseudoscientific content ("Pseudoscience Profile"); and 3) a user interested in both science and pseudoscience videos ("Science/Pseudoscience Profile").
To find the minimum number of videos required to be watched by a user for YouTube to understand the user's interests and generate more personalized recommendations, we use a newly created Google account with no watch history and we perform the following experiment. First, we randomly select a video, which we refer to as the "reference" one, from the "COVID-19" pseudoscientific videos of our ground truth dataset and we collect its top 20 recommended videos. Next, we create a list of 100 randomly selected videos from the "COVID-19" pseudoscientific videos of our ground truth dataset, and we repeat the following process iteratively: 1. We start by watching a video from the list of the randomly selected pseudoscientific videos;
2. We visit the reference video and we collect the top 20 recommendations, store them, and compare them using the Jaccard similarity index with all the recommendations of the reference video collected in the previous iterations;
3. If all the recommended videos of the reference video at the current iteration have also been recommended in the previous iterations then we stop our experiment; otherwise, we delete the watch history of the user, we increase the number of videos we watch at Step 1 by one, and proceed to the next iteration. We find that the minimum amount of videos required to be watched by a user in order for YouTube to start generating more personalized recommendations is 22. Finally, we select the most popular science and pseudoscience videos from our ground truth dataset, based on the number of views, likes, comments, etc., and use them to personalize the profiles of each one of the three Google Accounts. Since it is not clear how the satisfaction score on videos is measured by YouTube and how watch time affects this score, during profile training we always watch the same proportion of the video (50% of the total duration) and always like the videos we watch.
Controlling for noise. Some differences in search results and recommendations are likely due to other factors than the user's watch history and personalization in general. To diminish the possibility of such noise affecting our results, we take the following steps: 1) Experiments with identical search queries for all accounts are executed in parallel to avoid updates to search results over time for specific search queries; 2) All requests to YouTube are sent through the same US-based proxies to avoid location-related issues (i.e., differences in localized results); 3) We perform all experiments using the same browser user-agent and operating system; 4) To avoid the carry-over effect (previous search and watch activity affecting subsequent searches and recommendations), at each repetition of our experiments, we use the "Delete Watch and Search History" [28] to delete the activity of the user on YouTube from the date after the user profiles were build; and 5) Similarly to the profiles' watch history creation, in our experiments we always watch the same proportion of the video (50% of the total duration).
Implementation. The experiments are written as custom scripts using Selenium [44] in Python 3.7. We use Selenium since it provides all the features we need and allows for full control of the behavior and the configuration of the browser (e.g., cookies management). The Selenium WebDriver also offers a broad range of features including JavaScript execution, which allows for more realistic simulations. For each Google Account, we create a separate Selenium instance for which we set a custom data directory, thus being able to perform manual actions on the browser before starting our experiments, e.g., performing Google authentication, installing AdBlock Plus [49] to prevent advertisements within YouTube videos from interfering with our simulations, etc. Finally, for all our experiments, 
We begin by assessing the degree of the problem of pseudoscientific content on the homepage of a user on YouTube. To do so, using each one of the three user profiles (Science, Pseudoscience, and Science/Pseudoscience), as well as another user with no account (No Profile) that simulates the behavior of not logged-in users, we visit the homepage of the user and collect and classify the top 20 videos as ranked by YouTube on the homepage of each user. We repeat this experiment 20 times with a wait time of 10 minutes between each experiment. Note that we cannot perform this experiment using the YouTube Data API since this functionality is not supported by the API. We repeat the same experiment multiple times (20 times in this case) since YouTube shows different videos on the homepage each time a user visits YouTube. We perform this experiment between September 26, 2020 and September 27, 2020. Figure 4 shows the percentage of unique pseudoscientific videos in the homepage of each user profile. We find that, 27.4%, 26.5%, 25.6%, and 26.1%, of all the unique videos encountered by the Science, Pseudoscience, Science/Pseudoscience, and the No profile (browser), respectively, are pseudoscientific. Overall, all user profiles receive a similar amount of pseudoscientific content on their homepage. This indicates that YouTube recommends pseudoscientific content to users' homepages irrespectively of whether they have watched such videos in the past and even of whether they have watched a lot of benign, or even contradictory videos (e.g., Science videos).
Next, we focus on quantifying the prevalence of pseudoscientific content when users search for videos on YouTube. For this experiment, we use the 4 pseudoscientific topics used to create our ground truth dataset and, for each topic, we perform search queries on YouTube. For each search query, we retrieve the top 10 videos and use our classifier to classify each video in the result set. We repeat this experiment 20 times for each pseudoscientific topic using all three user profiles, as well as two users with no profile (one using a browser and another one using YouTube's Data API). Recall that we delete the user's watch history between each experiment repetition as well as between the experiments performed with different search queries to ensure that future search results are not affected by previous activity other than our initial, controlled watch history of the user. We perform this experiment between September 27, 2020 and October 1, 2020.
Overall, we find a big variation in the results across pseudoscientific topics (see Figure 5 ). For more traditional pseudoscientific topics like "Flat Earth", YouTube search returns more pseudoscientific content compared to other pseudoscientific topics. Furthermore, for more controversial and emerging topics like "Anti-vaccination" and "Anti-mask," most of the videos returned by YouTube are pseudoscientific.
On the other hand, for topics like "COVID-19," the majority of the recommended videos are not pseudoscientific, suggesting that YouTube's recommendation algorithm does a better job in recommending less harmful videos (at least for COVID-19). In addition, as for this topic, the user profiles (i.e., the watch history) affect the amount of pseudoscientific videos that are recommended to a user, since the users with pseudoscience and science/pseudoscience watch history receive a higher proportion of pseudoscientific content than the user with the science watch history. The fact that for "COVID-19," YouTube recommends much less pseudoscientific content may be related to the fact that YouTube has made substantial efforts to tackle COVID-related misinformation [33] , even publishing an official policy specifically for COVID-19 medical misinformation [27] . This is not the case, however, for other controversial pseudoscientific topics like "Anti-vaccination" or "Anti-mask." Nevertheless, YouTube has recently announced that they will also attempt to target COVID-19 vaccine misinformation [64] .
Finally, we set out to assess how prominent the problem of pseudoscientific content is, on a large scale, by performing controlled, live random walks on YouTube's recommendation graph, while again measuring the effect of user's watch history. This lets us simulate the behavior of users with different interests who search the platform for a video and then subsequently watch several videos according to recommenda- tions. Note that in YouTube's recommendation graph, videos are nodes and video recommendations are directed edges connecting a video to its recommended videos. For example, a YouTube video page can be seen as a snapshot of YouTube's recommendation graph showing a single node (video) and all the directed edges to all its recommended videos in the graph.
For the simulations, we use the 4 pseudoscientific topics considered for the creation of our ground truth dataset. For each pseudoscientific topic, we initially perform a search query on YouTube and randomly select one video from the top ten search results. Then, we watch the selected video, we obtain its top ten recommended videos, and we randomly select one. Again, we watch the randomly selected video and we randomly select one of its top ten recommendations. Following this process, we simulate the behavior of a user who watches videos based on recommendations selecting randomly the next video from among the top ten recommendations of the current video until we reach five hops (i.e., 6 total videos viewed), which constitutes the end of a single live random walk. We repeat this process for 20 random walks for each search term related to our pseudoscientific topics, while at the same time classifying each video we visit, using our classifier. We perform this experiment for all the three Google accounts, the user with no profile (browser), and using the YouTube Data API between October 1, 2020 and October 12, 2020.
Next, for the live random walks of each user profile we calculate the percentage of pseudoscientific videos encountered over all unique videos that the random walker reaches up to the kth hop. Figure 6 plots this percentage per hop for each of the pseudoscientific topics explored.
When observing the percentage of pseudoscientific videos encountered by each user profile in all the random walks of each pseudoscientific topic, we unveil some interesting findings. Initially, we observe that for "COVID-19," "Flat Earth," and "Anti-vaccination" the amount of pseudoscientific content being suggested to the Pseudoscience profile after five hops is higher than the Science profile (see individual plots in Figure 6 ). More precisely, the amount of unique pseudoscientific videos encountered by the Pseudoscience profile after five hops is 30.8%, 57.4%, and 36.7% for "COVID-19," "Flat Earth," and "Anti-vaccination," respectively, while for the Science profile is 25.7%, 46.5%, and 33.0% However, this is not the case for relatively new and more emerging topics like "Anti-mask," where the Science profile is being suggested a higher propor-tion of pseudoscientific content than the Pseudoscience profile after five hops (see "Anti-mask" in Figure 6 ).
Interestingly, we also find that, for more traditional pseudoscientific topics like "Flat Earth," YouTube suggests more pseudoscientific content to all type of users compared to the other three more recent pseudoscientific topics, another indication that YouTube has taken measures to counter the spread of pseudoscientific misinformation related to important topics like the COVID-19 pandemic.
Overall, we find that in most cases the watch history of the user does affect user recommendations and the amount of pseudoscientific content being suggested by YouTube's algorithm. This is also observed when looking at the results of the random walks performed on the browser by the user with no profile. This profile does not maintain a watch history and is recommended less pseudoscientific content than all the other user profiles after five hops in almost all random walks. Finally, the results of the random walks performed using the YouTube Data API do not consistently follow the trends of browser based walks with user profiles across all topics. For example, in the random walks of "Flat Earth", "Anti-vaccination", and "Antimask" we observe a higher amount of pseudoscientific content being suggested after five hops than any of the other walks.
We now summarize the proportion of pseudoscientific content found from the experiments discussed above and the main take ways from them. Table 5 reports the percentage of unique pseudoscientific videos appearing in the YouTube homepage, search results, and the video recommendations section for each user profile out of all the unique videos encountered by each user profile in each experiment.
The highest percentage of pseudoscientific videos occur in the search results section of YouTube. When looking at the results of each individual experiment, we make some interesting observations. First, for the search results experiment, we observe that for all pseudoscientific topics the Science profile encountered more pseudoscientific content when searching for these topics than the Pseudoscience profile, except for COVID-19, where the Pseudoscience profile encountered more pseudoscientific content. When it comes to video recommendations, in all the random walks of all topics except anti-mask, the Pseudoscience profile encountered more pseudoscientific content than the Science profile. Table 5 : Percentage of unique pseudoscientific videos out of all videos encountered by each user profile in each pseudoscientific topic in the three main parts of the YouTube platform.
Overall, the main take-away points from our analysis include:
1. The YouTube search results and video recommendations experiments show that the watch history of the user substantially affects what videos are suggested to the user.
2. It is more likely to encounter pseudoscientific videos in the search results page of the platform (i.e., when searching for a specific topic) than in the video recommendations sections or the homepage of a user.
3. For traditional pseudoscience topics (e.g., flat earth), there is a higher rate of recommended pseudoscientific content than for more emerging/controversial topics like COVID-19, anti-vaccination, and anti-mask. Furthermore, for COVID-19, we observe an even smaller amount of pseudoscientific content being suggested, which may be a result of measures YouTube took to mitigate pseudoscientific misinformation related to the COVID-19 pandemic.
In this section, we review prior work investigating pseudoscience and misinformation on YouTube, malicious activity on YouTube, auditing of the recommendation algorithm, and on user personalization across the Web.
Pseudoscience and Misinformation. The scientific community has extensively studied the phenomenon of misinformation and the credibility issues of online content [35, 67] . The majority of prior work focuses on analyzing misinformation and pseudoscientific content on other social networks [7, 5, 50, 32] Landrum et al. [37] investigate how users with different science comprehension and attitude towards conspiracies are susceptible to flat Earth arguments on YouTube finding that users with lower science intelligence and higher conspiracy mentality are more likely to be recommended flat earth-related videos. Faddoul et al. [24] develop a classifier to detect conspiratorial videos on YouTube and use it to perform a longitudinal analysis of conspiracy videos. In particular, they perform a simulation of YouTube's autoplay feature, without user personalization, and find that as the conspiracy likelihood of the source video increases so does the conspiracy likelihood of the recommended video.
Malicious activity on YouTube. A substantial body of work focuses on detecting and studying malicious content on YouTube. Jiang et al. [31] investigate how channel partisanship affects comment moderation on YouTube and they find that comments are usually moderated if the channel that posted the video is ideologically extreme. Zannettou et al. [66] propose a deep learning classifier for identifying videos on YouTube that use manipulative techniques to increase their views (i.e., clickbait). Agarwal et al. [3] present a binary classifier trained with user and video features to detect videos promoting hate and extremism on YouTube, while Mariconti et al. [42] build a classifier to predict, at upload time, whether or not a YouTube video will be "raided" by hateful users. Furthermore, Hussain et al. [29] analyze disinformation and crowd manipulation tactics on YouTube. They analyze the metadata of videos promoting conspiracy theories on the platform and apply social network analysis techniques to identify malicious behaviors.
YouTube's Recommendation Algorithm and Audits. Covington et al. [16] provide a description of YouTube's recom-mendation algorithm, focusing on two models: (1) a deep candidate generation model used to retrieve a small subset of videos from a large corpus; and (2) a deep ranking model used to rank those videos based on their relevance to the user's activity. Zhao et al. [68] introduce a large-scale ranking system for YouTube recommendations that extends the Wide & Deep model architecture with Multi-gate Mixture-of-experts for multi-task learning. The proposed model ranks the candidate recommendations of a given video taking into account user engagement (e.g., user clicks) and satisfaction objectives (e.g., video likes).
Ribeiro et al. [52] perform a large-scale audit of user radicalization on YouTube: they analyze videos from Intellectual Dark Web, Alt-lite, and Alt-right channels, showing that they increasingly share the same user base. They also analyze YouTube's recommendation algorithm finding that Altright channels can be reached from both Intellectual Dark Web and Alt-lite channels. Papadamou et al. [46] focus on characterizing and detecting disturbing videos on YouTube targeting young children, while they also propose a classifier for detecting such videos. Using the proposed classifier they analyze YouTube's recommendation algorithm finding that young children are likely to encounter disturbing videos when they randomly browse the platform starting from benign videos. Papadamou et al. [47] study the Incel community on YouTube and how inappropriate and hateful content relevant to this community spreads on the platform. They also analyze how such videos are recommended to users by quantifying the probability that a user will encounter an Incel-related video by virtue of YouTube's recommendation algorithm.
User Personalization. Most of the work on user personalization focuses on Web search engines and is motivated by the concerns around the Filter Bubble effect [48] . Hannak et al. [26] measure personalization on Web search and they propose a methodology for measuring personalization in Web search results. They apply this methodology to Google Search finding an 11.7% difference in search results due to personalization. They also find that account login status and the IP address of the user affect search results. Unlike their study, we focus on YouTube and its recommendation algorithm and we devise a different methodology that enables us to assess the effect of a user's watch history on video recommendations in all the parts of the platform.
Kliman-Silver et al. [34] propose a methodology for exploring the impact of location-based personalization on Google search results. Robertson et al. [54] focus on the personalization and composition of politically-related search engine results and they propose a methodology for auditing Google Search using a dynamic set of political queries. Le et al. [38] investigate whether politically oriented Google news search results are personalized based on the user's browsing history. Using a "sock puppet" audit system, they find significant personalization that tends to reinforce the presumed partisanship of a user.
Stöcker et al. [59] analyze the effect of extreme recommendations on YouTube, finding that YouTube's auto-play feature is problematic. They conclude that preventing inappropriate personalized recommendations is technically infeasible due to the nature of the recommendation algorithm. Finally, Hussein et al. [30] focus on measuring misinformation on YouTube and perform audit experiments considering five popular topics like 9/11 and chemtrail conspiracy theories to investigate whether personalization contributes to amplifying misinformation. They audit three YouTube parts, namely, search results, Up-next video, and Top 5 video recommendations. They find that, once a user develops watch history, the demographic attributes affect the extend of misinformation recommended to the users. More importantly, they also find a filter bubble effect in the video recommendations section for almost all the topics they analyze. Instead, we build a classifier and we use it to characterize and detect pseudoscientific misinformation on YouTube by mostly focusing on health-related topics (e.g., , which can have devastating effects on society. We also devise a methodology that allows us to better assess the effect of a user's watch history in all the main parts of the YouTube platform, including the homepage of the user. Unlike Hussein et al., our methodology also includes the simulation of the behavior of users with distinct watch histories who search the platform for a video and subsequently watch several videos according to recommendations.
Remarks. Unlike previous work, we build a classifier and we use it to characterize and detect pseudoscientific misinformation on YouTube, aiming to understand how a user's watch history affects YouTube's recommendations across multiple parts of the platform (i.e., homepage, search results page, and video recommendations). To do this, we devise a methodology that also includes the simulation of the behavior of users with distinct watch histories. Note that we also make our dataset and source code publicly available, hoping to enable further research on understanding the effect of personalization on YouTube, as well as studies focusing on auditing the recommendation algorithm, irrespectively of the topic of interest.
In this work, we studied pseudoscientific content on the YouTube platform. We collected a dataset of 6.6K YouTube videos and, using crowdsourcing, we annotated them according to whether or not they include pseudoscientific content. We then trained a deep learning classifier to detect pseudoscientific videos and used it to perform experiments assessing the prevalence of pseudoscientific content on various parts of the platform while accounting for the effects of the user's watch history. To do so, we crafted a set of accounts with different watch histories.
Overall, we found that the user's watch history indeed substantially affects future user recommendations by YouTube's algorithm. This result should be taken into consideration by communities aiming to audit the recommendation algorithm and understand how it drives users' content consumption patterns. We also found that YouTube search results are more likely to return pseudoscientific content than other parts of the platform like the recommendation engine or a user's homepage. However, we also observed a non-negligible number of pseudoscientific videos on both the video recommendations section as well as the homepage of the users.
Finally, by investigating the differences across multiple pseudoscientific topics, we showed that the recommendation algorithm is more likely to recommend pseudoscientific content from traditional pseudoscience topics, e.g., flat earth, compared to more controversial topics like COVID-19. This likely indicates that YouTube takes measures to counter the spread of harmful information related to important and emerging topics like the COVID-19 pandemic. However, achieving this in a proactive and timely manner across topics remains a challenge.
In addition, the low agreement score of our crowdsourced annotation, as well as the accuracy of our binary classifier point to the difficulty in identifying whether a video is pseudoscientific or not, and also indicates that it is not easy to automate the discovery of misinformation. Hence, we believe that the most potent way for YouTube to effectively cope with misinformation on the platform is a mitigation scheme that uses deep learning models that in turn provide signal of potential pseudoscientific videos to human annotators who examine the videos and make the final decision.
Our work provides insights on pseudoscientific videos on YouTube and provides a set of resources to the research community, as we will make the dataset, the classifier, as well as all the source code of our experiments publicly available. In particular, the ability to run these kind of experiments while taking into account users' watching history will arguably be particularly useful to researchers focusing on demystifying YouTube's recommendation algorithm-irrespective of the topic of interest. In other words, our methodology and codebase are generic, and can be used to study other topics besides pseudoscience, e.g., other conspiracy theories.
Limitations. Naturally, our work is not without limitations. First, we use crowdworkers who are unlikely to have any expertise on identifying pseudoscientific content. Hence, a small percentage of the annotated videos may be misclassified. However, we mitigated this issue by not including annotators with low accuracy on a classification task performed on a test dataset, and annotating each video based on the majority agreement. Second, our ground truth dataset is relatively small for such a subjective classification task. Nonetheless, we argue that the classifier provides a meaningful signal of the behavior of YouTube's recommendation algorithm with regards to recommending pseudoscientific content. Finally, as for user personalization, we only work with watch history, which is only a fraction of the signals YouTube uses for user personalization.
Future Work. A more comprehensive user personalization methodology to account for factors outside of watch history, like account characteristics, location, and user engagement is a clear direction for future work. We also plan to conduct studies to understand how people share and view pseudoscientific content on other social networks like Twitter and Facebook, and how people interact and engage with such content.