Bigfoot in Big Tech: Detecting Out of Domain Conspiracy Theories (2023)

Abstract: We investigate approaches to classifying texts into either conspiracy theory or mainstream using the Language Of Conspiracy (LOCO) corpus. Since conspiracy theories are not monolithic constructs, we need to identify approaches that robustly work in an out-of- domain setting (i.e., across conspiracy topics). We investigate whether optimal in-domain set- tings can be transferred to out-of-domain set- tings, and we investigate different methods for bleaching to steer classifiers away from words typical for an individual conspiracy theory. We find that BART works better than an SVM, that we can successfully classify out-of-domain, but there are no clear trends in how to choose the best source training domains. Addition- ally, bleaching only topic words works better than bleaching all content words or completely delexicalizing texts.

## introduction with the rise of social media over the last 10 years, there has also been a rise in the uses of the internet to spread different types of information, some of it of a more questionable nature. we are interested in the spread of conspiracy theories, which have morphed from a fringe phenomenon to a more widely visible, mainstream phenomenon. along with the increasing spread of misinformation, conspiracy theories have been shown to polarize opinions to extremes and to incite violence @xcite . while conspiracy theories are often seen as monolithic belief systems, the truth is more complex: people who admit to believing a specific conspiracy theory tend to also believe in other conspiracy theories, but they may only believe different subsets of factoids associated with a specific conspiracy theory @xcite . for any computational approach to detecting conspiracy theories, this means that we cannot expect to have access to accurate training data. instead, we will face novel mixes of factoids and conspiracy theories, which deviate from existing training data. for this reason, we investigate here whether it is possible to find out-of-domain conspiratorial texts. we use the language of conspiracy (loco) corpus @xcite to develop classifiers that label a text as either conspiratorial or mainstream, and we investigate under which conditions such classifiers work robustly out-of-domain. more specifically, we investigate bleaching methods to steer the classifiers away from words that are typical for a single conspiracy theory (e.g., 'global warming' for conspiracy theories revolving around climate change). the remainder of this paper is structured as follows: section 2 explains our research questions, section 3 describes related work, and section 4 describes our data and methodology. section 5 describes our results for the in-domain setting (section 5.1), for the out-of-domain setting (section 5.2), and for the bleaching experiments (section 5.3). we conclude in section 6. ## research questions in this paper, we investigate the following research questions: ## related work we restrict our review to work on conspiracy theories and their detection. we acknowledge work on propaganda detection and persuasive technology detection (e.g., @xcite @xcite . there is overlap between these areas of research and the detection of conspiracy theories, given that both approaches work on the document level and examine how information is manipulated. however, propaganda detection primarily focuses on politically related events, whereas conspiracy beliefs tend to span a wide array of topics. although exact markers have proven difficult to identify for conspiracy theories, @xcite showed that conspiracy theory proponents often subscribe to multiple conspiracies, some contradictory, which led them to conclude that conspiracy theories are not stand-alone phenomena from individuals. instead, conspiracies might come in clusters caused by general conspiratorial thinking. work by @xcite found that certain lexical items can be used to differentiate between some conspiratorial and non-conspiratorial texts in reddit posts and a forum popular among anti-vaccine proponents. examples of conspiracyindicative lexical items include so-called thoughtterminating cliches, such as 'agree to disagree', 'do [your/your own/the] research', and dysphemisms such as 'fraudulent', 'deceptive', and 'deceive' rather than 'lie.' attempts to identify linguistic characteristics used in conspiracy theories were explored by @xcite . they used the linguistic inquiry and word count (liwc) to analyze the conspiracy subreddit, in order to identify lexical categories based on a semantic knowledge base. in a majority of instances, conspiracy users exhibited a statistically relevant usage of words used to induce 'negative emotion' and 'anger' among others, making conspiracy texts more distinguishable. similar findings were noted within the language of conspiracy (loco) corpus @xcite . the corpus was seeded using phrases related to conspiracy theories to collect close to 100 000 text documents taken from 150 websites, dividing texts into those containing conspiratorial content and mainstream documents. a lexical analysis of conspiracy based on liwc categories and using empath, a tool that generates new lexical associations in texts, showed that conspiracy theories contain more emotionally charged language, particularly language indicating negative emotions such as anger. @xcite analyzed two conspiracy theories, sandy hook and coronavirus, in the loco corpus, to establish a set of unique features (e.g., linguistic) by which mainstream and conspiracy documents could be differentiated. they noted that a significant portion of conspiracy documents did not contain unique identifiable features, suggesting automatic classification would be difficult. they also found that mainstream documents were frequently irrelevant regarding the topic of the conspiracy theory for which they were retrieved. as new conspiracy theory corpora have been assembled, the capabilities of models to detect novel conspiracy theories have been explored. @xcite created a twitter data set covering four conspiracy topics: climate change, covid-19 origin, covid-19 vaccine, and the epstein-maxwell trial. they used several bert variants to classify tweets as conspiracy theory vs. non-ct, to identify the tweets' stance towards a conspiracy theory, and to detect the topic of the conspiracy theory. while they suggest that successful models can be built with relatively small data sets, they also note that annotator disagreement and class imbalance can contribute to difficulties in reliable classification. ## conclusion we presented a systematic set of experiments into how successfully we can classify conspiracy the-ories in both an in-domain and out-of-domain settings using different features and classifiers. results showed, unsurprisingly, that while an svm model presents strong baselines, a transformerbased model yields superior performance in both in-domain and out-of-domain settings. of more interest though is that determining good source topics for detecting out-of-domain conspiracy theories is extremely difficult and not intuitive. it remains unclear what exactly the core semantic and structural relationships between conspiracies and mainstream texts are. while bleaching too much content (replacing all words or content words by pos tags) yields poor performance, bleaching typical words per conspiracy theory is promising. one inherent difficulty that makes further indepth analysis difficult is data quality of the automatically retrieved loco documents @xcite , which may hinder the efficacy of the resulting models. however, it is also clear that conspiracy theories are not as monolithic as assumed here. research into the spread of conspiracy theories shows that people who believe in one conspiracy theory are also likely to believe in others, but not everybody believing in a ct will believe the same subset of factoids @xcite . this may also mean that the texts collected per ct are less homogeneous than necessary for classification. further research will need to investigate in more detail the inter-relatedness between different conspiracy theories. a better understanding of how they relate content-wise may allow us a better understanding of how to create a robust training set that can be used to detect conspiracy theories out of domain. additionally, we are planning to investigate better bleaching methods, along with having a closer look at the svm features that show the highest correlation with conspiracy theories, to determine defining characteristics of conspiratorial language across different domains.