COVIDL ies: Detecting COVID -19 Misinformation on Social Media (2020)

Abstract: The ongoing pandemic has heightened the need for developing tools to flag COVID-19-related misinformation on the internet, specifically on social media such as Twitter. However, due to novel language and the rapid change of information, existing misinformation detection datasets are not effective for evaluating systems designed to detect misinformation on this topic. Misinformation detection can be divided into two sub-tasks: (i) retrieval of misconceptions relevant to posts being checked for veracity, and (ii) stance detection to identify whether the posts Agree, Disagree, or express No Stance towards the retrieved misconceptions. To facilitate research on this task, we release COVIDLies ( https://ucinlp.github.io/covid19 ), a dataset of 6761 expert-annotated tweets to evaluate the performance of misinformation detection systems on 86 different pieces of COVID-19 related misinformation. We evaluate existing NLP systems on this dataset, providing initial benchmarks and identifying key challenges for future models to improve upon.

## introduction detecting spread of misinformation such as, rumors, hoaxes, fake news, propaganda, spear phishing, and conspiracy theories, is an important task for natural language processing @xcite @xcite . online social media networks provide particularly fertile ground for the spread of misinformationthey lack gate-keeping and regulations, users publish content without having to go through an editor, peer review, verification of qualification, or providing sources, and social networks tend to create "echo chambers" or closed networks of communication insulated from disagreements. tweet: "coronavirus cv19 was a top secret biological warfare experiment. that is why it is only affecting the poor." misconception: "coronavirus is genetically engineered." label: agree tweet: "it looks like we are all going to have to wait much longer for a #covid19 vaccine." misconception: "we're very close to a vaccine." label: disagree tweet: "cdc: coronavirus spreads rapidly in dense populations with public transit and regular social gatherings." misconception: "coronavirus cannot live in warm and tropical temperatures." label: no stance figure 1 : covidlies dataset. given a tweet, we annotate whether any of the known misconceptions are expressed in the tweet, in particular, if the tweet spreads the misconception (e.g., they agree), combats the spread of the misconception (e.g., they disagree), or takes no stance towards the misconception. the covid-19 pandemic has created a pressing need for tools to combat the spread of misinformation. since the pandemic affects the global community, there is a wide audience seeking information about the topic, whose safety is threatened by adversarial agents invested in spreading misinformation for political and economic reasons. furthermore, due to the complexity of medical and public health issues, it is also difficult to be completely accurate and factual, leading to disagreements that get exacerbated with misinformation. this difficulty is compounded by the rapid evolution of knowledge regarding the disease. as researchers learn more about the virus, statements that seemed true may turn out to be false, and vice versa. detecting this spread of pandemic-related misinformation, thus, has become a critical problem, receiving significant attention from government and public health organizations (who, 2020), social media platforms (techcrunch, 2020), and news agencies (bbc, 2020; cnn, 2020; new york times, 2020). tweet t misconception retrieval agree disagree no stance known misconceptions relevant misconceptions misconception m stance detection classification in this paper, we introduce the covidlies dataset for misconception detection on twitter. covidlies comprises of 86 common misconceptions about covid-19 along with 6761 related tweets, identified and annotated by researchers from the uci school of medicine. given a tweet, we annotate whether any of the known misconceptions, curated by the researchers, are expressed by the tweet. if they are not, then they are considered no stance. if they are, we further identify whether the tweet propagates the misconception (agree) or is informative by contradicting it (disagree). example misconception-tweet pairs for each label are illustrated in figure 1 . we provide benchmark results for existing nlp models for this task. first, we evaluate text similarity models on their ability to detect whether a tweet is relevant to a given misconception (a.k.a misconception retrieval). following prior work on fact verification @xcite and fake news detection @xcite , we evaluate nli models on misinformation (a.k.a. stance detection), by equating the class labels agree, disagree, and no stance to entailment, contradiction, and neutral, respectively. our results show that existing models struggle at both tasks (38.7 hits@1 for retrieval and 32.5 macro f1 on stance detection), however improve considerably after domain adaptation (gururangan et al. (2020); 61.3 hits@1 for retrieval and 50.2 macro f1 on stance detection). while our initial results using domain adaptation are encouraging, they leave much room for improvement. there is still much work that needs to be done before nlp systems can be seriously considered for combating covid-19-related misinformation, and we hope covidlies will be useful to help researchers understand when such systems are ready to be deployed. ## problem setup we assume access to a collection of positively phrased known misconceptions @xmath0, . . . , m |m | }, e.g., "wearing masks does not prevent spread of covid-19." is a misconception. as we describe later, the set of misconceptions in this work are vetted, curated, and maintained by medical researchers. given a collection of tweets, @xmath1, . . . , t |t | }, the task is to determine, for each input t, whether there exists a misconception @xmath2, and if so, whether the discussion propagates the misconception (i.e., identifies m as true, and thus is spreading the misconception) or refutes the misconception (i.e., identifies m as false). this task is naturally separated into the following steps (shown in figure 2 ): due to limited availability of labeled data specific to this problem, we expect that models will need to be supervised on other, related tasks. for misconception retrieval, for example, relevant misconceptions can be ranked by measuring the semantic similarity between the tweet and each misconception, e.g., using cosine similarity between average word embeddings or more recent transformerbased methods such as bertscore @xcite . for the stance detection sub-task, the problem can be recast as natural language inference (nli), mapping the tweet t to the premise, the misconception m to the hypothesis, and the agree, disagree, and no stance labels to entailment, contradiction, and neutral, respectively. due to novel language used to describe the disease and its associated misconceptions, existing misinformation detection dataset are unlikely to be effective for evaluating systems designed to detect covid-19-related misinformation on social media. to facilitate research on this problem, we collect an evaluation dataset, covidlies; the collection process is described below. misconceptions we extract misconceptions from a wikipedia article about misinformation related to the covid-19 pandemic @xcite . the extracted statements are manually examined, and statements that are not misinformation are removed. we manually rephrase the misinformation statements to a positive expression of that misinformation, e.g. "some conspiracy theorists also alleged that the coronavirus outbreak was cover-up for a 5g-related illness" is shortened to "coronavirus is caused by 5g". tweets our source of tweets is the collection of covid-19-related tweets identified by @xcite . we only use tweets from march and april 2020, and filter out non-english tweets. annotation process to help identify tweets related to our list of misconceptions, we use bertscore @xcite to compute a similarity metric on tweet-misconception pairs. for each given misconception, the 100 most similar tweets are selected for annotation. each of these tweet-misconception pairs is manually labeled by researchers in the uci school of medicine as either: agree (tweet is a positive expression of the misconception), disagree (tweet contradicts/disagrees with the misconception), or no stance (tweet is neutral or not relevant to the misconception). annotation quality to evaluate inter-rater reliability, we randomly chose a subset of 200 tweetmisconception pairs and had four researchers manually label the subset. percent agreement between researchers was 79%. fleiss kappa score was 0.69 which indicates substantial agreement between researchers (0.61-0.8). disagreements were discussed and resolved before continuing to label the remaining tweet-misconception pairs in the dataset. most disagreements came down to labeler interpretation. for example, given the misconception, "drinking large amounts of water will protect against coronavirus", and a tweet of "it's a good thing everyone is stocking up on water to survive the coronavirus because the 128 oz of diet coke, the double cheeseburgers, and radiation from our phones definitely won't kill us first", one researcher labeled the pair as no stance because it does not address any protective benefits but another researcher labeled the pair as agree because people were stocking up on water to survive coronavirus, which, to that researcher, implied water was protective. after discussions among the researchers, we concluded the pair was no stance as it did not implicitly address the benefits of water and the statement was stated in a sarcastic tone. other labeling challenges included deciding whether or not links or images in the tweet should be taken into account, as these could potentially change context of the tweet. we concluded that we would only evaluate the text as is since the various models would not be able to take images and links into account. ## performance of benchmark models supervised classifiers have been used extensively for detecting misinformation @xcite @xcite . however, existing tasks involve static or slowly evolving domains, and topics that do not require specific expertise to annotate. gathering an annotated dataset large enough to be used for training a covid-19 misinformation detector is difficult: the way misconceptions are expressed rapidly evolves, and identifying whether or not something is a misconception requires expertise in public health and medicine. further, even the misconceptions themselves change over time as we learn more about the disease and the pandemic. thus, it is desirable that covid-19 misinformation detection systems are: (i) data efficient, e.g., trained with little to no supervision, and (ii) flexible, e.g., allow the addition, removal, or modification of the known misconceptions. in this section, we investigate whether models trained for related tasks in natural language processing can be adapted to misinformation detection on the covidlies dataset without additional training. we specifically focus on models that can be used to score two input sequences, i.e., tweet-misconception pairs. because these models come pretrained on different tasks, they are naturally data efficient, and furthermore, due to their pairwise nature, are also flexible as modification of supported misconceptions is performed at the input level. our code, dataset, and a demo of our best performing system are all available at https://ucinlp.github.io/covid19 .