Stance Detection in COVID -19 Tweets (2021)

Abstract: The prevalence of the COVID-19 pandemic in day-to-day life has yielded large amounts of stance detection data on social media sites, as users turn to social media to share their views regarding various issues related to the pandemic, e.g. stay at home mandates and wearing face masks when out in public. We set out to make use of this data by collecting the stance expressed by Twitter users, with respect to topics revolving around the pandemic. We annotate a new stance detection dataset, called COVID-19-Stance. Using this newly annotated dataset, we train several established stance detection models to ascertain a baseline performance for this specific task. To further improve the performance, we employ self-training and domain adaptation approaches to take advantage of large amounts of unlabeled data and existing stance detection datasets. The dataset, code, and other resources are available on GitHub.

## introduction we live in unprecedented times caused by a global covid-19 pandemic, which has forced major changes in our daily lives. given the developments concerning covid-19, communities and governments need to take appropriate action to mitigate the effects of the novel coronavirus, which is at the root of the pandemic. for example, states in the united states that have imposed strict social distancing mandates were able to slow the growth of the virus within their communities @xcite . for such measures to work, however, it is important that the public fully adhere to these guidelines and mandates. "pandemic fatigue," or when people become tired of pandemic mandates and begin to ease in adherence, can lead to resurgences of the novel coronavirus @xcite . to reduce the spread of covid-19, it is essential to understand the public's opinion on the various initiatives, such as stay at home orders, wearing a face mask in public, school closures, etc. understanding how the public feels about these mandates could help health officials better estimate the expected efficacy of their mandates, as well as detect pandemic fatigue before it leads to a serious resurgence of the virus. in the era of web 2.0, and especially during a pandemic in which people often resort to online communications, social media platforms provide an astounding amount of data relating to the stance and views held by various populations with respect to a variety of current and important topics. however, the total amount of data that is being generated each second makes it impossible for humans alone to fully make use of them. fortunately, recent developments in deep learning have yielded stateof-the-art performance in text classification.this makes deep learning an ideal solution for extracting and making sense of the large amounts of data currently in circulation on social media sites. in particular, given the current events, it is evident that automated approaches for detecting the stance of the population towards targets, such as health mandates related to covid-19, using twitter posts, or tweets, can help gauge the level of cooperation with the mandates. stance detection is a natural language processing (nlp) task in which the goal is for a machine to learn how to automatically determine from text alone an author's stance, or perspective/view, towards a controversial topic, or target. research in the area of stance detection has yielded accurate results, especially in the united states politics @xcite @xcite . however, research on stance detection for targets relevant to covid-19 health mandates lags behind, due to the recency of the pandemic and a lack of benchmark datasets. we set out to address this problem by constructing a covid-19 stance detection dataset (called covid-19-stance), which includes tweets that express views towards four targets, specifically "anthony s. fauci, m.d.", "keeping schools closed", "stay at home orders", and "wearing a face mask." this is a challenging task, which is related but different from sentiment analysis. a tweet may express support for a target, while using a negative language, and expressing a negative sentiment overall. furthermore, the opinion expressed in a tweet may not be explicitly towards the target of interest, while the stance can be implicitly inferred. some examples of tweet/target pairs labeled with respect to stance, target of opinion and sentiment are shown in ## related work we discuss related work in terms of existing datasets and approaches for stance detection. ## covid-19-stance dataset the recency of the covid-19 pandemic means there was no established stance detection dataset for this broader topic, when we began our research. therefore, we set out to construct our own dataset, called covid-19-stance, by following the methodology introduced by @xcite , which is generic and applicable for any controversial topic discussed on twitter. data collection. we began crawling twitter, using the twitter streaming api, on february 27 th , 2020. we collected tweets that contained general keywords pertaining to the novel coronavirus (e.g. "coronavirus", "covid-19", "corona virus", "#covid19", etc.). as new hashtags emerged, we iteratively added additional, more specific keywords to the search (e.g., "#lockdown", "stay at home", "#socialdistancing", "#washhands", etc.). we continued crawling until august 20 th , 2020. the full list of keywords that was used over this time period is provided in appendix a. we only stored original tweets (not a retweet or quoted tweet) that contained no hyperlinks, and ended up collecting a grant total of 30,331,993 tweets. target selection. after being able to analyze the initial tweets, and following the developments of the covid-19 events, we began to identify controversial topics that arose as the virus continued its spread in the united states (us). four topics that we found to be among the most prevalent in our collection of tweets, and are understood by a large number of people in the us, were "stay at home orders", "wearing a face mask", "keeping schools closed", and "anthony s. fauci, m.d.". ## baseline models having described our covid-19-stance dataset, we now briefly review several models that we use to establish baseline results on this dataset. ## results and discussion the results of the experiments are shown in ## conclusions and future work in this work, we have constructed a covid-19-stance dataset that can be used to further the research on stance detection, especially in the context of covid-19 pandemic. in addition to the dataset, we have established baselines using several supervised models used in prior works on stance detection, and also two models that can make use of unlabeled data and data from a prior stance detection task, respectively. our results show the pre-trained covid-twitter-bert model constitutes a strong baseline. when a larger amount of labeled data is available for a target, the bert-ns and bert-dan can help further improve the performance. as part of future work, we plan to study the benefits of the opinion and sentiment data that we annotated towards the stance detection. we also plan to study the usefulness of multi-task learning, where we train models for all our targets concurrently. other transfer learning approaches that can leverage existing datasets will also be explored. home, herdimmunity, coronaviruslockdownuk, trumpvirus2020, trumpburialpits, shutit-down, 5gcoronavirus, homeoffice, resistance, chineseviruscorona, chinesevirus, panicbuyinguk, kungflu, nyclockdown, facemask, trumpandemic, coronahoax, homeoffice, chi-nesecoronavirus, pandumbic, coronalockdown, openamericanow, togetherathome, testing, feverdetectioncamera, wherearethetests, vaccines, plandemic, scamdemic, firefauci, studentlivesmatter, stayathome, endthelockdown, reopenamerica, lockdown2020, cance-lapexamspromotestudents, schoolreopening, healthoverexams, promotestudentssavefuture, testingtestingtesting, schools, lockdownuknow, saferathome, contacttracing, freethemall, trumpcoronavirustestfailure, trumpliedamer-icansdied, handwashing, chinaliedpeopledie, stayathomeorder, openamerica, vaccine, remoteworking, californialockdown, testtraceisolate, endtheshutdown, wholiedpeopledied, curfew, curfew, reopenamerica, testing, testvirusnow, socialdistance, plandemic, fakepandemic, stayhomestaysafe, trumppan-demicfailure, backtowork, backtowork, chinavirus, reopenamericanow, makechi-napay, testandtrace,#masksoff, masksoff, saynotomasks, #saynotomasks, constitu-tionovercoronavirus, #constitutionovercoronavirus, endthelockdownuk, #endthelockdownuk, #studentban, #studentban, schoolsmustopen-infall, schoolreopening, reopeningschools, #schoolsmustopeninfall, #schoolreopening, #reopeningschools, #hydroxychloroquine, hydroxychloroquine b questionnaire used for amazon mechanical turk workers q1: from reading the tweet, which of the options below is most likely to be true about the tweeter's stance or outlook towards stance to prevent the spread of covid-19: q2: from reading the tweet, which of the options below is most likely to be true about the focus of opinion/sentiment in the tweet: q3: what kind of language is the speaker using?