SlowGuess/NAACL_2022_OCR huggingface dataset:
Momchil Hardalov<sup>1,2</sup> Arnav Arora<sup>1,3</sup> Preslav Nakov<sup>1,4</sup> Isabelle Augenstein<sup>1,3</sup>
<sup>1</sup>Checkstep Research
$^{2}$ Sofia University "St. Kliment Ohridski", Bulgaria
<sup>3</sup>University of Copenhagen, Denmark
$^{4}$ Qatar Computing Research Institute, HBKU, Doha, Qatar
{momchil, arnav, preslav.nakov, isabelle}@checkstep.com
Understanding attitudes expressed in texts, also known as stance detection, plays an important role in systems for detecting false information online, be it misinformation (unintentionally false) or disinformation (intentionally false information). Stance detection has been framed in different ways, including (a) as a component of fact-checking, rumour detection, and detecting previously fact-checked claims, or (b) as a task in its own right. While there have been prior efforts to contrast stance detection with other related tasks such as argumentation mining and sentiment analysis, there is no existing survey on examining the relationship between stance detection and mis- and disinformation detection. Here, we aim to bridge this gap by reviewing and analysing existing work in this area, with mis- and disinformation in focus, and discussing lessons learnt and future challenges.
The past decade is characterized by a rapid growth in popularity of social media platforms such as Facebook, Twitter, Reddit, and more recently, Parler. This, in turn, has led to a flood of dubious content, especially during controversial events such as Brexit and the US presidential election. More recently, with the emergence of the COVID-19 pandemic, social media were at the center of the first global infodemic (Alam et al., 2021), thus raising yet another red flag and a reminder of the need for effective mis- and disinformation detection online.
In this survey, we examine the relationship between automatically detecting false information online – including fact-checking, and detecting fake news, rumors, and hoaxes – and the core underlying Natural Language Processing (NLP) task needed to achieve this, namely stance detection. Therein, we consider mis- and disinformation, which both refer to false information, though disinformation has an additional intention to harm.
Detecting and aggregating the expressed stances towards a piece of information can be a powerful tool for a variety of tasks including understanding ideological debates (Hasan and Ng, 2014), gathering different frames of a particular issue (Shurafa et al., 2020) or determining the leanings of media outlets (Stefanov et al., 2020). The task of stance detection has been studied from different angles, e.g., in political debates (Habernal et al., 2018), for fact-checking (Thorne et al., 2018), or regarding new products (Somasundaran et al., 2009). Moreover, different types of text have been studied, including social media posts (Zubiaga et al., 2016b) and news articles (Pomerleau and Rao, 2017). Finally, stances expressed by different actors have been considered, such as politicians (Johnson et al., 2009), journalists (Hanselowski et al., 2019), and users on the web (Derczynski et al., 2017).
There are some recent surveys related to stance detection. Zubiaga et al. (2018a) discuss the role of stance in rumour verification, Aldayel and Magdy (2021) survey stance detection for social media, and Küçük and Can (2020) survey stance detection holistically, without a specific focus on veracity. There are also surveys on fact-checking (Thorne and Vlachos, 2018; Guo et al., 2022), which mention, though do not exhaustively survey, stance.
However, there is no existing overview of the role that different formulations of stance detection play in the detection of false content. In that respect, stance detection could be modelled as fact-checking — to gather the stances of users or texts towards a claim or a headline (and support fact-checking or studying misinformation) —, or as a component of a system that uses stance as part of its process of judging the veracity of an input claim. Here, we aim to bridge this gap by surveying the research on stance for mis- and disinformation detection, including task formulations, datasets, and methods, from which we draw conclusions and lessons, and we forecast future research trends.
Findings of the Association for Computational Linguistics: NAACL 2022, pages 1259 - 1277 July 10-15, 2022 ©2022 Association for Computational Linguistics
Dataset	Source(s)	Target	Context	Evidence	#Instances	Task
English Datasets
Rumour Has It (Qazvinian et al., 2011)	✔	Topic	Tweet	✔	10K	Rumours
PHEME (Zubiaga et al., 2016b)	✔	Claim	Tweet	✔	4.5K	Rumours
Emergent (Ferreira and Vlachos, 2016)	国	Headline	Article*	✔	2.6K	Rumours
FNC-1 (Pomerleau and Rao, 2017)	国	Headline	Article	国	75K	Fake news
RumourEval '17 (Derczynski et al., 2017)	✔	Implicit1	Tweet	✔	7.1K	Rumours
FEVER (Thorne et al., 2018)	W	Claim	Facts	✔	185K	Fact-checking
Snopes (Hanselowski et al., 2019)	Snopes	Claim	Snippets	✔	19.5K	Fact-checking
RumourEval '19 (Gorrell et al., 2019)	✔	Implicit1	Post	✔	8.5K	Rumours
COVIDLies (Hossain et al., 2020)	✔	Claim	Tweet	国	6.8K	Misconceptions
TabFact (Chen et al., 2020)	W	Statement	WikiTable	✔	118K	Fact-checking
Non-English Datasets
Arabic FC (Baly et al., 2018b)	国	Claim	Document	国	3K	Fact-checking
DAST (Danish) (Lillie et al., 2019)	✔	Submission	Comment	✔	3K	Rumour
Croatian (Bošnjak and Karan, 2019)	国	Title	Comment	国	0.9K	Claim verifiability
ANS (Arabic) (Khouja, 2020)	国	Claim	Title	国	3.8K	Claim verification
Ara(bic)Stance (Alhindi et al., 2021)	国	Claim	Title	国	4K	Claim verification
Table 1: Key characteristics of stance detection datasets for mis- and disinformation detection. #Instances denotes dataset size as a whole; the numbers are in thousands (K) and are rounded to the hundreds. *the article's body is summarised. Sources: Twitter, @ News, Wikipedia, @ Reddit. Evidence: Single, Multiple, Thread.
2 What is Stance?
In order to understand the task of stance detection, we first provide definitions of stance and the stance-taking process. Biber and Finegan (1988) define stance as the expression of a speaker's standpoint and judgement towards a given proposition. Further, Du Bois (2007)) define stance as "a public act by a social actor; achieved dialogically through overt communicative means, of simultaneously evaluating objects, positioning subjects (self and others), and aligning with other subjects, with respect to any salient dimension of the sociocultural field", showing that the stance-taking process is affected not only by personal opinions, but also by other external factors such as cultural norms, roles in the institution of the family, etc. Here, we adopt the general definition of stance detection by Kucuk and Can (2020): "for an input in the form of a piece of text and a target pair, stance detection is a classification problem where the stance of the author of the text is sought in the form of a category label from this set: Favor, Against, Neither. Occasionally, the category label of Neutral is also added to the set of stance categories (Mohammad et al., 2016), and the target may or may not be explicitly mentioned in the text" (Augenstein et al., 2016; Mohammad et al., 2016). Note that the stance detection definitions and the label inventories vary somewhat, depending on the target application (see Section 3).
Finally, stance detection can be distinguished from several other closely related NLP tasks: (i) biased language detection, where the existence of an inclination or tendency towards a particular perspective within a text is explored, (ii) emotion recognition, where the goal is to recognise emotions such as love, anger, etc. in the text, (iii) perspective identification, which aims to find the point-of-view of the author (e.g., Democrat vs. Republican) and the target is always explicit, (iv) sarcasm detection, where the interest is in satirical or ironic pieces of text, often written with the intent of ridicule or mockery, and (v) sentiment analysis, which checks the polarity of a piece of text.
3 Stance and Factuality
Here, we offer an overview of the settings for mis- and disinformation identification to which stance detection has been successfully applied. As shown in Figure 1, stance can be used (a) as a way to perform fact-checking, or more typically, (b) as a component of a fact-checking pipeline. Table 1 shows an overview of the key characteristics of the available datasets. We include the source of the data and the target towards which the stance is expressed in the provided textual context.
<sup>1</sup>The target can either be explicit, e.g., a topic such as Public Healthcare, or implicit, where only the context is present and the target is not directly available and is usually a topic (Derczynski et al., 2017; Gorrell et al., 2019), e.g., Germanwings, or 'Prince to play in Toronto'. When the target is implicit, the task becomes similar to sentiment analysis.
(a) stance detection as fact-checking
(b) stance detection as a component of a fact-checking pipeline
Figure 1: Two stance detection formulations.
Figure 2: Types of stance. The Target is the object of the stance expressed in the Context.
We further show the type of evidence: Single is a single document/fact, Multiple is multiple pieces of textual evidence, often facts or documents, Thread is a (conversational) sequence of posts or a discussion. The final column is the type of the target Task. Finally, we present a dataset-agnostic summary of the terminology used for the different types of stance (see Figure 2), which we describe in a four-level taxonomy:  $(i)$  sources, i.e., where the dataset was collected from,  $(ii)$  inputs that represent the stance target (e.g., claim), and the accompanying context (e.g., news article),  $(iii)$  categorisation – meta-level characteristics of the input, and  $(iv)$  the textual object types for a particular stance scenario (e.g., topic, tweet, etc.). Appendix A discusses different stance scenarios with corresponding contexts and targets, with illustrations in Table 3.
3.1 Fact-Checking as Stance Detection
As stance detection is the core task within fact-checking, prior work has studied it in isolation, e.g., predicting the stance towards one or more documents. More precisely, the stance of the textual evidence(s) toward the target claim is considered as a veracity label, as illustrated in Figure 1a.
Fact-Checking with One Evidence Document Pomerleau and Rao (2017) organised the first Fake News Challenge (FNC-1) with the aim of automatically detecting fake news. The goal was to detect the relatedness of a news article's body w.r.t. a headline (possibly from another news article), based on the stance that the former takes regarding the latter. The possible categories were positive, negative, discuss, and unrelated. This was a standalone task, as it provides stance annotations only, omitting the actual "truth labels", with the motivation of assisting fact-checkers in gathering several distinct arguments pertaining to a particular claim.
Fact-Checking with Multiple Evidence Documents The FEVER (Thorne et al., 2018, 2019) shared task was introduced in 2018, aiming to determine the veracity of a claim based on a set of statements from Wikipedia. Claims can be composite and can contain multiple (contradicting) statements, which requires multi-hop reasoning, and the claim-evidence pairs are annotated as SUPPORTED, REFUTED, and NOT ENOUGH INFO. The latter category includes claims that are either too general or too specific, and cannot be supported or refuted by the available information in Wikipedia. This setup may help fact-checkers understand the decisions a model made in their assessment of the veracity of a claim, or assist human fact-checkers.
The second edition (2019) of FEVER evaluated the robustness of models to adversarial attacks, where the participants were asked to provide new examples to "break" existing models, then to propose "fixes" for the system against such attacks.
Note that FEVER slightly differs from typical stance detection, as it considers evidence supporting or refuting a claim, rather than the stance of an author towards a claim. An alternative way to look at this is in terms of argument reasoning, i.e., extracting and providing factual evidence for a claim.
FEVER also has a connection to Natural Language Inference, i.e., determining the relationship between two sentences. We view FEVER as requiring stance detection as it resembles FNC, which is commonly seen as a stance detection task.
Apart from FEVER, Hanselowski et al. (2019) presented a task constructed from manually fact-checked claims on Snopes. For this task, a model had to predict the stance of evidence sentences in articles written by journalists towards claims. Unlike FEVER, this task does not require multihop reasoning.
Chen et al. (2020) studied the verification of claims using tabular data. The TabFact dataset was generated by human annotators who created positive and negative statements about Wikipedia tables. Two different forms of reasoning in a statement are required: (i) linguistic, i.e., semantic understanding, and (ii) symbolic, i.e., using the table structure.
3.2 Stance as a (Mis-/Dis-)information Detection Component
Fully automated systems can assist in gauging the extent and studying the spread of false information online. This is in contrast to the previously discussed applications of stance detection - as a stand-alone system for detecting mis- and disinformation. Here, we review its potency to serve as a component in an automated pipeline. Figure 1b illustrates the setup, which can also include steps such as modelling the user or profiling the media outlet among others. We discuss in more detail media profiling and misconceptions in Appendix B.
Rumors Stance detection can be used for rumour detection and debunking, where the stance of the crowd, media, or other sources towards a claim are used to determine the veracity of a currently circulating story or report of uncertain or doubtful factuality. More formally, for a textual input and a rumour expressed as text, stance detection here is to determine the position of the text towards the rumour as a category label from the set {Support, Deny, Query, Comment}. Zubiaga et al. (2016b) define these categories as whether the author: supports (Support) or denies (Deny) the veracity of the rumour they are responding to, “asks for additional evidence in relation to the veracity of the rumour” (Query) or “makes their own comment without a clear contribution to assessing the veracity of the rumour” (Comment). This setup was widely explored for microblogs and social media.
Qazvinian et al. (2011) started with five rumours and classified the user's stance as endorse, deny, unrelated, question, or neutral. While they were among the first to demonstrate the feasibility of this task formulation, the limited size of their study and the focus on assessing the stance of individual posts limited its real-world applicability.
Zubiaga et al. (2016b) analysed how people spread rumours on social media based on conversational threads. They included rumour threads associated with nine newsworthy events, and users' stance before and after the rumours were confirmed or denied. Dungs et al. (2018) continued this line of research, but focused on the effectiveness of stance for predicting rumour veracity. Hartmann et al. (2019) explored the flow of (dis-)information on Twitter after the MH17 Plane Crash.
The two RumourEval (Derczynski et al., 2017; Gorrell et al., 2019) shared tasks on automated claim validation aimed to identify and handle rumours based on user reactions and ensuing conversations in social media, offering annotations for both stance and veracity. The two editions of RumourEval were similar in spirit, with the second one providing more tweets and also additionally Reddit posts. RumourEval demonstrated the importance of modelling the context of a story instead of drawing conclusions based on a single post.
Ferreira and Vlachos (2016) collected claims and news articles from rumour sites with annotations for stance and veracity by journalists as part of the Emergent project. The goal was to use the stance of a news article, summarised into a single sentence, towards a claim as one of the components to determine its veracity. A downside is the need to summarise, in contrast to FNC-1 (Pomerleau and Rao, 2017), where entire news articles were used.
Multiple languages While the above research has focused exclusively on primarily on English, interest in stance detection for other languages has started to emerge. Baly et al. (2018b) integrated stance detection and fact-checking for Arabic in a single corpus. Khouja (2020) proposed a dataset for Arabic following the FEVER setup. Alhindi et al. (2021) introduced AraStance, a multi-country and multi-domain dataset of Arabic stance detection for fact-checking. Lillie et al. (2019) collected data for stance and veracity from Danish Reddit threads Zubiaga et al. (2016b). Bošnjak and Karan (2019) studied stance detection and claim verification of comments for Croatian news articles.
In this section, we discuss various ways to use stance detection for mis- and disinformation detection, and list the state-of-the-art results in Table 2.
Fact-Checking as Stance Detection Here, we discuss approaches for stance detection in the context of mis- and disinformation detection, where veracity is modelled as stance detection as outlined in Section 3.1. One such line of research is the Fake News Challenge, which used weighted accuracy as an evaluation measure (FNC score), to mitigate the impact of class imbalance. Subsequently, Hanselowski et al. (2018a) criticized the FNC score and F1-micro, and argued in favour of F1-macro (F1) instead. In the competition, most teams used hand-crafted features such as words, word embeddings, and sentiment lexica (Riedel et al., 2017; Hanselowski et al., 2018a). Hanselowski et al. (2018a) showed that the most important group of features were the lexical ones, followed by features from topic models, while sentiment analysis did not help. Ghanem et al. (2018) investigated the importance of lexical cues, and found that report and negation are most beneficial, while knowledge and denial are least useful. All these models struggle to learn the Disagree class, achieving up to 18 F1 due to major class imbalance. In contrast, Unrelated is detected almost perfectly by all models (over 99 F1). Hanselowski et al. (2018a) showed that these models exploit the lexical overlap between the headline and the document, but fail when there is a need to model semantic relations or complex negation, or to understand propositional content in general. This can be attributed to the use of  $n$ -grams, topic models, and lexica.
Mohtarami et al. (2018) investigated memory networks, aiming to mitigate the impact of irrelevant and noisy information by learning a similarity matrix and a stance filtering component, and taking a step towards explaining the stance of a given claim by extracting meaningful snippets from evidence documents. Like previous work, their model performs poorly on the Agree/Disagree classes, due to the unsupervised way of training the memory networks, i.e., there are no gold snippets justifying the document's stance w.r.t. the target claim.
More recently, transfer learning with pre-trained Transformers has been explored (Slovikovskaya and Attardi, 2020), significantly improving the performance of previous state-of-the-art approaches.
Guderlei and Aßenmacher (2020) showed the most important hyper-parameter to be learning rate, while freezing layers did not help. In particular, using the pre-trained Transformer RoBERTa improved F1 from 18 to 58 for Disagree, and from 50 to 70 for Agree. The success of these models is also seen in cross-lingual settings. For Arabic, Khouja (2020) achieved 76.7 F1 for stance detection on the ANS dataset using mBERT. Similarly, Hardalov et al. (2022) applied pattern-exploiting training (PET) with sentiment pre-training in a cross-lingual setting showing sizeable improvements on 15 datasets. Alhindi et al. (2021) showed that language-specific pre-training was pivotal, outperforming the state of the art on AraStance (52 F1) and Arabic FC (78 F1).
Some formulations include an extra step for evidence retrieval, e.g., retrieving Wikipedia snippets for FEVER (Thorne et al., 2018). To evaluate the whole fact-checking pipeline, they introduced the FEVER score – the proportion of claims for which both correct evidence is returned and a correct label is predicted. The top systems that participated in the FEVER competition Hanselowski et al. (2018b); Yoneda et al. (2018); Nie et al. (2019) used LSTM-based models for natural language inference, e.g., enhanced sequential inference model (ESIM Chen et al. (2017)). Nie et al. (2019) proposed a neural semantic matching network, which ranked first in the competition, achieving 64.2 FEVER score. They used page view frequency and WordNet features in addition to pre-trained contextualized embeddings (Peters et al., 2018).
More recent approaches used bi-directional attention (Li et al., 2018), a GPT language model (Malon, 2018; Yang et al., 2019), and graph neural networks (Zhou et al., 2019; Atanasov et al., 2019; Liu et al., 2020b; Wang et al., 2020; Zhong et al., 2020; Weinzierl et al., 2021; Si et al., 2021). Zhou et al. (2019) showed that adding graph networks on top of BERT can improve performance, reaching 67.1 FEVER score. Yet, the retrieval model is also important, e.g., using the gold evidence set adds 1.4 points. Liu et al. (2020b); Zhong et al. (2020) replaced the retrieval model with a BERT-based one, in addition to using an improved mechanism to propagate the information between nodes in the graph, boosting the score to 70. Recently, Ye et al. (2020) experimented with a retriever that incorporates co-reference in distant-supervised pre-training, namely, CorefRoBERTa.
Wang et al. (2020) added external knowledge to build a contextualized semantic graph, setting a new SOTA on Snopes. Si et al. (2021) and Ostrowski et al. (2021) improved multi-hop reasoning using a model with eXtra Hop attention (Zhao et al., 2020), a capsule network aggregation layer, and LDA topic information. Atanasova et al. (2022) introduced the task of evidence sufficiency prediction to more reliably predict the NOT ENOUGH INFO class.
Another notable idea is to use pre-trained language models as fact-checkers based on a masked language modelling objective (Lee et al., 2020), or to use the perplexity of the entire claim with respect to the target document (Lee et al., 2021). Such models do not require a retrieval step, as they use the knowledge stored in language models. However, they are prone to biases in the patterns used, e.g., they can predict date instead of city/country and vice-versa when using "born in/on". Moreover, the insufficient context can seriously confuse them, e.g., for short claims with uncommon words such as "Sarawak is a ..." where it is hard to detect the entity type. Finally, the performance of such models remains well below supervised approaches; even though recent work shows that few-shot training can improve results (Lee et al., 2021).
Error analysis suggests that the main challenges are (i) confusing semantics at the sentence level, e.g., "Andrea Pirlo is an American professional footballer." vs. "Andrea Pirlo is an Italian professional footballer who plays for an American club.", (ii) sensitivity to spelling errors, (iii) lack of relation between the article and the entities in the claim, (vi) dependence on syntactic overlaps, e.g., "Terry Crews played on the Los Angeles Chargers." (NotEnoughInfo) is classified as refuted, given the sentence "In football, Crews played ... for the Los Angeles Rams, San Diego Chargers and Washington Redskins, ...," (v) embedding-level confusion, e.g., numbers tend to have similar embeddings, "The heart beats at a resting rate close to 22 bpm." is not classified as refuted based on the evidence sentence "The heart beats at a resting rate close to 72 bpm," and similarly for months.