SlowGuess/NAACL_2022_OCR huggingface dataset:
Chereen Shurafa, Kareem Darwish, and Wajdi Zaghouani. 2020. Political framing: US COVID19 blame game. In Social Informatics, pages 333-351.
Jiasheng Si, Deyu Zhou, Tongzhe Li, Xingyu Shi, and Yulan He. 2021. Topic-aware evidence reasoning and stance-aware aggregation for fact verification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, ACL-IJCNLP '21, pages 1612-1622.
Valeriya Slovikovskaya and Giuseppe Attardi. 2020. Transfer learning from transformers to fake news challenge stance detection (FNC-1) task. In Proceedings of the 12th Language Resources and Evaluation Conference, LREC '20, pages 1211-1218, Marseille, France.
Peter Stefanov, Kareem Darwish, Atanas Atanasov, and Preslav Nakov. 2020. Predicting the topical stance and political leaning of media using tweets. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL '20, pages 527-537, Online.
James Thorne and Andreas Vlachos. 2018. Automated fact checking: Task formulations, methods and future directions. In Proceedings of the 27th International Conference on Computational Linguistics, COLING '18, pages 3346-3359, Santa Fe, New Mexico, USA.
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERIFICATION. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT '18, pages 809-819, New Orleans, Louisiana, USA.
James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, and Arpit Mittal. 2019. The FEVER2.0 shared task. In Proceedings of the Second Workshop on Fact Extraction and VERIFICATION, FEVER '19, pages 1-6, Hong Kong, China.
Lin Tian, Xiuzhen Zhang, Yan Wang, and Huan Liu. 2020. Early detection of rumours on Twitter via stance transfer learning. In Advances in Information Retrieval, ECIR '20, pages 575-588, Cham.
Jannis Vamvas and Rico Sennrich. 2020. X-Stance: A multilingual multi-target dataset for stance detection. In Proceedings of the 5th Swiss Text Analytics Conference (SwissText) & the 16th Conference on Natural Language Processing (KONVENS), Zurich, Switzerland.
Amir Pouran Ben Veyseh, Javid Ebrahimi, Dejing Dou, and Daniel Lowd. 2017. A temporal attentional model for rumor stance classification. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM '17, pages 2335-2338, Singapore.
Nguyen Vo and Kyumin Lee. 2020. Where are the facts? Searching for fact-checked information to alleviate the spread of fake news. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP '20, pages 7717-7731, Online.
William Yang Wang. 2017. "Liar, Liar Pants on Fire": A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL '17, pages 422-426, Vancouver, Canada.
Yongyue Wang, Chunhe Xia, Chengxiang Si, Beitong Yao, and Tianbo Wang. 2020. Robust reasoning over heterogeneous textual information for fact verification. IEEE Access, 8:157140-157150.
Penghui Wei, Nan Xu, and Wenji Mao. 2019. Modeling conversation structure and temporal dynamics for jointly predicting rumor stance and veracity. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP '19, pages 4787-4798, Hong Kong, China.
Maxwell Weinzierl, Suellen Hopfer, and Sanda M. Harabagiu. 2021. Misinformation adoption or rejection in the era of COVID-19. In Proceedings of the Fifteenth International AAAI Conference on Web and Social Media, volume 15 of ICWSM'21, pages 787-795, Virtual.
Weiming Wen, Songwen Su, and Zhou Yu. 2018. Cross-lingual cross-platform rumor verification pivoting on multimedia content. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP '18, pages 3487-3496, Brussels, Belgium.
Dustin Wright and Isabelle Augenstein. 2020. Transformer based multi-source domain adaptation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP '20, pages 7963-7974, Online.
Ruoyao Yang, Wanying Xie, Chunhua Liu, and Dong Yu. 2019. BLCU_NLP at SemEval-2019 task 7: An inference chain-based GPT model for rumour evaluation. In Proceedings of the 13th International Workshop on Semantic Evaluation, SemEval '19, pages 1090-1096, Minneapolis, Minnesota, USA.
Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Peng Li, Maosong Sun, and Zhiyuan Liu. 2020. Coreferential reasoning learning for language representation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP '20, pages 7170-7186, Online.
Takuma Yoneda, Jeff Mitchell, Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2018. UCL machine reading group: Four factor framework for fact finding (HexaF). In Proceedings of the
First Workshop on Fact Extraction and VERIFICATION, FEVER '18, pages 97-102, Brussels, Belgium.
Jianfei Yu, Jing Jiang, Ling Min Serena Khoo, Hai Leong Chieu, and Rui Xia. 2020. Coupled hierarchical transformer for stance-aware rumor verification in social media conversations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP '20, pages 1392-1401, Online.
Chen Zhao, Chenyan Xiong, Corby Rosset, Xia Song, Paul N. Bennett, and Saurabh Tiwary. 2020. Transformer-XH: Multi-evidence reasoning with extra hop attention. In Proceedings of the 8th International Conference on Learning Representations, ICLR '20, Addis Ababa, Ethiopia.
Wanjun Zhong, Jingjing Xu, Duyu Tang, Zenan Xu, Nan Duan, Ming Zhou, Jiahai Wang, and Jian Yin. 2020. Reasoning over semantic-level graph for fact checking. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL '20, pages 6170-6180, Online.
Jie Zhou, Xu Han, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2019. GEAR: Graph-based evidence aggregating and reasoning for fact verification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, ACL '19, pages 892-901, Florence, Italy.
Dimitrina Zlatkova, Preslav Nakov, and Ivan Koychev. 2019. Fact-checking meets fauxtography: Verifying claims about images. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing, EMNLP-IJCNLP '19, pages 2099-2108, Hong Kong, China.
Arkaitz Zubiaga, Ahmet Aker, Kalina Bontcheva, Maria Liakata, and Rob Procter. 2018a. Detection and resolution of rumours in social media: A survey. ACM Comput. Surv., 51(2).
Arkaitz Zubiaga, Elena Kochkina, Maria Liakata, Rob Procter, and Michal Lukasik. 2016a. Stance classification in rumours as a sequential task exploiting the tree structure of social media conversations. In Proceedings of the 26th International Conference on Computational Linguistics, COLING '16, pages 2438-2448, Osaka, Japan.
Arkaitz Zubiaga, Elena Kochkina, Maria Liakata, Rob Procter, Michal Lukasik, Kalina Bontcheva, Trevor Cohn, and Isabelle Augenstein. 2018b. Discourse-aware rumour stance classification in social media using sequential classifiers. Information Processing & Management, 54(2):273-290.
Arkaitz Zubiaga, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, and Peter Tolmie. 2016b. Analysing how people orient to and spread rumours in social media by looking at conversational threads. PLOS ONE, 11(3):1-29.
A Examples of Stance
As outlined in Section 3, there are different formulations in which the task of stance definition is materialised. In Table 3, we present some instances of these as exemplified by different stance detection datasets. The target with respect to which the stance is assessed can vary, e.g., a headline, a comment, a claim, a topic, etc., which in turn can differ in length and form. Moreover, the context where the stance is expressed can vary not only in its domain, e.g., News in (Ferreira and Vlachos, 2016) and Twitter in (Qazvinian et al., 2011), but also in its structure, as seen in the example of multiple evidence sentences in (Thorne et al., 2018) and threaded comments in (Gorrell et al., 2019).
In a more detailed view of Table 3, we see that each group of examples has its own important specifics that alter the task of stance detection for mis- and disinformation detection.
Figure 3a shows an example from the News domain, where we have a headline and an entire article body, and the goal is to find how the two are related in terms of the body's stance(s) towards the headline. In this scenario, the models need to be able to handle very long documents, on one hand, and on the other to reason over multiple fragments of the input text, which might potentially express different stances. It is possible to simplify the task by extracting a summary of the news article beforehand, and evaluating only the stance of that summary, as shown in Figure 3d. However, obtaining such summaries is not a trivial task: (a) they can be extracted by a human annotator (e.g., a journalist), which is time-consuming and expensive, and can require a priori knowledge about the headline/topic of interest as the article might have more than one highlight or viewpoint, or (b) they can be automatically generated using text summarisation methods, but the result can be noisy.
Stance is often expressed in social media such as Twitter, Facebook, Reddit, etc. We illustrate two such scenarios in Figures 3b and 3e. In contrast to the usually long and well-written news documents, social media posts are mostly short and depend on additional context such as the previous posts in a conversational thread (Figure 3e), or external URLs and implicit topics (Figure 3b). Moreover, these texts also need normalisation, as users tend to use slurs, emojis, and other informal language.
Next, in Figure 3c we highlight another interesting setup: claim verification using multiple pieces of evidence. Here, the reasoning is carried in multiple hops over a set of texts. In particular, there might not exist a single passage from a document/post that supports/refutes the claim directly. In that case, a large enough chain of evidence might be needed, which can cover enough contextual knowledge in order to allow the model (or a person) to assess the veracity of the input claim.
Finally, the examples in Figure 3 demonstrate that stance can be used for mis- and disinformation detection in different ways: (i) directly, as in the examples in Figures 3a and 3b, or (ii) as multiple viewpoints, which are later aggregated into a final decision, as in Figure 3c, 3d and 3e.
We thoroughly discussed all of the aforementioned setups in Section 3, including the publicly available datasets that focus on stance in the context of mis- and disinformation identification.
B Additional Formulations of Stance as a Component for Fact-Checking
Beyond the approaches that we outlined in Section 3.2, stance has also been used for detecting misconceptions and for profiling media sources as part of a fact-checking pipeline. Below, we describe some work that follows these formulations.
Misconceptions Hossain et al. (2020) focused on detecting misinformation related to COVID-19, based on known misconceptions listed in Wikipedia. They evaluated the veracity of a tweet depending on whether it agrees, disagrees, or has no stance with respect to a set of misconceptions. A related formulation of the task is detecting previously fact-checked claims (Shaar et al., 2020). This allows to assess the veracity of dubious content by evaluating the stance of a claim regarding already checked stories, known misconceptions, and facts.
Media Profiling Stance detection has also been used for media profiling. Stefanov et al. (2020) explored the feasibility of an unsupervised approach for identifying the political leanings (left, center, or right bias) of media outlets and influential people on Twitter based on their stance on controversial topics. They built clusters of users around core vocal ones based on their behaviour on Twitter such as retweeting, using the procedure proposed by Darwish et al. (2020). This is an important step towards understanding media biases.
4 For illustrative purposes the text is trimmed to include only the relevant passage.
Headline: Robert Plant Ripped up $800M Led Zeppelin Reunion Contract
Body: ...Led Zeppelin's Robert Plant turned down £500 MILLION to reform supergroup..
(a) Example from Pomerleau and Rao (2017)
Claim: The Rodney King riots took place in the most populous county in the USA.
Wiki Evidence 1: The 1992 Los Angeles riots, also known as the Rodney King riots were a series of riots, lootings, arson, and civil disturbances that occurred in Los Angeles County, California in April and May 1992.
Wiki Evidence 2: Los Angeles County, officially the County of Los Angeles, is the most populous county in the USA.
(c) Example from Thorne et al. (2018)
Topic: Sarah Palin getting divorced?
Tweet: OneRiot.com - Palin Denies First Dude Divorce Rumors http://url
Topic: N/A (Implicit)
Tweet:Wow, that is fascinating! I hope you never mock our proud Scandi heritage again.
(b) Examples from Qazvinian et al. (2011) and Derczynski et al. (2017)
Headline: Jess Smith of Chatham, Kent was the smiling sun baby in the Teletubbies TV show
Summary 1: Canterbury Christ Church University student Jess Smith, from Chatham, starred as Teletubbies sun
Summary 2: This College Student Claims She Was The Teletubbies Sun Baby
(d) Example from Ferreira and Vlachos (2016)
u1: We understand that there are two gunmen and up to a dozen hostages inside the cafe under siege at Sydney.. ISIS flags remain on display #7News
u2: @u1 not ISIS flags
u3: @u1 sorry - how do you know its an ISIS flag? Can you actually confirm that? ②
u4: @u3 no she cant cos its actually not
u5: @u1 More on situation at Martin Place in Sydney, AU LINK
u6: @u1 Have you actually confirmed its an ISIS flag or are you talking shit ②
(e) Example from Gorrell et al. (2019)
Table 3: Illustrative examples for different stance detection scenarios included in our survey. We annotate the expressed stance with  $\square$  (support, for),  $\square$  (deny, against),  $\square$  (query), and  $\square$  (comment).
The reliability of entire news media sources has been automatically rated based on their stance with respect to manually fact-checked claims, without access to gold labels for the overall medium-level factuality of reporting (Mukherjee and Weikum, 2015; Popat et al., 2017, 2018). The assumption in such methods is that reliable media agree with true claims and disagree with false ones, while for unreliable media, the situation is reversed. The trustworthiness of Web sources has also been studied from a data analytics perspective, e.g., Dong et al. (2015) proposed that a trustworthy source is one that contains very few false claims.
More recently, Baly et al. (2018a) used gold labels from Media Bias/Fact Check, $^{5}$  and a variety of information sources: articles published by the medium, what is said about the medium on Wikipedia, metadata from its Twitter profile, URL structure, and traffic information. In follow-up work, Baly et al. (2019) used the same representation to jointly predict a medium's factuality of reporting (high vs. mixed vs. low) and its bias (left vs. center vs. right) on an ordinal scale, in a multi-task ordinal regression setup.
Baly et al. (2020) extended the information sources to include Facebook followers and speech signals from the news medium's channel on YouTube. Finally, Hounsel et al. (2020) proposed to use domain, certificate, and hosting information about the infrastructure of the website. See (Nakov et al., 2021) for a recent survey on media profiling.
There is a well-known connection between factuality and bias. For example, hyper-partisanship is often linked to low trustworthiness (Potthast et al., 2018), e.g., appealing to emotions rather than sticking to the facts, while center media tend to be generally more impartial and also more trustworthy.
User Profiling In the case of social media and community fora, it is important to model the trustworthiness of the user. In particular, there has been research on finding opinion manipulation trolls, paid (Mihaylov et al., 2015b) or just perceived (Mihaylov et al., 2015a), sockpuppets (Maity et al., 2017; Kumar et al., 2017), Internet water army (Chen et al., 2013), and seminar users (Darwish et al., 2017).
<sup>5</sup>http://mediabiasfactcheck.com
$^{6}$ http://www.poynter.org/fact-checking/media-literacy/2021/should-you-trust-media-bias-charts/
C Systems and Applications
The systems and applications below use stance detection as part of a pipeline for identifying mis- and disinformation, see Section 4 for more details about the methods.
Wen et al. (2018) worked in a cross-lingual cross-platform rumour verification setup. They included multimodal content from fake and from real posts with images or videos shared on Twitter. They then collected supporting documents from two search engines, Google and Baidu, in English and Chinese, which they used for veracity evaluation. They trained their stance detection model on English data (FNC-1) using pre-trained multilingual sentence embeddings, and further added cross-platform features in their final neural model.
Popat et al. (2018) proposed CredEye, a system for automatic credibility assessment of textual claims. The system takes a claim as an input and analyses its credibility by considering relevant articles it retrieved from the Web, by combining the predicted stance of the articles regarding the claim with linguistic features to obtain a credibility score (Popat et al., 2017).
Nguyen et al. (2018) designed a prototype fact-checker Web tool. $^{8}$  Their system leverages a probabilistic graphical model to assess a claim's veracity taking into consideration the stance of multiple articles regarding this claim, the reputation of the news sources, and the annotators' reliability. In addition, it offers explanations to the fact-checkers based on the aforementioned features, which was shown to improve the overall user satisfaction and trust in the predictions.
Zubiaga et al. (2018a) considered a four-step tracking process as a pipeline for rumour verification: (i) rumour detection, i.e., given a stream of claims, determine whether they are worth verifying or they do contain no rumours, (ii) rumour tracking for finding relevant information about the rumour using social media posts, sentence descriptions, and keywords, (iii) stance classification to collect stances towards the rumour, and (iv) veracity classification to aggregate the information from the tracking component, the collected stances, and optionally other relevant information about the sources, metadata about the users, etc., to predict a truth value for the rumour.
Nadeem et al. (2019) developed FAKTA, a system for automatic end-to-end fact-checking of claims. It retrieves relevant articles from Wikipedia and as well as from selected media sources, which it then uses for verification. FAKTA uses a stance detection model, trained in a FEVER setting, to predict the stance and to obtain entailed spans. These predictions, combined with linguistic analysis, are used to provide both document- and sentence-level explanations and a factuality score.
Nguyen et al. (2020) proposed the Factual News Graph (FANG) model, which models the social context for fake news detection. In particular, FANG uses the stance of user comments with respect to the target news article as an integral component of its model, together with temporality, user-user interactions, article-source interactions, as well as source reliability information.
<sup>7</sup>https://gate.d5.mpi-inf.mpg.de/credeye/
<sup>8</sup>http://fcweb.pythonanywhere.com/