Stance Detection on Social Media with Background Knowledge (2023)

Abstract: Identifying users’ stances regarding specific targets/topics is a significant route to learning public opinion from social media platforms. Most existing studies of stance detection strive to learn stance information about specific targets from the context, in order to determine the user’s stance on the target. However, in real-world scenarios, we usually have a certain understanding of a target when we express our stance on it. In this paper, we investigate stance detection from a novel perspective, where the background knowledge of the targets is taken into account for better stance detection. To be specific, we categorize background knowledge into two categories: episodic knowledge and discourse knowledge, and propose a novel Knowledge-Augmented Stance Detection (KASD) framework. For episodic knowledge, we devise a heuristic retrieval algorithm based on the topic to retrieve the Wikipedia documents relevant to the sample. Further, we construct a prompt for ChatGPT to filter the Wikipedia documents to derive episodic knowledge. For discourse knowledge, we construct a prompt for ChatGPT to paraphrase the hashtags, references, etc., in the sample, thereby injecting discourse knowledge into the sample. Experimental results on four benchmark datasets demonstrate that our KASD achieves state-of-the-art performance in in-target and zero-shot stance detection.

## introduction stance detection has been essential in learning the public options from social media platforms, which aims to automatically identify the author's opinionated standpoint or attitude (e.g., favor, against, * corresponding author target: brazil world cup stance: against text: . . . in the future, the fifa world cup should only be held in countries within the top 15 gdp per capita. target: joe biden stance: favor text: the whole idea of potus . . . #voteblue #blue-wave or neutral, etc.) expressed in the content towards a specific target, topic, or proposition (somasundaran and wiebe, 2010; @xcite . existing work has achieved promising results on different types of stance detection tasks on text, based on conventional machine learning methods @xcite @xcite and deep learning methods @xcite @xcite . however, identifying a stance on social media is still challenging because the background knowledge of targets is not included in the posts, and the content often comprises implicit information in a concise format. for this reason, it is necessary to integrate background knowledge into the stance learning of the target to enhance the ability of the model's stance detection by fully understanding the target. to better exploit the background knowledge of the target, we divide it into two types: episodic knowledge and discourse knowledge. episodic knowledge @xcite refers to our understanding of a target, which is the basis for us to express our stance on a target. that is, when we express our stance on a topic, we usu-ally have a certain understanding of it. here, the episodic knowledge generally is not explicitly mentioned in the text. as the red part of the first example in in addition, in real-world social media platforms, users are accustomed to using nicknames to express certain targets. therefore, we present discourse knowledge @xcite to understand the expressions of acronyms, hashtags, slang, and references in social media texts. the blue part of the second example in incorporating background knowledge into stance detection on social media poses two major challenges. first, the required knowledge lacks ground truth labels. second, the knowledge retrieval module necessitates an in-depth comprehension of expressions to retrieve relevant background knowledge rooted in semantics and incorporate the knowledge into the original text for making stance judgments. typically, unsupervised algorithms lack these abilities. however, large language models (llms), such as chatgpt foot_0 , bard foot_1 , and llama @xcite , exhibit exceptional abilities in reading and generating text. they have been pre-trained on extensive wikipedia data and hence possess an immense amount of knowledge. in this paper, we propose knowledge-augmented stance detection framework (kasd), leveraging chatgpt to extract and inject the aforementioned two types of knowledge. we crawl wikipedia pages related to each target and develop a heuristic retrieval algorithm based on topic modeling and an instruct-based filter to obtain episodic knowledge. to incorporate discourse knowledge, we employ instruct prompting to decipher acronyms, hashtags, slang, and references in the text and rephrase the original text. we apply kasd to both fine-tuned model and large language model, and conduct experiments on four benchmark stance detection datasets. the results show that, on the fine-tuned model, kasd outperforms the baseline models which have additional designs or background knowledge incorporation. on the large language model, the results demonstrate that knowledge retrieval-augmented chatgpt based on kasd can effectively improve the performance on stance detection. additionally, we find that with kasd distilling the understanding and relevant background knowledge of the large language model, the finetuned model can achieve better results with significantly fewer parameters. the main contributions of our work are summarized as follows: ## related work incorporating episodic knowledge current retrieval methods typically employ keyword-based filtering @xcite or direct use of knowledge graphs @xcite for knowledge retrieval. however, these retrieval methods necessitate that the required background knowledge overlaps with the text, which is not always the case and could result in poor retrieval effects. @xcite introduced financial signals as background knowledge to improve the stance detection of the wtwt dataset. @xcite leveraged unannotated data with a variational auto-encoding architecture for detecting vaccine attitudes on social media. the knowledge incorporated in these works lacks generality. zhang et al. (2020) proposed a framework that extracts word-level semantic and emotional understanding to facilitate knowledge transfer across various targets. @xcite splited hashtags into individual words and employed substitute vocabulary to clarify these expressions. @xcite proposed a prompt-based contrastive learning approach to stimulate knowledge for lowresource stance detection tasks. @xcite and @xcite utilized data augmentation to enhance the model's comprehension of both text and target. @xcite introduced a background knowledge injection to modeling hashtags and targets. as experiment results reported in this paper, their approaches are suboptimal. ## methodology given @xmath0, t n } n @xmath1, where x denotes the input text and t denotes the corresponding target, the goal of stance detection is to identify the stance label y for the corresponding target t. {w 1 n , w 2 n , ..., w m n } = x n represents each word in the given text. as shown in figure 1 , we aim to retrieve multiple episodic knowledge {e i n } k @xmath2, and inject the required discourse knowledge into x n , resulting in d n . to achieve knowledge augmentation, we detect the stance of the sample using {t n , d n , e 1 n , ...e k n }. ## experimental results we conduct experiments on two different methods of stance detection: in-target stance detection, ## conclusion in this paper, we propose a knowledge-augmented stance detection (kasd) framework, providing heuristic retrieval and filtering of episodic knowledge and utilizing contextual information to inject discourse knowledge. we conduct experiments on in-target stance detection and zero-shot stance detection using four benchmark datasets. the experimental results demonstrate significant improvement in performance for both fine-tuned models and large language models utilizing kasd.