LLM -Driven Knowledge Injection Advances Zero-Shot and Cross-Target Stance Detection (2024)

Abstract: Stance detection aims at inferring an author’s attitude towards a specific target in a text. Prior methods mainly consider target-related background information for a better understanding of targets while neglecting the accompanying input texts. In this study, we propose to prompt Large Language Models (LLMs) to explicitly extract the relationship between paired text and target as contextual knowledge. We then inject such LLM-driven knowledge into a generation model BART to exploit the rich contexts and semantics. Moreover, to further enhance the decoding capability of BART, a novel prototypical contrastive scheme is designed to align input contents with stance labels. Our experimental results demonstrate the state-of-the-art performance across several publicly available datasets, showcasing effectiveness in both zero-shot and cross-target stance detection scenarios. We publicly release our code to facilitate future research.

## introduction the objective of stance detection is to ascertain an individual's stance regarding a specified target in a text, which may either be explicitly mentioned or implied only @xcite . the significance of stance detection is evident in its role across various fields, such as predicting election and referendum outcomes @xcite , classifying rumors @xcite , detecting fake news @xcite , and identifying instances of disinformation @xcite . stance detection can be divided into three categories according to the availability of test targets: 1) target-specific: stance detection for fixed targets @xcite ; 2) cross-target: stance detection for related targets @xcite ; 3) zero-shot: stance detection for unseen targets @xcite . among these tasks, target: joe biden stance: favor text: #correct! i am a #republican and will vote for @joebiden. prior target-related knowledge: joe biden was a member of the democratic party. recently, llms, such as chatgpt released by openai foot_0 , have shown superior natural language understanding performance. in this work, we specially design the prompt to make the most of chat-gpt's internal knowledge and reasoning capabilities for extracting the relationship between texts and targets explicitly. we refer to the consequent response as llm-driven knowledge, a snippet of which can also be found in figure 1 . compared to prior ones, llm-driven knowledge is more specific and context-rich for stance detection. in order to bridge the semantic gap between llm-driven knowledge and stance labels, we adopt a bidirectional autoregressive model -bart @xcite as our backbone. as a result, the detection scheme is reformulated as a conditional stance label generation task, where the condition consists of a partially filled template, the input text, and knowledge derived from llms. this transformation allows us to go beyond direct classification and leverage rich semantics for stance decoding. based on the above framework, we further devise a prototypical contrastive learning scheme to align stance representations with label semantics. specifically, we adopt class-wise prototypes to model the feature space. then a contrastive loss is optimized, forcing stance representations to be appealed to the corresponding prototype and repelled to prototypes of other stances. as a result, per-class representations can be clustered compactly on the feature space, making it easier to learn a mapping from stance representations to label semantics. we refer to our method as llm-driven knowledge injection bart (lki-bart), and an overview is depicted in figure 2 . extensive experiments on two benchmark datasets suggest that lki-bart achieves state-of-the-art performance on zero-shot and cross-target stance detection tasks. additional ablation studies also indicate the effectiveness of each technique, namely llm-driven knowledge, bart backbone, and prototypical contrastive learning, in lki-bart. utilize llms for stance detection. recently, some works have directly used llms for stance detection. @xcite explores various methods to prompt chatgpt for stance reasoning. @xcite adopts distinct llm-based agents to create a collaborative stance detection system. also, some studies have expressed a negative opinion on using llms for stance detection. @xcite uses chatgpt to annotate stance labels but obtains inferior results. @xcite conclude that overall accuracy is not much better than supervised models. ## experiments we evaluate our lki-bart on vast @xcite and p-stance @xcite , please refer to appendix b and appendix c for details of the training datasets as well as our experimental setup. ## conclusion in this paper, we propose to collect llm-driven knowledge to incorporate connections between input texts and unseen targets for zero-shot and crosstarget stance detection. a generation framework bart is adopted to better leverage llm-driven knowledge for detection and a prototypical contrastive loss is optimized for better alignment between input materials and stance semantics. combining all the above techniques, our lki-bart finally achieves state-of-the-art performance on vast and p-stance datasets.