Reinforcement Tuning for Detecting Stances and Debunking Rumors Jointly with Large Language Models (2024)

Abstract: Learning multi-task models for jointly detecting stance and verifying rumors poses challenges due to the need for training data of stance at post level and rumor veracity at claim level, which are difficult to obtain. To address this issue, we leverage large language models (LLMs) as the foundation annotators for the joint stance detection (SD) and rumor verification (RV) tasks, dubbed as JSDRV. We introduce a novel reinforcement tuning framework to enhance the joint predictive capabilities of LLM-based SD and RV components. Specifically, we devise a policy for selecting LLM-annotated data at the two levels, employing a hybrid reward mechanism to choose high-quality labels for effective LLM fine-tuning on both tasks. Results demonstrate that JSDRV improves the capabilities of LLMs in the joint tasks, not only outperforming state-of-the-art methods but also generalizing to non-LLMs accommodated as task models.

## introduction social media has transformed the ways people access information by facilitating rapid information sharing. however, it has become a fertile ground for nurturing rumors and misinformation due to its lack of systematic moderation @xcite . their rampant spread has become a considerable global societal issue that can profoundly impact people's beliefs and normal life @xcite . in general, stance provides insights into the attitudes, opinions, and beliefs of individuals regarding a specific target @xcite . stance and rumor are deeply coupled since the stance expressed by social media posts toward a rumorous event, i.e., rumor stance, offers valuable cues for assessing the overall credibility of the target claim. figure 1 shows p3: she only wants a divorce cause she knows justin is available. early research has found that detecting stances expressed in related conversation threads on social media is beneficial for rumor detection and monitoring @xcite @xcite . nowadays, researchers are increasingly focusing on leveraging related stances for improving rumor detection and verification effectiveness @xcite @xcite @xcite as well as performing stance-rumor joint detection tasks through multi-task learning @xcite @xcite @xcite . however, most existing approaches require extensive post-level stance labels and claim-level veracity labels for training stance detection and rumor verification models, respectively, which are very expensive to obtain. while some "for free" unsupervised @xcite @xcite meth-ods were developed, they are only for the stance detection task and suffer from poor generalizability due to reliance on crafted features or specific models pre-trained on the data (e.g., online debate) unmatched with rumor stances from social media. more recently, a weakly supervised method @xcite based on multiple instance learning @xcite is proposed to predict the stance of an individual post and the rumor veracity of the claim only being supervised with claim veracity labels. however, determining post stances via dispersing the veracity of a source claim down to stances of many individual posts, as illustrated by figure 1 , is intuitively much more challenging than inferring a source claim's veracity via aggregating post stances in an opposite direction. the emergence of disruptive technologies for large language models (llms) such as chat-gpt @xcite and smaller variants such as llama @xcite and al-paca @xcite have shown performance and explainability on a par with human or even better in various nlp tasks. however, their abilities in rumor-stance related tasks within social media context are still understudied, especially for the joint prediction for detecting posts stances and debunking rumors from claims at the same time. on one hand, llms might suit such tasks thanks to their rich pre-trained knowledge and strong zeroand few-shot capabilities; on the other hand, their predictive power based on a large number of rumorrelated social media posts might be compromised by noise, unreliability, and lack of supervision. to study and unleash the potential of llms for this joint task, we propose a reinforcement tuning framework for joint stance detection and rumor verification (jsdrv) based on llms foot_0 . the framework contains three complementary parts: the llm stance detection (sd) network, the reinforcement label selector, and the llm rumor verification (rv) network. specifically, assuming merely a small set of seeding veracity labels at claim level, the reinforcement selector chooses high-quality examples for fine-tuning the sd and rv llms based on their generated labels and explanations. we present an end-to-end joint optimization mechanism to boost the integrated framework. our contributions are summarized as follows: ## related work rumor verification. early studies on rumor verification train supervised classifiers by utilizing content @xcite or contextual features @xcite extracted from claim and related posts from social media. nowadays, most methods predominately focus on utilizing neural networks such as rnn @xcite , cnns @xcite , tree-/graph-based @xcite @xcite , transformer-based @xcite @xcite ) and adversarial contrastive @xcite models. researchers also find that stances towards the specific claim shared among social media users, can assist rumor verification by revealing crucial cues and dissemination patterns @xcite @xcite , leading to more recent stance-aware approaches for rumor verification @xcite @xcite @xcite . recently, research has leveraged pre-trained language models (plms) including llms for misinformation-related tasks such as fact checking @xcite @xcite @xcite . @xcite proposed zero-shot prompt learning for rumor detection using a multilingual plm addressing diverse languages and domains on social media. little has been done on using llms for rumor stance detection and verification in social media contexts. stance detection. stance detection, initially relies on hand-crafted features @xcite , and later has advanced to use deep learning @xcite @xcite . subsequent research has explored incorporating propagation structure @xcite . recent approaches delve into reinforcement learning @xcite , contrastive learning @xcite and teacher-student models @xcite . yet these methods require large annotated corpora for training. to address this limitation, unsupervised @xcite @xcite and weakly supervised model @xcite have emerged, albeit with concerns about their weak detection efficacy and generalizability. plms elevate stance detection by utilizing variants of bert @xcite , setting new standards this task @xcite . chatgpt demonstrates its accuracy in stance detection @xcite via zero-shot and few-shot prompt engineering. rumor-stance dual task. the rumor-stance dual task commenced from rumoreval shared task series @xcite as a two-step pipeline, where rumor verification (subtask b) performs veracity prediction based on the claim and sdqc stances of the posts classified in stance detection (subtask a). then, joint detection has been studied through multi-task learning @xcite @xcite @xcite . however, such approach is fully supervised by large training sets labeled for claim veracity and posts stance. @xcite proposed a weakly supervised neural model with multiple instance learning only using a full set of veracity-labeled claims for training. we assume only a small set of training examples at claim level as seeds for llm-based annotation, and learn a policy to select high-quality annotations for fine-tuning sd and rv llms. ## problem statement we define a rumor dataset as @xmath0, x i )} |c| @xmath1, where each instance (c i , x i ) is a tuple consisting of a source claim c i and a conversation thread of posts responding to c i denoted as x @xmath2,1 , x i,2 , • • • , x i,t }. the posts are presented in a chronological order while reply structure may exist via '@user' symbol in the text. we define the dual tasks as follows: traditionally, the ground-truth of y i,1 y i,2 • • • y i,t and y i of all training instances are assumed available for full supervision @xcite @xcite @xcite , or only y i of each training instance is available for weak supervision @xcite . in contrast, we target a more challenging setting, where only a small set of seeding claims c ′ ∈ c (|c ′ |≪ |c|) are provided with veracity labels, while no post stance is provided for training. ## our jsdrv framework using zero-/few-shot prompting @xcite and parameter-efficient fine-tuning (peft) @xcite @xcite techniques, llms can perform on a par with or even better than traditional supervised models. however, using llms for the stance-rumor joint tasks faces two major issues: 1) human labels, especially stance labels of posts in social conversation about a claim, are difficult to obtain. even if they were abundant, it could be hard to fully utilize them for fine-tuning due to the limit of computing resources; 2) while one could use llms to annotate data in scale, llm-provided labels may be unreliable, subject to further refinement for quality. thus, how to label and select high-quality instances becomes paramount for effective prompting or fine-tuning. we propose a reinforcement tuning framework to perform data annotation, selection, and model fine-tuning for the dual tasks, based on a small set of seeding claims with veracity labeled manually. by iteratively refining the selection policy with llm's annotations, the model prioritizes instances that align with the desired task objectives. with heterogeneous reward functions based on limited ground-truth data, jsdrv enhances the instance selection process, ultimately boosting the overall performance of both tasks. figure 2 illustrates our end-to-end framework, encompassing an sd network, a reinforcement selection policy model, and an rv network. we adopt llama-2 7b version 2 @xcite , an open-source llm for both sd and rv networks, which can be fine-tuned with peft techniques such as lora @xcite . ## conclusion and future work we introduced a novel reinforcement tuning framework, jsdrv, for joint stance detection and rumor verification tasks based on llms. by leveraging llms as annotators and employing a reinforcement label selector, we effectively fine-tune the llms for both tasks supervised only with minimal human-labeled claims. our experiments on benchmark datasets demonstrated the superiority of jsdrv over existing methods, suggesting its promising potential for addressing the challenges of joint stance detection and rumor verification in social media environments. our future work aims to develop unsupervised methods for both tasks.