Evidence-Driven Retrieval Augmented Response Generation for Online Misinformation (2024)

Abstract: The proliferation of online misinformation has posed significant threats to public interest. While numerous online users actively participate in the combat against misinformation, many of such responses can be characterized by the lack of politeness and supporting facts. As a solution, text generation approaches are proposed to automatically produce counter-misinformation responses. Nevertheless, existing methods are often trained end-to-end without leveraging external knowledge, resulting in subpar text quality and excessively repetitive responses. In this paper, we propose retrieval augmented response generation for online misinformation (RARG), which collects supporting evidence from scientific sources and generates counter-misinformation responses based on the evidences. In particular, our RARG consists of two stages: (1) evidence collection, where we design a retrieval pipeline to retrieve and rerank evidence documents using a database comprising over 1M academic articles; (2) response generation, in which we align large language models (LLMs) to generate evidence-based responses via reinforcement learning from human feedback (RLHF). We propose a reward function to maximize the utilization of the retrieved evidence while maintaining the quality of the generated text, which yields polite and factual responses that clearly refutes misinformation. To demonstrate the effectiveness of our method, we study the case of COVID-19 and perform extensive experiments with both in- and cross-domain datasets, where RARG consistently outperforms baselines by generating high-quality counter-misinformation responses.

## introduction as social media discussions on trending topics continue to grow, increased presence of online misinformation has been observed on such sources @xcite ). yet timely intervention against the spreading of misinformation is often lacking, resulting in potential threats to public interest @xcite . for instance, vaccine hesitancy is strongly associated with the exposure of antivaccine misinformation @xcite . therefore, it is essential to identify and curb the spreading of misinformation before it leads to severe consequences @xcite . to identify text-based misinformation, various approaches have been proposed by leveraging language models @xcite @xcite @xcite @xcite . upon the detection of misinformation, crowdsourcing approaches (e.g, users, fact-checkers) are designed to prevent the continued propagation of misinformation via counter-responses @xcite @xcite @xcite @xcite . nevertheless, such responses to misinformation often lack politeness, direct refutation or fail to provide supporting evidence @xcite . moreover, human efforts are often less effective when confronted with an overwhelming amount of online misinformation @xcite . to facilitate early intervention at scale, language generation methods are proposed to generate counter-responses against detected misinformation @xcite @xcite . for example, misinfocorrect adopts reinforcement learning to generate responses to refute misinformation @xcite . despite their effectiveness, existing generation methods are trained end-to-end on a specific domain. as a result, the generated texts often highly resemble the training responses and demonstrate deteriorated quality upon domain shifts (as we show in section 4). additionally, such generation models are unaware of external knowledge and must be frequently updated to incorporate up-to-date domain knowledge @xcite @xcite . as such, we consider a retrieval augmented generation (rag) setting that incorporates relevant scientific documents to utilize external evidence without retraining. that is, given input misinformation, our first objective is to retrieve evidence from an extensive collection of documents (e.g., academic publications). then, we select the relevant documents as evidence to perform retrieval augmented generation, with the goal of extracting supporting facts to debunk misinformation. we illustrate an example of our framework in figure 1 , where evidence-backed counter-response is generated upon the identification of misinformation. in this paper, we focus on counter-response generation against online misinformation and propose an evidence-driven retrieval augmented response generation (rarg) framework, which efficiently retrieves supporting documents and generates responses using the collected supporting facts. specifically, rarg consists of two modules: (1) evidence collection, in which we design a twostage retrieval pipeline based on a collection of over 1m academic articles we collected. for efficient retrieval, our pipeline first performs a coarse search over the data collection, followed by a reranking stage with improved relevance estimation of the retrieved documents; (2) evidence-based response generation, here, we align large language models (llms) to generate responses upon the collected evidence via reinforcement learning from human feedback (rlhf). in particular, we propose a reward design that maximizes the utilization of the retrieved evidence while maintaining the quality of the responses. as such, our rarg generates polite and factual responses that clearly refutes input misinformation. to the best of our knowledge, we are the first to introduce a retrieval augmented generation framework with two-stage retrieval and fine-grained rlhf. to demonstrate the effectiveness of rarg, we study the case of covid-19, where we perform both in-and cross-domain experiments with comprehensive quantitative and qualitative analyses. experimental results highlight the effectiveness of the proposed framework, where rarg consistently outperforms state-of-theart baselines by generating high-quality evidencebased responses against online misinformation. we summarize our contributions as follows: we propose a retrieval augmented generation setting for counter-misinformation response generation. in this work, we focus on covid-19 misinformation and collected over 1m academic publications as the source for evidencebased response generation. 2. we design rarg, a response generation framework against online misinformation. our framework combines two-stage retrieval for evidence collection and rlhf-based llm alignment, such that rarg is optimized to generate polite and factual counter-responses. 3. we show the effectiveness of rarg by experimenting on both in-domain and cross-domain covid misinformation to validate the generalization of rarg. both quantitative and qualitative results demonstrate that rarg can outperform state-of-the-art methods in generating high-quality responses. ## limitations despite introducing rarg for evidence-based counter-response generation against online misinformation, we have not discussed the setting where test domains significantly differ from covid (e.g., fake news), which may hinder the deployment of the proposed method for more generalized applications. in addition, we have not studied the case when the reliability of evidence sources can not be guaranteed (e.g., documents from unverifiable online sources), which may introduce inaccuracies and impact the validity of the generated responses. due to the limited research scope and budgets, we have not conducted human evaluations to assess the overall response quality of rarg. therefore, we plan to explore a more generalized and domain-adaptive solution with an improved evaluation protocol for counter-response generation in the future. besides rarg, we also report results of the supervised fine-tuned version of rarg (i.e., rarg (s)). for reward modeling, we train three bert models on the binary human feedback data w.r.t refutation, factuality and politeness. as the label distribution is imbalanced, we adopt stratified sampling to split train/test sets and apply class-balanced cross entropy loss for training. besides accuracy, f1, precision and recall, we also use balanced accuracy (ba) to evaluate class-balanced performance on human feedback classification. the evaluation results for reward modeling are in