DELL : Generating Reactions and Explanations for LLM -Based Misinformation Detection (2024)

Abstract: Large language models are limited by challenges in factuality and hallucinations to be directly employed off-the-shelf for judging the veracity of news articles, where factual accuracy is paramount. In this work, we propose DELL that identifies three key stages in misinformation detection where LLMs could be incorporated as part of the pipeline: 1) LLMs could generate news reactions to represent diverse perspectives and simulate user-news interaction networks; 2) LLMs could generate explanations for proxy tasks (e.g., sentiment, stance) to enrich the contexts of news articles and produce experts specializing in various aspects of news understanding; 3) LLMs could merge task-specific experts and provide an overall prediction by incorporating the predictions and confidence scores of varying experts. Extensive experiments on seven datasets with three LLMs demonstrate that DELL outperforms state-of-the-art baselines by up to 16.8% in macro f1-score. Further analysis reveals that the generated reactions and explanations are greatly helpful in misinformation detection, while our proposed LLM-guided expert merging helps produce better-calibrated predictions.

## introduction large language models (llms) have demonstrated impressive capabilities to follow instructions @xcite , perform knowledgeintensive tasks @xcite , and confront societal challenges @xcite . however, llms are also hindered by hallucinations @xcite @xcite , lack of factuality @xcite , and challenges to adapt to new knowledge @xcite . despite preliminary efforts @xcite , llms cannot yet be employed off-the-shelf for analyzing the veracity of news articles where factual accuracy is paramount @xcite . together with emerging risks of generating misinformation at scale @xcite , these limitations call for new solutions to leverage llms to counter online fake news and misinformation campaigns. while llms are not reliable in detecting misinformation directly, we propose dell foot_0 , employing three key stages where llms could be integrated to provide more context and explanations for reliable assessment of news veracity (figure 1 ): diverse reaction generation explainable proxy tasks llm-based expert ensemble gender: female age: 18-29 ethnicity: white political leaning: republican … a fabricated news story regarding the highly renowned hollywood superstar denzel washington, allegedly praising the president-elect of the united states, … user profile news news sentiment detection news / gnn stance detection news / gnn knowledge retrieval news gnn entities prediction … prediction confidence … prediction +confidence expert selection final pred. (c) selective (b) confidence (a) vanilla user-news interaction network graph structure … … : i'm surprised … : i agree… : this news… : exciting news… : as a … : glad to… 0.61 0.33 0.97 figure 1 : overview of dell. we first employ llms to generate news reactions from diverse perspectives and form user-news interaction networks. we then design six explainable proxy tasks to refine the feature embeddings with llm-generated explanations. we finally propose three llm-based strategies to selectively merge the predictions of task-specific experts and enhance calibration. we conduct extensive experiments to evaluate dell and state-of-the-art baselines with three llms on seven datasets spanning three tasks related to news veracity, featuring both humanwritten and machine-generated misinformation. dell outperforms the strongest baseline across all datasets, achieving an improvement of up to 16.8% in macro f1-score. further analysis reveals that llm-generated news reactions and explanations to proxy tasks contribute greatly to model performance, while the llm-guided expert merging results in better-calibrated misinformation detectors for both human-and machine-generated news. ## methodology we propose three strategies to integrate llms in evaluating news veracity: (i) diverse reaction generation, leveraging llms to generate synthetic news reactions from diverse perspectives and forming networks of user-news interactions; (ii) explainable proxy tasks, enriching news contexts and refining node embeddings in user-news interaction networks with llm-generated task explanations; (iii) llm-based expert ensemble, adopting llms to selectively merge the predictions of task-specific experts and enhance calibration. ## results we present the performance of dell and stateof-the-art baselines in ## analysis quality of generated comments we verify the quality of llm-generated comments on whether it matches the user attributes and whether it is related to the news article. we conduct a human evaluation with four annotators to manually evaluate 50 generated comments from two datasets on a five-point likert scale, where the higher scores mean better quality. the average score is 4.52, the standard deviation is 0.69 and the annotator agreement in fleiss' kappa is 0.216, which indicates that annotators generally agree that the llm-generated comments are related and on-brand for user attributes. we additionally employ gpt-4 evaluation (chiang and lee, 2023; @xcite for quantitative evaluation, where we randomly sample 700 generated comments and prompt gpt-4 with "does the user's comment on the news match the profile?" and "does the comment relate to the news?" to solicit a response on a five-point likert scale. figure 2 demonstrates that the automatic evaluation also finds that the generated comments are consistent with the user attributes and relevant to the news. we conduct an additional evaluation to "put a more challenging control group of comments generated by the same framework but with a different demographic": we sample comments from users with other attributes, and then employ gpt-4 evaluation to check whether the generated comments figure 4 : gpt-4 evaluation of the matching degree between different user groups. "cg" denotes "college grad", "non-cg" denotes "haven't graduated from college", and "hsd" denotes "have a high school diploma or less". the diagonal numbers are the highest both row-wise and column-wise, indicating that the generated comments are consistent with the user attributes. match each attribute. for example, we sample 100 synthetic comments (50 with the attribute democrat and 50 with the attribute republican), and then we employ gpt4 to evaluate to what extent, on a scale of 1-5, do these comments match democrats and republicans. then we could obtain 200 scores and draw a heat map. we similarly experiment with the education attribute, spanning "college grad", "haven't graduated from college", and "have a high school diploma or less". in figure 4 , we find that the diagonal numbers, where the user attribute matches what gpt-4 evaluates, are the highest both row-wise and column-wise, indicating that the generated comments are consistent with the user attributes. ## conclusion we propose dell for identifying fake news where llms could be incorporated as part of the pipeline. first, we employ llms to generate news reactions from diverse perspectives and simulate user-news networks. second, we design six explainable proxy tasks that help identify misinformation. llms perform these tasks and generate explanations to produce experts specializing in various aspects of news articles. finally, we develop three strategies for llms to merge task-specific experts and provide an overall prediction. extensive experiments demonstrate that dell achieves state-of-the-art performance on three tasks across seven datasets, presenting a misinformation detector better calibrated and better grounded in diverse perspectives. academy of engineering "the online and offline mixed educational service system for 'the belt and road' training in mooc china". we would like to express our gratitude for the support of k. c. wong education foundation.