DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection (2024)

Abstract: Large language models are limited by challenges in factuality and hallucinations to be directly employed off-the-shelf for judging the veracity of news articles, where factual accuracy is paramount. In this work, we propose DELL that identifies three key stages in misinformation detection where LLMs could be incorporated as part of the pipeline: 1) LLMs could generate news reactions to represent diverse perspectives and simulate user-news interaction networks; 2) LLMs could generate explanations for proxy tasks (e.g., sentiment, stance) to enrich the contexts of news articles and produce experts specializing in various aspects of news understanding; 3) LLMs could merge task-specific experts and provide an overall prediction by incorporating the predictions and confidence scores of varying experts. Extensive experiments on seven datasets with three LLMs demonstrate that DELL outperforms state-of-the-art baselines by up to 16.8% in macro f1-score. Further analysis reveals that the generated reactions and explanations are greatly helpful in misinformation detection, while our proposed LLM-guided merging helps produce better-calibrated predictions. 1

heading: 1 Introduction
text: Large language models (LLMs) have demonstrated impressive capabilities to follow instructions (Ouyang et al., 2022), perform knowledgeintensive tasks (Rubin et al., 2022; Shi et al., 2023), and confront societal challenges (Jiang et al., 2023c; Roy et al., 2023). However, LLMs are also hindered by hallucinations (Kryściński et al., 2020; Pagnoni et al., 2021; Dong et al., 2022), lack of factuality (Kandpal et al., 2023; Mallen et al., 2023), and challenges to adapt to new knowledge (De Cao
*These authors contributed equally. BCorresponding Author: minnluo@xjtu.edu.cn 1Available at https://github.com/whr000001/DELL.
et al., 2021; Hase et al., 2021). Despite preliminary efforts (Chen and Shu, 2023; Lucas et al., 2023), LLMs cannot yet be employed off-the-shelf for analyzing the veracity of news articles where factual accuracy is paramount (Leite et al., 2023; Hu et al., 2024). Together with emerging risks of generating misinformation at scale (Chen and Shu, 2023; Wu and Hooi, 2023b), these limitations call for new solutions to leverage LLMs to counter online fake news and misinformation campaigns.
While LLMs are not reliable in detecting misinformation directly, we propose DELL2, employing three key stages where LLMs could be integrated to provide more context and explanations for reliable assessment of news veracity (Figure 1):
• Community reactions and comments to news articles have been shown to improve misinformation detection systems (Grover et al., 2022). However, it is not always feasible to collect sufficient volumes of real-time user interactions (He et al., 2023a). Guided by LLMs’ potential in simulating human samples and populations (Argyle et al., 2023), we employ LLMs to generate synthetic reactions and comments to news articles from diverse perspectives, turning the news context into a rich network of user-news interactions.
• Previous research shows that additional pragmatic contexts such as sentiment and stance, as well as external knowledge help aid misinformation detection (Zhang et al., 2021; Hu et al., 2021; Sengan et al., 2023). To this end, we employ LLMs for proxy tasks, i.e., tasks where predictions and explanations could be helpful to better understand the news article. For example, LLMs evaluate the sentiment of news articles and generate both predictions and explanations: these additional contexts are then encoded as initial
2DELL stands for Diverse Reaction Generation; Explainable Proxy Tasks; and LLM-Based Expert Ensemble.
2637
embeddings in the user-news interaction network for classification based on graph neural networks (GNNs). By employing six proxy tasks focusing on the news article and generated reactions, we obtain a suite of specialized predictors that specialize in various aspects of news understanding.
• Finally, we employ LLMs as judges to merge the task-specific experts and predict the news veracity. Since not all experts are equally helpful/confident for a given news article, we provide LLMs with the predictions and confidence scores of experts specializing in each proxy task: the LLM is then instructed to selectively incorporate the predictions of experts for an overall decision.
We conduct extensive experiments to evaluate DELL and state-of-the-art baselines with three LLMs on seven datasets spanning three tasks related to news veracity, featuring both humanwritten and machine-generated misinformation. DELL outperforms the strongest baseline across all datasets, achieving an improvement of up to 16.8% in macro f1-score. Further analysis reveals that LLM-generated news reactions and explanations to proxy tasks contribute greatly to model performance, while the LLM-guided expert merging results in better-calibrated misinformation detectors for both human- and machine-generated news.

heading: 2 Methodology
text: We propose three strategies to integrate LLMs in evaluating news veracity: (i) Diverse Reaction
Generation, leveraging LLMs to generate synthetic news reactions from diverse perspectives and forming networks of user-news interactions; (ii) Explainable Proxy Tasks, enriching news contexts and refining node embeddings in user-news interaction networks with LLM-generated task explanations; (iii) LLM-Based Expert Ensemble, adopting LLMs to selectively merge the predictions of task-specific experts and enhance calibration.

heading: 2.1 Diverse Reaction Generation
text: Integrating the public discourse to evaluate news veracity is widely employed to better ground news articles and provide more context (Grover et al., 2022; Sheng et al., 2022; Wu and Hooi, 2023a; Shovon and Shin, 2023). However, real-world comments and reactions are challenging to collect, while malicious comments aiming to bolster misinformation might be removed from social media platforms and hinder reproducible research (Jung et al., 2020; Grover et al., 2022; He et al., 2023a). Motivated by LLMs’ successes in simulating human samples (Argyle et al., 2023) and reflecting diverse perspectives (Sorensen et al., 2024), we propose to generate synthetic comments and reactions by LLMs, simulating how populations from diverse perspectives might respond to news articles.
Diverse User Attribute We first define the space of social media user attributes to simulate. Specifically, each synthetic user is represented as an intersection of seven categories:3 gender, age, ethnicity,
3We select these categories and attributes from The Pew Research Center’s American Trends Panel. Full list of poten-
education, family income, political leaning, and voter registration. Formally, for a user attribute P i (1 ≤ i ≤ n, n = 7), its candidate set is {pji}nij=1 where ni denote the number of possibilities for a given attribute category. We sample uniformly for each user attribute to represent a social media user. We then verbalize these attributes and concatenate them as the prompt u for the synthetic user.
Generating User-News Networks Aside from news content, the non-sequential propagation structure of news comments is shown to aid in evaluating news veracity (Ma et al., 2018; Lu and Li, 2020; Ma et al., 2023a). Formally, given a news article s, we aim to generate a user-news interaction network G(V, E), where V and E denote the node and edge sets. We develop three strategies for LLMs to simulate the comment propagation process: (i) generate a comment on the news article; (ii) generate a comment on a given comment; and (iii) select a comment to engage:
• Comment on news. We first generate a synthetic user description u (§2.1) and append the following prompt: “You view a piece of news with the following content. News: s”. The LLM is then instructed to generate a comment representing the user’s perspective, specifically with the prompt “Please comment on this news on social media.”
• Comment on a comment. Similarly, we first provide LLMs with the user description u and news article s. We append a comment chain C = [c1∥c2∥ . . . ∥cm], where ci is a comment on ci−1. The LLM is then instructed to generate a comment to the last comment with “Please reply to the last comment.”
• Select a comment to comment. Social media users would selectively engage with certain comments informed by their perspectives. We employ LLMs to simulate this process by appending u, s, and multiple comment chains C, while instructing the LLM with “Please select a comment chain that you would most like to reply.”
We iteratively adopt these prompts to generate a user-news interaction network for a given news article. Algorithm 1 in Appendix A.2 presents details on the user-news network generation process.
tial attributes and example prompts in Appendix A.1.

heading: 2.2 Explainable Proxy Tasks
text: Integrating LLM-generated contexts about a given document has proven effective in analyzing textattribute graphs such as scholarly networks (He et al., 2023b; Chen et al., 2023c; Li et al., 2023a). In the domain of misinformation detection, there is often much implied context that goes beyond the news text itself, such as author stances, sentiment, external knowledge, and more. We propose to employ LLM-generated explanations for proxy tasks, i.e., tasks that help evaluate news veracity, enriching news contexts and refining the feature embeddings of user-news interaction networks with the generated explanations. Specifically, we propose four proxy tasks to enhance news articles:
• Sentiment Analysis News articles often feature sentiment signals that are indicative of their veracity (Zhang et al., 2021). We employ six basic emotions (Ekman et al., 1999) (e.g., anger and surprise) and prompt LLMs to choose the three most likely emotions and provide explanations.
• Framing Detection Framing is a strategic device in political communication (Entman, 1993) and has been an integral part of evaluating news veracity (Kwak et al., 2020; Mendelsohn et al., 2021). Similarly, we follow the taxonomy of 14 media frames (Card et al., 2015a) (e.g., economic) and prompt LLMs to choose the five most likely media frames and provide explanations.
• Propaganda Tactics Detection Propaganda tactics are employed to influence people’s mindsets to advance a specific agenda (Glowacki et al.). We follow the taxonomy of 19 propaganda tactics (Piskorski et al., 2023) (e.g., doubt and red herring) and employ LLMs to identify the underlying tactics in news articles with explanations.
• Knowledge Retrieval Retrieval-augmented language models (Borgeaud et al., 2022; Shi et al., 2023; Asai et al., 2023; Chen et al., 2023b) have demonstrated impressive potential to expand the knowledge access of LLMs. We employ LLMs to identify key entities in a news article through prompting and retrieve Wikipedia passages about these entities4. We prepend the retrieved external knowledge in the news article to facilitate better contextual understanding.
Besides news content, we also propose two proxy tasks to enhance the generated comments:
4We employ the Wikipedia API for retrieval.
• Stance Detection Given two text nodes s1 and s2 (news or comments) that are connected in the user-news interaction network G, we employ LLMs to evaluate whether s1 and s2 are supportive, neutral, or opposed to each other with explanations.
• Response Characterization Given two text nodes s1 and s2 (news or comments) in G, we employ LLMs to analyze whether one is in response to another. The generated explanations would help better understand the propagation structure of news and comments.
By employing any of the six proxy tasks5, we obtain an LLM-generated explanation paragraph sext that analyzes the news article from one specialized aspect. We leverage the LLM-generated explanations to refine the feature embeddings of user-news interaction networks. Specifically, we first adopt a separate encoder-based LM enc(·) to encode the news article sori and the explanation sext, i.e., hori(ext) = enc(sori(ext)), where we employ DeBERTa (He et al., 2021) in practice. We then concatenate hori and hext and feed it into a linear layer to obtain initial node features h(0).
We employ graph neural networks as the model for downstream tasks, which conduct message passing over the user-news network. Formally, suppose h (ℓ) vi is the representation of node vi at the ℓ-th GNN layer, the feature update procedure is:
h(ℓ)vi = Aggr∀vj∈N (vi) ({Prop(h(ℓ−1)vi ;h(ℓ−1)vj )}),
where N (vi) denotes the set of neighbors of node vi, Aggr(·) and Prop(·) are aggregation and propagation functions, where GIN (Xu et al., 2019) is employed in practice. To obtain the graph-level representation of G, we employ the mean pooling operator as the Readout(·) function, i.e.,
h = Readout({h(ℓ)vi }vi∈V). Given a user-news network G and a label y, we compute the probability of y being the correct prediction as p(y | G) ∝ exp(MLP(h)), where MLP(·) denotes an MLP layer. For binary classification, we optimize models using the cross-entropy loss and predict the most plausible label as argmaxy p(y | G). For multi-label classification, we optimize models using the ZLPR (Su et al., 2022) loss and predict the label set as {y : p(y | G) > λ}, where λ is a hyperparameter.
5We provide the prompts for proxy tasks in Appendix A.3.

heading: 2.3 LLM-Based Expert Ensemble
text: By adopting different proxy tasks and LLMgenerated explanations, we obtain a set of experts, where each specializes in one proxy task and various aspects of news articles. To obtain an overall prediction, we propose an LLM-based expert ensemble to selectively leverage experts, their predictions, and confidence scores. We first use one sentence di to describe each expert, e.g., “This expert focuses on the emotion of news.” We then propose three modes for LLMs to merge experts6:
Vanilla LLMs are first provided with news content and an instruction, i.e., “Some experts give predictions about the news.” We then append the description and prediction of each expert: for an expert ei with prediction ℓi and its description di, the expert prompt is “Expert i: di. The expert predicts the label of this news is ℓi.” Finally, the LLM is instructed to reason and generate a final prediction based on the experts’ feedback.
Confidence In Vanilla, we assume that all experts should be equally important. However, experts could have varying levels of confidence and we take this into account by additionally providing the confidence scores. The confidence scores are obtained from the classification layer of the GNN-based model (§2.2). We aim to improve the calibration of LLM-based expert ensemble by incorporating confidence scores of individual experts.
Selective In Vanilla and Confidence, we assume that every news article would benefit from the input of all experts. However, this could introduce noise in the LLM reasoning process (Feng et al., 2023d; Zhao et al., 2024). To this end, we propose the Selective approach, putting LLMs in charge to selectively activate experts. Specifically, we provide news content and expert descriptions, then prompt LLMs with “To understand this news, which expert knowledge do you need?” We ensemble the selected experts with the Confidence strategy to obtain the final predictions.

heading: 3 Experiment Settings
text: Models and Settings We leverage Mistral-7B (Jiang et al., 2023a), LLaMA2-70B (Touvron et al., 2023), and ChatGPT as the base LLMs. We mainly employ Mistral-7B to generate comments and conduct proxy tasks, and ChatGPT to ensemble ex-
6We provide prompts in Appendix A.4.
perts. We set the temperature τ = 0.6 for Mistral7B and τ = 0.1 for ChatGPT. We present more results from other LLMs in Appendix C.
Baselines We compare DELL with three types of state-of-the-art baselines: 1) LLM-only: ZEROSHOT, FEW-SHOT, RETRIEVAL-augmented generation, F3 Z-CoT (Lucas et al., 2023), F3 DeFGen (Lucas et al., 2023), TAPE W/O GRAPH (He et al., 2023b), and DEBERTA (He et al., 2021); 2) LLM+Graph: K-HOPS (Huang et al., 2023a) and K-ATTENTION (Huang et al., 2023a), and TAPE W/ GRAPH (He et al., 2023b); 3) Graph-based: GCN (Kipf and Welling, 2017), RVNN (Ma et al., 2018), DEFEND (Shu et al., 2019a), HYPEHN (Grover et al., 2022), GET (Xu et al., 2022), and WSDMS (Yang et al., 2023b). We provide more details about baselines in Appendix B.2.
Tasks and Datasets We evaluate DELL and baselines on three tasks related to chacterizing misinformation, i.e., 1) fake news detection: Pheme (Buntain and Golbeck, 2017) and LLM-mis (Chen and Shu, 2023), which feature a binary classification setting; 2) framing detection: MFC (Card et al., 2015b) and SemEval-23F (Piskorski et al., 2023), which feature a multi-label classification setting; 3) propaganda tactic detection: Gener-
ated generated by ChatGPT, SemEval-20 (Martino et al., 2020), and SemEval-23P (Piskorski et al., 2023), which feature a multi-label classification setting. The datasets are all in English and we provide more dataset details in Appendix B.1. To evaluate the ability to evaluate machine-generated news, LLM-mis is an extended version of FakeNewsNet (Shu et al., 2020) and Generated is generated by LLMs.

heading: 4 Results
text: We present the performance of DELL and stateof-the-art baselines in Table 1. We present more ablation study results in Table 6 in Appendix C.
DELL achieves state-of-the-art performance. DELL outperforms the strongest baseline on all seven benchmarks by 1.46% to 16.80% on macro f1-score, indicating the success of integrating LLMs in multiple stages of news veracity evaluation. We find that LLM-only in-context learning approaches struggle in performance, indicating that LLMs are limited by factuality challenges and hallucinations to evaluate the veracity of news articles.
Generated news reactions help ground news articles. Compared to news-only approaches, models enhanced with generated comments (both ours and
graph-based baselines) achieve better performance. The average performance on MFC of the commentenhanced models is 15.2% higher on MaF. It indicates that LLM-generated diverse comments are beneficial in characterizing misinformation.
Proxy tasks improve news understanding ability. DELL single denotes the performance of the best single expert focusing on one proxy task. We find that a single expert could already achieve a substantial improvement in most cases: for example, on benchmark Generated, it achieves a 6.16% improvement on the macro f1-score than the strongest baseline. This indicates that our explainable proxy tasks are effective strategies for incorporating LLMs for evaluating news veracity.
LLMs could ensemble expert predictions. Compared to a single expert, the proposed LLM ensemble strategies achieve improvements on six out of seven datasets. In addition to simple aggregation (Vanilla), Confidence and Selective improve the ensemble by accessing the confidence scores and selectively incorporating certain experts, indicating that LLMs have preliminary capabilities of understanding verbalized confidence scores (Tian et al., 2023; Feng et al., 2024). We further investigate if LLM-based ensembling could lead to better-
calibrated misinformation detectors in Section 5.

heading: 5 Analysis
text: Quality of Generated Comments We verify the quality of LLM-generated comments on whether it matches the user attributes and whether it is related to the news article. We conduct a human evaluation with four annotators to manually evaluate 50 generated comments from two datasets on a five-point Likert scale, where the higher scores mean better quality. The average score is 4.52, the standard deviation is 0.69 and the annotator agreement in Fleiss’ Kappa is 0.216, which indicates that annotators generally agree that the LLM-generated comments are related and on-brand for user attributes.
We additionally employ GPT-4 evaluation (Chiang and Lee, 2023; Kim et al., 2023b) for quantitative evaluation, where we randomly sample 700 generated comments and prompt GPT-4 with “Does the user’s comment on the news match the profile?” and “Does the comment relate to the news?” to solicit a response on a five-point Likert scale. Figure 2 demonstrates that the automatic evaluation also finds that the generated comments are consistent with the user attributes and relevant to the news.
We conduct an additional evaluation to “put a more challenging control group of comments generated by the same framework but with a different demographic”: we sample comments from users with other attributes, and then employ GPT-4 evaluation to check whether the generated comments
match each attribute. For example, we sample 100 synthetic comments (50 with the attribute Democrat and 50 with the attribute Republican), and then we employ GPT4 to evaluate to what extent, on a scale of 1-5, do these comments match Democrats and Republicans. Then we could obtain 200 scores and draw a heat map. We similarly experiment with the education attribute, spanning “college grad”, “haven’t graduated from college”, and “have a high school diploma or less”. In Figure 4, we find that the diagonal numbers, where the user attribute matches what GPT-4 evaluates, are the highest both row-wise and column-wise, indicating that the generated comments are consistent with the user attributes.
Network Generation Ability To establish that the generated interaction networks resemble realworld networks, we compare our generated networks with the real networks in datasets Pheme (Buntain and Golbeck, 2017), Twitter-15 (Ma et al., 2018), and Twitter-16 (Ma et al., 2018). Specifically, we calculate the average edge betweenness of each edge, the average shortest path length, the ratio of maximum degree to number of nodes, and the diameter of each graph. Then we average the value over the whole dataset to compare in Table 2. The results show that our generated networks are statistically similar to those in dataset Pheme as for network structure, indicating our generation strategy could stimulate the network structures similar to the real situation. In addition, hyperparameters in Algorithm 1 enable the control of generating user reaction networks. For example, by setting α = 0.8 and β = 0.05, generated networks resemble those in datasets Twitter-15
GCN RvNN Hyphen GET WSDMS DELL .70
.75
.80
M ac
ro F
1- sc
or e
Pheme
Democratic Mixed Republican
GCN RvNN Hyphen GET WSDMS DELL .80
.85
.90
M ac
ro F
1- sc
or e
LLM-mis
Expert Selection In the Selective LLM-based ensemble strategy, LLMs determine which experts are activated and incorporated in the overall decision. To evaluate each expert’s contribution, we examine the frequency of expert selection and the performance when a given expert is selected. Figure 6 illustrates that experts who have been selected more times tend to perform better, indicating that LLMs have preliminary capabilities to select helpful experts based on the news content.
Model Calibration Robust fake news detectors should provide not only a binary prediction but also a well-calibrated confidence score to facilitate content moderation. We evaluate how well DELL and baselines are calibrated with the fake news detection datasets in Figure 8. We use the probability of the prediction token (“fake” or “real”) from the LLM as the confidence score, bin it into five buckets (0.5 to 1.0), and calculate the estimated calibration error (ECE) (Guo et al., 2017). It is demonstrated that DELL are better-calibrated with an ECE of 0.2357 while achieving an improvement of up to 19.1% compared to baselines. We hypothesize that by integrating expert confidence scores in the LLM-guided ensemble, the overall decision

heading: Democrat man middle-aged
text: is better-calibrated and thus more trustworthy.
Case Study We study a specific case of news article and its LLM-generated comments in Figure 7. The red area indicates that the generated comments match the user attributes about partisanship and age groups. The green areas indicate strong continuity in the comment chain. Overall, the example showcases the effectiveness of DELL in generating diverse comments that ground news articles and facilitate characterization.

heading: 6 Related Work
text: Existing fake news detection methods (Zeng and Gao, 2022; Biamby et al., 2022; Mendes et al., 2023; Sung et al., 2023; Xu et al., 2023a; Liao et al., 2023) mostly fall into text-based (Pelrine et al., 2021; Jin et al., 2022; Chen et al., 2023d) and graph-based approaches (Wu et al., 2022; Zhou et al., 2022; Karami et al., 2023; Feng et al., 2023e; Lin et al., 2023; Phan et al., 2023; Chang et al., 2023; Ma et al., 2023b). Text-only approaches take news context and employ NLP methodologies
for classification such as recurrent neural networks (Goonathilake and Kumara, 2020; Liu et al., 2023), attention mechanism (Shu et al., 2019a; Dun et al., 2021), and pre-trained language models (Hartl and Kruschwitz, 2022). In addition to solely considering news content, graph-based approaches first construct networks composed of entities such as news articles, sources (Nguyen et al., 2020), users (Shu et al., 2019b; Dou et al., 2021), and more. These approaches then employ graph neural networks (Bian et al., 2020; Zhang et al., 2024) for classification. Among graph-based approaches, the widely used is to employ comments, i.e., user reactions to news article on social media (Yang et al., 2021; Tian et al., 2022; Mehta et al., 2022; Yang et al., 2023b; Russo et al., 2023; Min and Ananiadou, 2023). In this work, we seek to employ LLMs to generate synthetic comments from diverse perspectives to complement the scarce and incomplete comment networks in real-world datasets (Jung et al., 2020; Micallef et al., 2020; Heidari et al., 2021).
With the advent of autoregressive large language models, previous works have attempted to gauge their risks and generate misinformation with LLMs (Zellers et al., 2019; Fung et al., 2021; Huang et al., 2023c; Wang et al., 2023). They find that LLMs are capable of generating misinformation that is challenging to detect and characterize (Huang et al., 2023b; Chen and Shu, 2023; Pan et al., 2023b; Goldstein et al., 2023; Su et al., 2023b; Xu et al., 2023b; Uchendu et al., 2023). On the other hand, researchers have attempted to employ LLMs off-theshelf for misinformation research through prompting and in-context learning (Stiff and Johansson, 2022; Gabriel et al., 2022; Kim et al., 2023a; Pelrine et al., 2023; Russo et al., 2023; Jiang et al., 2023b; Nakshatri et al., 2023; Sundriyal et al., 2023; Su et al., 2023a; Li et al., 2023b; Chen et al., 2023a; Feng et al., 2023c; Yue et al., 2023; Yang et al., 2023a; Choi and Ferrara, 2024). We argue
that LLMs face challenges of hallucination (Ji et al., 2023; Du et al., 2023), factuality (Kandpal et al., 2023; Pan et al., 2023a), and temporal knowledge update (Feng et al., 2023a; Luo et al., 2024): as a result, they could not be directly used off-theshelf for predicting a True-of-False label since they lack accurate and up-to-date information about realworld news events, while such information is crucial in characterizing fake news campaigns. To this end, we identify three key stages in evaluating news veracity and propose strategies to integrate LLMs in countering online misinformation campaigns.

heading: 7 Conclusion
text: We propose DELL for identifying fake news where LLMs could be incorporated as part of the pipeline. First, we employ LLMs to generate news reactions from diverse perspectives and simulate user-news networks. Second, we design six explainable proxy tasks that help identify misinformation. LLMs perform these tasks and generate explanations to produce experts specializing in various aspects of news articles. Finally, we develop three strategies for LLMs to merge task-specific experts and provide an overall prediction. Extensive experiments demonstrate that DELL achieves state-of-the-art performance on three tasks across seven datasets, presenting a misinformation detector better calibrated and better grounded in diverse perspectives.

heading: Acknowledgement
text: This work was supported by the National Nature Science Foundation of China (No. 62192781, No. 62272374), the Natural Science Foundation of Shaanxi Province (2024JC-JCQN-62), the National Nature Science Foundation of China (No. 62202367, No. 62250009, No. 62137002), Project of China Knowledge Center for Engineering Science and Technology, and Project of Chinese
academy of engineering “The Online and Offline Mixed Educational Service System for ‘The Belt and Road’ Training in MOOC China”. We would like to express our gratitude for the support of K. C. Wong Education Foundation.

heading: Limitation
text: While DELL could generate synthetic news reactions from diverse perspectives and form networks of user-news interactions, the iterative process with LLMs in computationally heavy. Scaling our solution to the real-world scale of millions of real-time news reactions could be challenging, while we expect efficient LLM inference approaches could help alleviate this limitation.
While we develop six proxy tasks for LLMs to generate explanations and enrich news contexts, they may not be able to fully tap into the diverse capabilities of LLMs and their potential for evaluating the veracity of news articles. Future work could focus on automatically generating and proposing proxy tasks for a more general LLM-as-enhancer framework.

heading: Ethics Statement
text: The development of fake news detectors is essential in countering online misinformation campaigns. This research demonstrates that LLMs could be integrated as part of the news analysis pipeline. However, it may increase the risk of dual-use, where malicious actors may develop advanced misinformation campaigns that are evasive to LLM-generated comments and explanations. We will establish controlled access to ensure that the data and trained model checkpoint are only publicly available to researchers.
LLMs have been widely shown to have inherent social biases (Bender et al., 2021; Jin et al., 2021; Shaikh et al., 2023), and such biases could have an impact on fake news detection (Feng et al., 2023b). Informed by LLMs’ internal biases, stereotypes, and spurious correlations, DELL might struggle to simulate certain demographic groups and provide incorrect explanations of news articles. We argue that the predictions of DELL should be interpreted as an initial screening, while content moderation decisions should be made with experts in the loop.

heading: A.1 User Attribute Details

heading: A Methodology Details
text: Algorithm 1: Pseudo-code of user-news network generation. Data: news content s; graph size m; α to
control the probability of commenting on the news; β to control the balance of tree height and width; k to control candidate set size
Result: user-news network G(V, E) 1 V = [s] 2 E = [] 3 F = [s] 4 H denoting height of each node 5 W denoting width of each node 6 while ∥V ∥ ≤ m do 7 u = ψ() 8 p ∼ U(0, 1) 9 if p <= α then
10 PROMPT = ϕ1(s,u) 11 f = s 12 else 13 P = β ∗ H+ (1− β) ∗W 14 {ci}i = Sample(V,P, k) 15 SELECT = ϕ3(s,u, {Ci}i) 16 j = LLM(SELECT) 17 f = C = Cj 18 PROMPT = ϕ2(s,u,C) 19 end 20 sout = LLM(PROMPT) 21 V .append(sout) 22 E .append((sout,f)) 23 F .append(f ) 24 update H and W 25 end
Return: G(V, E)

heading: A.2 User-News Networks Details
text: Our generated user-news interaction networks G forms a tree structure, where ∥V∥ = ∥E∥ + 1. To control the iterative process of generating usernews networks, we design hyperparameters α to control the probability of commenting on news and β to control the balance of tree height and width. We present an algorithmic summary of the generation process in Algorithm 1, where ϕ1(s,u), ϕ2(s,u,C), and ϕ3(s,u, {Ci}i) denotes comment on news, Comment on a comment, and Select a comment to comment prompt generation process, ψ() denotes the diverse user attribute prompt generation process as mentioned in Appendix A.1, Sample(V,P, k) denote the sample function that
sample k instances from V according to the probability P . Table 9 presents the prompt templates.

heading: A.3 Explainable Proxy Task Details
text: We propose four proxy tasks to enhance news articles, the details of Sentiment Analysis, Framing Detection, and Propaganda Tactics Detection are as follows:
• Sentiment Analysis: we employ six basic emotions: anger; disgust; fear; happiness; sadness; and surprise.
• Framing Detection: we employ 14 news frames: Economic; Capacity and resources; Morality; Fairness and equality; Legality, constitutionality and jurisprudence; Policy prescription and evaluation; Crime and punishment; Security and defense; Health and safety; Quality of life; Cultural identity; Among public opinion; Political; External regulation and reputation.
• Propaganda Tactics Detection: we employ 19 propaganda tactics: Conversation Killer; Whataboutism; Doubt; Straw Man; Red Herring; Loaded Language; Appeal to Fear-Prejudice; Guilt by Association; Flag Waving; False Dilemma-No Choice; Repetition; Appeal to Popularity; Appeal to Authority; Name Calling-Labeling; Slogans; Appeal to Hypocrisy; Exaggeration-Minimisation; Obfuscation-Vagueness-Confusion; Causal Oversimplification.
Table 10 presents the presents the prompt template of each proxy task.

heading: A.4 LLM-Based Expert Ensemble Details
text: We propose three LLM-based approaches to selectively integrate the prediction of each expert. The description di of each expert ei is as follows:
• w/o expert: This expert is comprehensive.
• sentiment: This expert focuses on the emotion of this news.
• framing: This expert focuses on the framing of this news.
• propaganda tactics: This expert focuses on the propaganda tactics of this news..
• retrieval: This expert focuses on the external knowledge of this news.
• stance: This expert focuses on the stance of related comments on this news.
• relation: This expert focuses on the relation of related comments on this news.
to obtain the confidence score, we employ a softmax operator score for binary classification and absolute value operator for multi-label classification. We provide the prompt templates in Table 11.

heading: B Experiment Setting Details

heading: B.1 Dataset Details
text: We evaluate DELL and baselines on three tasks related to fake news detecton.
1) Fake News Detection:
• Pheme (Buntain and Golbeck, 2017) is a dataset of potential rumors on Twitter and journalistic assessments of their accuracies.
• LLM-mis (Chen and Shu, 2023) is a LLMgenerated misinformation dataset with different LLM generators and generation approaches.
2) Framing Detection:
• MFC (Card et al., 2015b) contains labeled and unlabeled articles on six issues from 14 newspapers covering the years 1990-2014, though some issues have broader coverage. The issues include climate; the death penalty; gun control; immigration; same-sex sex; and tobacco. We sample the labeled articles as a benchmark.
• SemEval-23F (Piskorski et al., 2023) aim to identify one or more frames used in an article from a pool of 14 generic frames: Security and defense; Fairness and equality; Political; Capacity and resources; Economic; Morality; Policy prescription and evaluation; Legality Constitutionality and jurisprudence; External regulation and reputation; Quality of life; Health and safety; Cultural identity; Crime and punishment; and Public opinion.
3) Propaganda Tactic Detection:
• Generated is a benchmark generated by ChatGPT. We first determine 4 topics: Russia and Ukraine; Palestine and Israel; the Republican Party; and the Democratic Party. Around these topics, we generated 5 paragraphs for each tactic.
• SemEval-20 (Martino et al., 2020) contains 14 possible propaganda tactics: Appeal to fearprejudice; Black-and-White Fallacy; Name Calling, Labeling; Slogans; Whataboutism,Straw Men,Red Herring; Exaggeration, Minimisation; Loaded Language; Repetition; Causal Oversimplification; Bandwagon,Reductio ad hitlerum; Flag-Waving; Thought-terminating Cliches; Appeal to Authority; and Doubt. This benchmark merges some tactics into one category.
• SemEval-23F (Piskorski et al., 2023) contains 6 main categories: Attack on reputation; Justification; Simplification; Distraction; Call; and Manipulative wording. It contains 19 propaganda tactics: Conversation Killer; False Dilemma-No Choice; Appeal to Popularity; Doubt; Flag Waving; Slogans; Whataboutism; Straw Man; Loaded Language; Name Calling-Labeling; ObfuscationVagueness-Confusion; Appeal to Fear-Prejudice; Causal Oversimplification; Red Herring; Repetition; Exaggeration-Minimisation; Appeal to Authority; Guilt by Association; and Appeal to Hypocrisy.
We randomly sample 1,000 instances from each benchmark (select all if there are less than 1,000 instances), and divided the training set, validation set, and test set according to the ratio of 7:2:1.

heading: B.2 Bseline Details
text: • ZERO-SHOT asks LLMs to conduct detection.
• FEW-SHOT first provides LLMs with some pairs of news instances and labels and then asks LLMs to conduct detection.
• RETRIEVAL-augmented generation first provides LLMs with the external knowledge retrieved from Wikipedia, which is the same as Knowledge Retrieval proxy task. It then asks LLM to conduct detection.
• F3 Z-CoT (Lucas et al., 2023) uniquely leverages LLMs’ self-formulated rationales by integrating a standard instruction with the simple phrase, “Let’s think step by step known as Chain of Thoughts (CoT).”
• F3 DeF-Gen (Lucas et al., 2023) focuses contextually, emphasizing deductive and abductive reasoning.
• TAPE W/O GRAPH (He et al., 2023b) focus on leveraging LLMs to capture textual information as features, which can subsequently enhance GNN performance on downstream tasks. Here we only employ the textual information generated by LLMs to enhance news content. DEBERTA (He et al., 2021) leverages the pretrained language model DeBERTa to encode news content, then feed them into an MLP classifier.
• K-HOPS (Huang et al., 2023a) incorporates randomly selected neighbors into the prompt, where the idea behind is to aggregate information from neighboring nodes, following GCN.
• K-ATTENTION (Huang et al., 2023a) is designed to weigh the influence of neighboring nodes during the prediction process, following GAT.
• TAPE W/ GRAPH (He et al., 2023b) puts the enhanced news content into the user-news network and employs graph neural networks to conduct detection.
• GCN (Kipf and Welling, 2017) adopt multiple GNN layers and a meaning pooling to obtain the user-news network representations.
• DEFEND (Shu et al., 2019a) conducts explainable detection by the attention weights, we set maximum sentence length and maximum comment length as 96, maximum sentence count as 64, and maximum comment count as 10 to reproduce so that the approach is applicable to our tasks and datasets.
• RVNN (Ma et al., 2018) proposes two recursive neural model stratages: bottom-up and top-down tree-structured neural networks. We employ the top-down structure.
• HYPEHN (Grover et al., 2022) is a discourseaware hyperbolic spectral co-attention network. It is a fusion of hyperbolic graph representation learning with a novel Fourier co-attention mechanism in an attempt.
• GET (Xu et al., 2022) models claims and related evidences as graph-structured data and capture the long-distance semantic dependency among dispersed relevant snippets via neighborhood propagation.
• WSDMS (Yang et al., 2023b) needs bag-level labels for training but possesses the capability to infer both sentence-level misinformation and article-level veracity, facilitated by pertinent social media conversations meticulously contextualized with news sentences.
For the LLM-based baseline, we provide the prompt templates in Table 12. Each baseline prompt template contains a task-related prompt to describe the task and a baseline-related prompt.

heading: B.3 Hyperparameters
text: The hyperparameter settings of DELL are presented in Table 5 to facilitate reproduction.

heading: C Additional Results
text: We leverage Mistral-7B (Jiang et al., 2023a), LLaMA2-70B (Touvron et al., 2023), and ChatGPT as the base LLMs. Tabel 7 presents more results. DELL still outperforms other baselines.
For context, DELL has three components: Diverse Reaction Generation; Explainable Proxy Tasks; and LLM-Based Expert Ensemble. On the component level, we conduct more ablation studies as follows:
• w/o Diverse Reaction Generation: we remove the network part and only employ the news content and related proxy task outputs.
• w/o Explainable Proxy Tasks: we remove all proxy tasks and conduct experiments with news content and generated content.
• w/o LLM-Based Expert Ensemble: we replace LLM-based ensembling with simple majority
voting: majority vote; confidence weighted sum; and train the weights on validation set.
We present the results of the ablation study in Table 6. Every module of DELL could improve the fake news detection performance.

heading: D Additional Analysis

heading: D.1 Model Robustness to Comments (cont.)
text: Figure 9 presents the results of the other six benchmarks. On other benchmarks, DELL drops the least in performance with reduced comments. Specifically, DELL almost keeps the same on Pheme and drops 11.8% on MFC, 1.1% on SemEval-23F, 7.1% on Generated, 3.9% on SemEval-20, and 3.3% on SemEval-23P.

heading: D.2 Expert Selection (cont.)
text: Figure 10 presents the results of the other six benchmarks. Besides this experiment, we also examine the count of experts in every selection and corresponding performance. The results are shown in Table 8.

heading: D.3 Case Study (cont.)
text: Table 14, 15, and 16 provide more cases of the explanations of proxy tasks generated by DELL. It illustrates that LLMs could generate reasonable explanations of proxy tasks, providing more information in identifying fake news.
News: Given the recent wave of allegations pertaining to sexual misconduct on the part of congressmen in Washington, it is not inconceivable that similar accusations could be directed at Senator Chuck Schumer. While I acknowledge that some of my previous posts may have been hasty and not grounded in factual evidence, I must clarify that I do not make such remarks maliciously. I hope this insight assuages any potential doubts about my intentions. User profile: You are a social media user. You are female. You are 18 to 29 years old. Racially, you are White. Financially, your annual family income is less than 30,000. Educationally, you haven’t graduated from college. Politically, you are a Democrat. Meanwhile, you are not registered to vote. Comment: As a fellow Democrat, I urge Senator Schumer to address these allegations swiftly and transparently. It’s important that we hold all elected officials accountable for their actions.

heading: Future Work
text: Future work could focus on automatically generating and proposing proxy tasks for a more general LLM-as-enhancer framework