A Coarse-to-fine Cascaded Evidence-Distillation Neural Network for Explainable Fake News Detection (2022)

Abstract: Existing fake news detection methods aim to classify a piece of news as true or false and provide veracity explanations, achieving remarkable performances. However, they often tailor automated solutions on manual fact-checked reports, suffering from limited news coverage and debunking delays. When a piece of news has not yet been fact-checked or debunked, certain amounts of relevant raw reports are usually disseminated on various media outlets, containing the wisdom of crowds to verify the news claim and explain its verdict. In this paper, we propose a novel Coarse-to-fine Cascaded Evidence-Distillation (CofCED) neural network for explainable fake news detection based on such raw reports, alleviating the dependency on fact-checked ones. Specifically, we first utilize a hierarchical encoder for web text representation, and then develop two cascaded selectors to select the most explainable sentences for verdicts on top of the selected top-K reports in a coarse-to-fine manner. Besides, we construct two explainable fake news datasets, which is publicly available. Experimental results demonstrate that our model significantly outperforms state-of-the-art detection baselines and generates high-quality explanations from diverse evaluation perspectives.

## introduction during the covid-19 pandemic, almost 80% of consumers in the united states received fake news, which has caused confusion and undermined public health efforts 1 . the proliferation of fake news has increased the demand for automatic fake news detection @xcite . to further clarify and explain detection results, explainable fake news detection has gained more importance recently, aiming to classify the truthfulness of a piece of news and generate veracity explanations 2 (kotonya and figure 1 : an example for veracity explanation generation. the underlined explanations can be semantically inferred from some relevant sentences in the reports 𝑅 1 and 𝑅 𝑛 . "𝑅" denotes the raw report. toni, 2020a). however, existing methods have a limitation in detecting fake news timely as they heavily relied on debunked reports of investigated journalism. thus, it is urgent to develop explainable yet general methods to mitigate this issue. many previous approaches detected fake news without any justifications @xcite . recently, some explainable methods highlighted salient words or phrases in relevant reports as explanations @xcite , which lack readable complete sentences. to alleviate these issues, some methods aimed to extract salient sentences from relevant reports via attention mechanisms @xcite , or pre-trained extractive-abstractive summarization @xcite , etc. as the human justification about veracity labels can significantly improve the performance of veracity prediction @xcite , @xcite proposed the first study on producing veracity explanations jointly with veracity prediction utilizing the debunked report released by fact-checking websites. however, such a debunked report is based on manual endeavors, thus prone to be coveragelimited and relatively inefficient. a new study by mit researchers suggests that crowds of laypeople reliably rate claims as effectively as fact-checkers do @xcite . to use the wisdom of crowds, we assume that crowds of relevant raw reports (e.g., media reports, user comments, blogs, etc.) published by different media outlets contain evidence for effectively detecting fake news and explaining verdicts @xcite . as shown in figure 1 , given a false claim "microwaving fabric masks is a good way to sanitize them for reuse", the checkworthy reports 𝑅 1 and 𝑅 𝑛 are selected from all reports [𝑅 1 , 𝑅 2 , • • • , 𝑅 𝑛 ] and then some evidential sentences (underlined) can be used to generate veracity explanations. in contrast, existing methods usually tailor models on one manual fact-checked article, rarely attempting to detect fake news based on raw reports. to this end, we propose a general coarse-to-fine cascaded evidence-distillation (cofced) network to detect fake news and explain verdicts directly using raw reports, mitigating the dependency on fact-checked reports. specifically, we design a hierarchical encoder for text representation, and then we develop two coarse-to-fine cascaded selectors to distill explainable sentences on top of the selected top-𝐾 check-worthy reports. our predictions of explainable sentences can be obtained by explicitly considering four features, i.e., claim relevance, richness, salience, and non-redundancy. different from fever @xcite using humancrafted claims with credible wikipedia articles, the claims in our task are real-world news containing some unreliable reports. thus, detecting fake news on raw reports is much more challenging and significant than that in fever task. our contributions are as follows: 1) to the best of our knowledge, we present the first study on explainable fake news detection directly utilizing the wisdom of crowds, alleviating the dependency on fact-checked reports; 2) our model has the advantage of revealing insight into the generation of veracity explanations from various perspectives; 3) we construct two realistic datasets, i.e., rawfc and liar-raw, consisting of raw reports for each claim. experimental results on benchmarks demonstrate the effectiveness of cofced for detecting fake news and and explaining verdicts based on raw reports. our resources are publicly available at https://github.com/nicozwy/cofced . ## related work we review prior works closely related to ours based on several surveys @xcite . black-boxed fake news detection. many existing studies on fake news detection achieved promising performances by incorporating claim metadata to facilitate the detection, such as user profiles @xcite @xcite . besides, various deep learning methods have been proposed to capture report features, e.g., credibility @xcite , stances @xcite , writing styles @xcite , extra knowledge @xcite , etc. although these methods could improve the detection performance, they are lack of explainability on verdicts. explainable fake news detection. to address the above issue, many explainable methods on this task explored attention mechanisms to highlight salient words @xcite , news attributes @xcite , and suspicious users @xcite , to obtain relevant evidence, providing a certain explainability. to improve the readability in word-level methods, there are some methods obtained evidential sentences using attention weights @xcite , semantic matching @xcite , and entailment @xcite . more recently, @xcite proposed the first study on directly producing veracity explanations using extractive summarization, and kotonya and toni (2020b) made use of extractiveabstractive summarization for explanation generation, independent of the veracity prediction. however, they significantly relied on the manual factchecked report and rarely attempted to consider fine-grained features for this task. thus, we utilize the wisdom of crowds for fake news detection based on raw reports, providing a highly explainable structure for explanation generation. ## problem statement given a fake news dataset {c}, @xmath0, d) is a tuple representing a given claim 𝑐 and its rel- we formulate this task as a multi-task learning problem by considering check-worthy report selection, explainable sentence extraction, and veracity prediction. formally, 𝑓 : 𝑓 (𝑐, d) → ( ŷ, ŷ 𝑑 , ŷ 𝑠 , ê), where ê denotes the veracity explanation (i.e., evidence) consisting of a set of predicted sentences (i.e., ŷ𝑑 𝑖 = 1 and ŷ𝑠 𝑖, 𝑗 = 1). fig. 2 gives an overview of our proposed cofced, which consists of four parts: hierarchical encoding, report selection, explainable sentence extraction, and veracity prediction. ## conclusion we present a coarse-to-fine cascaded evidencedistillation (cofced) neural network for explainable fake news detection that achieves the best detection performance and distills accurate veracity explanations directly from raw reports. besides, cofced has the additional advantage of being explainable in producing veracity explanations, explicitly considering the semantic features, e.g., claim relevance, richness, salience, and nonredundancy. experimental results on real-world datasets demonstrate the effectiveness of cofced for explainable fake news detection utilizing the wisdom of crowds, effectively mitigating the dependency on fact-checked reports.