DTCA : Decision Tree-based Co-Attention Networks for Explainable Claim Verification (2020)

Abstract: Recently, many methods discover effective evidence from reliable sources by appropriate neural networks for explainable claim verification, which has been widely recognized. However, in these methods, the discovery process of evidence is nontransparent and unexplained. Simultaneously, the discovered evidence is aimed at the interpretability of the whole sequence of claims but insufficient to focus on the false parts of claims. In this paper, we propose a Decision Tree-based Co-Attention model (DTCA) to discover evidence for explainable claim verification. Specifically, we first construct Decision Tree-based Evidence model (DTE) to select comments with high credibility as evidence in a transparent and interpretable way. Then we design Co-attention Self-attention networks (CaSa) to make the selected evidence interact with claims, which is for 1) training DTE to determine the optimal decision thresholds and obtain more powerful evidence; and 2) utilizing the evidence to find the false parts in the claim. Experiments on two public datasets, RumourEval and PHEME, demonstrate that DTCA not only provides explanations for the results of claim verification but also achieves the state-of-the-art performance, boosting the F1-score by more than 3.11%, 2.41%, respectively.

## introduction the increasing popularity of social media has brought unprecedented challenges to the ecology of information dissemination, causing rampancy of a large volume of false or unverified claims, like extreme news, hoaxes, rumors, fake news, etc. research indicates that during the us presidential election (2016), fake news accounts for nearly 6% of all news consumption, where 1% of users are exposed to 80% of fake news, and 0.1% of users are responsible for sharing 80% of fake news @xcite , and democratic elections are vulnerable to manipulation of the false or unverified claims on social media @xcite , which renders the automatic verification of claims a crucial problem. currently, the methods for automatic claim verification could be divided into two categories: the first is that the methods relying on deep neural networks learn credibility indicators from claim content and auxiliary relevant articles or comments (i.e., responses) @xcite @xcite . despite their effectiveness, these methods are difficult to explain why claims are true or false in practice. to overcome the weakness, a trend in recent studies (the second category) is to endeavor to explore evidencebased verification solutions, which focuses on capturing the fragments of evidence obtained from reliable sources by appropriate neural networks @xcite @xcite . for instance, @xcite build multi-task learning to extract evidence from wikipedia and synthesize information from multiple documents to verify claims. @xcite capture signals from external evidence articles and model joint interactions between various factors, like the context of a claim and trustworthiness of sources of related articles, for assessment of claims. @xcite propose hierarchical attention networks to learn sentence-level evidence from claims and their related articles based on coherence modeling and natural language inference for claim verification. although these methods provide evidence to solve the explainability of claim verification in a manner, there are still several limitations. first, they are generally hard to interpret the discovery process of evidence for claims, namely, lack the interpretability of methods themselves because these methods are all based on neural networks, belong-ing to nontransparent black box models. secondly, the provided evidence only offers a coarse-grained explanation to claims. they are all aimed at the interpretability of the whole sequence of claims but insufficient to focus on the false parts of claims. to address the above problems, we design decision tree-based co-attention networks (dtca) to discover evidence for explainable claim verification, which contains two stages: 1) decision tree-based evidence model (dte) for discovering evidence in a transparent and interpretable way; and 2) co-attention self-attention networks (casa) using the evidence to explore the false parts of claims. specifically, dte is constructed on the basis of structured and hierarchical comments (aiming at the claim), which considers many factors as decision conditions from the perspective of content and meta data of comments and selects high credibility comments as evidence. casa exploits the selected evidence to interact with claims at the deep semantic level, which is for two roles: one is to train dte to pursue the optimal decision threshold and finally obtain more powerful evidence; and another is to utilize the evidence to find the false parts in claims. experimental results reveal that dtca not only achieves the state-of-the-art performance but also provides the interpretability of results of claim verification and the interpretability of selection process of evidence. our contributions are summarized as follows: ## related work claim verification many studies on claim verification generally extract an appreciable quantity of credibility-indicative features around semantics @xcite @xcite , emotions @xcite , stances @xcite @xcite , write styles @xcite , and source credibility @xcite ) from claims and relevant articles (or comments). for a concrete instance, @xcite devise sifted multi-task learning networks to jointly train stance detection and fake news detection tasks for effectively utilizing common features of the two tasks to improve the task performance. despite reliable performance, these methods for claim verification are unexplainable. to address this issue, recent research concentrates on the discovery of evidence for explainable claim verification, which mainly designs different deep models to exploit semantic matching @xcite , semantic conflicts @xcite @xcite , and semantic entailments @xcite between claims and relevant articles. for instance, @xcite develop neural semantic matching networks that encode, align, and match the semantics of two text sequences to capture evidence for verifying claims. combined with the pros of recent studies, we exert to perceive explainable evidence through semantic interaction for claim verification. explainable machine learning our work is also related to explainable machine learning, which can be generally divided into two categories: intrinsic explainability and post-hoc explainability @xcite . intrinsic explainability @xcite @xcite ) is achieved by constructing self-explanatory models that incorporate explainability directly into their structures, which requires to build fully interpretable models for clearly expressing the explainable process. however, the current deep learning models belong to black box models, which are difficult to achieve intrinsic explainability @xcite . post-hoc explainability @xcite @xcite needs to design a second model to provide explanations for an existing model. for example, @xcite combine the strengths of the embeddings-based model and the tree-based model to develop explainable recommendation, where the tree-based model obtains evidence and the embeddings-based model improves the performance of recommendation. in this paper, following the post-hoc explainability, we harness decision-tree model to explain the discovery process of evidence and design co-attention networks to boost the task performance. ## decision tree-based co-attention networks (dtca) in this section, we introduce the decision treebased co-attention networks (dtca) for explainable claim verification, with architecture shown in figure 1 , which involves two stages: decision treebased evidence model (dte) and co-attention selfattention networks (casa) that consist of a 3-level hierarchical structure, i.e., sequence representation layer, co-attention layer, and output layer. next, we describe each part of dtca in detail. ## experiments as the key contribution of this work is to verify claims accurately and offer evidence as explanations, we design experiments to answer the following questions: ## conclusion we proposed a novel framework combining decision tree and neural attention networks to explore a transparent and interpretable way to discover evidence for explainable claim verification, which constructed decision tree model to select comments with high credibility as evidence, and then designed co-attention networks to make the evidence and claims interact with each other for unearthing the false parts of claims. results on two public datasets demonstrated the effectiveness and explainability of this framework. in the future, we will extend the proposed framework by considering more context (meta data) information, such as time, storylines, and comment sentiment, to further enrich our ex-