Heterogeneous-Graph Reasoning and Fine-Grained Aggregation for Fact Checking (2022)

Abstract: Fact checking is a challenging task that requires corresponding evidences to verify the property of a claim based on reasoning. Previous studies generally i) construct the graph by treating each evidence-claim pair as node which is a simple way that ignores to exploit their implicit interaction, or building a fully-connected graph among claim and evidences where the entailment relationship between claim and evidence would be considered equal to the semantic relationship among evidences; ii) aggregate evidences equally without considering their different stances towards the verification of fact. Towards the above issues, we propose a novel heterogeneous-graph reasoning and fine-grained aggregation model, with two following modules: 1) a heterogeneous graph attention network module to distinguish different types of relationships within the constructed graph; 2) fine-grained aggregation module which learns the implicit stance of evidences towards the prediction result in details. Extensive experiments on the benchmark dataset demonstrate that our proposed model achieves much better performance than state-of-the-art methods.

## introduction today, social media is considered as the biggest platform to share news and seek information. however, misinformation is spreading at increasing rates and may cause great impact to society. the reach of fake news was best highlighted during the critical months of the 2016 u.s. presidential election generated millions of shares and comments on facebook @xcite . therefore, automatic detection of fake news on social media has become a significant and beneficial problem. we pay more attention on fact checking task, which utilizes external knowledge to determine the claim veracity when given a claim. verifying the truthfulness of a claim with respect to evidence can be regarded as a special case of recognizing textual entailment (rte) @xcite or natural language inference (nli) @xcite . typically, existing approaches contain the representation learning process and evidence aggregation process. representation process tries to enhance the semantic expression of claim and evidence via sequence structure methods @xcite or graph based neural networks @xcite where they utilize simple combination methods such as just dealing with claim-evidence pair as graph nodes. the evidence aggregation process aims to find out the most important evidence which contributes more to claim verification with different methods like mean pooling, attention-based aggregation, etc. however, existing approaches such as @xcite establish a semantic-based graph, which ignore the difference between relationships among nodes in reasoning graph. for example in figure 1 , given the claim "al jardine is an american rhythm guitarist." and the retrieved evidence sentences (i.e., e1-e5), making the correct prediction requires model to reason that "al jardine" is the person mentioned in e2 and "rhythm guitarist" is occurred in e1 based on the entailment interaction of claim with the evidences. furthermore, we also expect the semantical coherence of multiple evidences from e1 to e5 to automatically filter unrelated evidence such as e3-e5. we believe it's crucial for verification to mine distinct relationships within the reasoning graph. besides, in previous methods @xcite , stance of evidences towards claim are aggregated equally or some irrelevant evidences are prevented from predicting the veracity of claim roughly via simple attention mechanism. however, each piece of evidence has a different impact on the claim, which needs to be exploited on fine-grained perspective. to alleviate above issues, we propose a novel heterogeneous-graph reasoning and finegrained aggregation model (hgrga), which not only enhances the representation learning for claim and evidences by capturing different types of relationships within the constucted graph but also aggregating stances of evidences towards claim concretely. more specifically, we construct a heterogeneous evidence-evidence-claim graph based on graph attention network to enhance the representation of claim and evidences. besides, we utilize an capsule network to further aggregate evidences with different implicit stances towards the claim, and learn the weights via dynamic routing which indicate how each of evidence attributes the veracity of claim. we conduct experiments on the real-world benchmark dataset. extensive experimental results demonstrate the effectiveness of our model. hgrga boosts the performance for fact checking and the main contributions of this work are summarized as follows: ## proposed method in this section, we present an overview of the architecture of the proposed framework hgrga for fact verification. as shown in figure 2 , given a claim and the retrieved evidence, we first utilize a sentence encoder to obtain representations for the claim and the evidences. then we build a heterogeneous evidence-evidence-claim graph to propagate information among claim and evidence. finally, we use the capsule network to model the implicit stances of evidences towards claim on fine-grained perspective. ## experimental results in this section, we first present the overall performance of our model hgrga compared with other approaches. then we conduct an ablation study to explore the effectiveness of the heterogeneous graph structure and the fine-grained capsule net- ## error analysis we randomly select 200 incorrectly predicted instances and summarize the primary types of errors. the first type of errors is caused by failing to match the semantic meaning of some phrases on some complex cases. for example, the claim "philomena is a film nominated for seven awards." is supported by the evidence "it was also nominated for four bafta awards and three golden globe awards." the model needs to understand that four plus three equals seven in this case. another case is that the claim states "winter's tale is a book", while the evidence states "winter's tale is a 1983 novel by mark helprin". the model fails to understand the relationship between novel and book. solving this type of problem requires the incorporation of additional knowledge, such as math logic and common sense. the second type of errors is due to the failure of retrieving relevant evidences. for example, the claim states "lyon is a city in southwest france.", and the ground-truth evidence states "lyon had a population of 506,615 in 2014 and is france's third-largest city after paris and marseille.", which gives not enough information to help model make a true judgement. ## consultion in this work, we present a novel heterogeneousgraph reasoning and fine-grained aggregation framework on the claim verification subtask of fever. we propose heterogeneous graph attention network to better exploit different types of relationships between nodes within reasoning graph. furthermore, the capsule network is used to observe fine-grained distributions of stances towards claim from multiple pieces of evidence. the framework is proven to be effective and achieve significant and explainable performance. in the future, we would like to explore a fine-grained reasoning mechanism within graph and jointly learn evidence selection and claim verification.