Modeling Entity Knowledge for Fact Verification (2021)

Abstract: Fact verification is a challenging task of identifying the truthfulness of given claims based on the retrieval of relevant evidence texts. Many claims require understanding and reasoning over external entity information for precise verification. In this paper, we propose a novel fact verification model using entity knowledge to enhance its performance. We retrieve descriptive text from Wikipedia for each entity, and then encode these descriptions by a smaller lightweight network to be fed into the main verification model. Furthermore, we boost model performance by adopting and predicting the relatedness between the claim and each evidence as additional signals. We demonstrate experimentally on a large-scale benchmark dataset FEVER that our framework achieves competitive results with a FEVER score of 72.89% on the test set.

## introduction the rapid development of online applications provides open and efficient platforms for spreading information. however, false information, including fake news and online rumors, have also been growing and spreading widely over the past several years. @xcite shows that false news travels even faster, deeper and broader than the truth. to prevent harm from this false information, automatically verifying the truthfulness of textual contents is becoming an urgent need for our society. in this work, we study fact verification with the goal of automatically assessing the veracity of a textual claim given supporting evidence. most existing methods consider fact verification as a natural language inference task @xcite . usually, these systems concatenate claim and its supporting evidence sentences, and then feed them into a classification model @xcite . alternatively, previous studies construct graph structures based on claim and evidence, and reason over this graph with graph neural networks @xcite or transformer models @xcite , which are used in top systems in the fever challenge @xcite . while these studies focus on reasoning based on claim and evidence text, we believe entity knowledge is also important for precise fact verification. for example, given the first claim from fever dataset in in this work, we present a fact verification model that can effectively incorporate external entity information. given a claim and its evidence sentences, we first recognize named entities from them, linking them with wikipedia articles, and then retrieve the lead sections of these articles as the entity descriptions. to make the most of this entity knowledge while not introducing noisy information, we propose a lightweight entity knowledge encoder module for representing external entity knowledge. our large fact verification network then accesses this knowledge by a unidirectional attention mechanism at each encoding layer. meanwhile, since the input evidence sentences are obtained by an upstream retrieval module, some evidence may be irrelevant to the claim. thus, we predict and adopt this relatedness between each evidence and the claim as an auxiliary signal to train our model. we experiment with our approach on fever @xcite , one influential benchmark dataset for fact verification. fever contains over 185k labeled claims and each verifiable claim is paired with several natural language sentences from wikipedia as their claim #1: assassin's creed has only ever been released on a microsoft and sony platform. ## fever challenge in this paper, we tackle the large-scale challenge for fact extraction and verification: the fever challenge @xcite . it contains 185,445 claims generated by altering sentences extracted from wikipedia. to verify a claim in fever, a model typically follows a three-step pipeline framework, i.e. document retrieval, sentence selection and claim verification. in document retrieval, a system matches the claim to wikipedia articles by extracted named entities and phrases using a search engine built on wikipedia. in sentence selection, a system ranks the sentences from retrieved articles by their similarity scores against the claim. the similarity score can be calculated by a trainable regression model, like enhanced lstm @xcite , or pretrained language models like bert and roberta @xcite . in claim verification, a system classifies the truthfulness of the claim based on top-ranked sentences from the previous step, also known as the evidence sentences. like most participants in this challenge, we adopt existing approaches for document retrieval and sentence selection, while mainly focusing on the claim verification model. ## method in this section, we first formalize the fact verification problem in section 4.1 and then introduce our model for incorporating entity knowledge in section 4.2. finally, we present our complete solution to the fever challenge including document retrieval, evidence selection and entity description collection in section 4.3. ## conclusion in this paper, we present a novel framework for fact verification. when assessing the truthfulness of a claim, we first identify the entities within the claim and evidence, and then retrieve external entity descriptions from wikipedia. we design a decomposable entity knowledge encoder with unidirectional attention for effectively incorporating entity knowledge. furthermore, we propose to use the prediction of input evidence sentences' relatedness as an auxiliary task. experimental results show that our model achieves competitive results on the large-scale fact verification dataset fever. and we conduct ablation studies to showcase the effectiveness of our proposed components.