Topic-Aware Evidence Reasoning and Stance-Aware Aggregation for Fact Verification (2021)

Abstract: Fact verification is a challenging task that requires simultaneously reasoning and aggregating over multiple retrieved pieces of evidence to evaluate the truthfulness of a claim. Existing approaches typically (i) explore the semantic interaction between the claim and evidence at different granularity levels but fail to capture their topical consistency during the reasoning process, which we believe is crucial for verification; (ii) aggregate multiple pieces of evidence equally without considering their implicit stances to the claim, thereby introducing spurious information. To alleviate the above issues, we propose a novel topic-aware evidence reasoning and stance-aware aggregation model for more accurate fact verification, with the following four key properties: 1) checking topical consistency between the claim and evidence; 2) maintaining topical coherence among multiple pieces of evidence; 3) ensuring semantic similarity between the global topic information and the semantic representation of evidence; 4) aggregating evidence based on their implicit stances to the claim. Extensive experiments conducted on the two benchmark datasets demonstrate the superiority of the proposed model over several state-of-the-art approaches for fact verification. The source code can be obtained from https://github.com/jasenchn/TARSA .

## introduction the internet breaks the physical distance barrier among individuals to allow them to share data and information online. however, it can also be used by people with malicious purposes to disseminate misinformation or fake news. such misinformation may cause ethnics conflicts, financial losses and political unrest, which has become one of the greatest threats to the public @xcite ; zhou * corresponding author @xcite . moreover, as shown in @xcite , compared with truth, misinformation diffuses significantly farther, faster, and deeper in all genres. therefore, there is an urgent need for quickly identifying the misinformation spread on the web. to solve this problem, we focus on the fact verification task @xcite , which aims to automatically evaluate the veracity of a given claim based on the textual evidence retrieved from external sources. recent approaches for fact verification are dominated by natural language inference models @xcite or textual entailment recognition models @xcite , where the truthfulness of a claim is verified via reasoning and aggregating over multiple pieces of retrieved evidence. in general, existing models follow an architecture with two main sub-modules: the semantic interaction module and the entailment-based aggregation module @xcite @xcite . the semantic interaction module attempts to grasp the rich semantic-level interactions among multiple pieces of evidence at the sentence-level @xcite @xcite or the semantic roles-level @xcite . the entailment-based aggregation module aims to filter out irrelevant information to capture the salient information related to the claim by aggregating the semantic information coherently. however, the aforementioned approaches typically learn the representation of each evidenceclaim pair from the semantic perspective such as obtaining the semantic representation of each evidence-claim pair through pre-trained language models @xcite or graph-based models @xcite , which largely overlooked the topical consistency between claim and evidence. for example in figure 1 , given the claim "a high school student named cole withrow was claim a high school student named cole withrow was charged for leaving an unloaded shotgun in his vehicle while parking at school. ## related work in general, fact verification is a task to assess the authenticity of a claim backed by a validated corpus of documents, which can be divided into two stages: fact extraction and claim verification @xcite . fact extraction can be further split into the document retrieval phase and the evidence selection phase to shrink the search space of evidence @xcite . in the document retrieval phase, researchers typically reuse the top performing approaches in the fever1.0 challenge to extract the documents with high relevance for a given claim @xcite @xcite . in the evidence selection phase, to select relevant sentences, researchers generally train the classification models or rank models based on the similarity between the claim and each sentence from the retrieved documents @xcite @xcite @xcite . many fact verification approaches focus on the claim verification stage, which can be addressed by natural language inference methods @xcite @xcite . typically, these approaches contain the representation learning process and evidence aggregation process. hanselowski et al. (2018b) and nie et al. (2019a) concatenate all pieces of evidence as input and use the max pooling to aggregate the information for claim verification via the enhanced sequential inference model (esim) @xcite . in a similar vein, @xcite incorporate the identification of evidence to further improve claim verification using esim with different granularity levels. @xcite leverage the co-attention mechanism between claim and evidence to generate claim-specific evidence representations which are used to infer the claim. benefiting from the development of pre-trained language models, @xcite are the first to learn evidence representations by bert @xcite , which are subsequently used in a constructed evidence graph for claim inference by aggregating all claim-evidence pairs. zhong et al. (2020) further establish a semantic-based graph for representation and aggregation with xlnet @xcite . @xcite incorporate two sets of kernels into a sentence-level graph to learn a more fine-grained evidence representations. subramanian and lee (2020) further incorporate evidence set retrieval and hierarchical attention sum block to improve the performance of claim verification. different from all previous approaches, our work for the first time handles the fact verification task by considering the topical consistency and the semantic interactions between claim and evidence. moreover, we employ the capsule network to model the implicit stance relations of evidence toward the claim. ## method in this section, we present an overview of the architecture of the proposed framework tarsa for fact verification. as shown in figure 2 , our approach consists of three main layers: 1) the representation layer to embed claim and evidence into three types of representations by a semantic encoder and a topic encoder; 2) the coherence layer to incorpo-rate the topic information into our model by two attention components; 3) the aggregation layer to model the implicit stances of evidence toward claim using the capsule network. ## experimental setting this section describes the datasets, evaluation metrics, baselines, and implementation details in our experiments. ## experimental results in this section, we evaluate our tarsa model in different aspects. firstly, we compare the overall performance between our model and the baselines. then we conduct an ablation study to explore the effectiveness of the topic information and the capsule network structure. finally, we also explore the advantages of our model in single-hop and multihop reasoning scenarios. ## conclusion we have presented a novel topic-aware evidence reasoning and stance-aware aggregation model for fact verification. our model jointly exploits the topical consistency and the semantic interaction to learn evidence representations at the sentence level. moreover, we have proposed the use of the capsule network to model the implicit stances of evidence toward a claim for a better aggregation of information encoded in evidence. the results on two public datasets demonstrate the effectiveness of our model. in the future, we plan to explore an iterative reasoning mechanism for more efficient evidence aggregation for fact checking.