F act KG : Fact Verification via Reasoning on Knowledge Graphs (2023)

Abstract: In real world applications, knowledge graphs (KG) are widely used in various domains (e.g. medical applications and dialogue agents). However, for fact verification, KGs have not been adequately utilized as a knowledge source. KGs can be a valuable knowledge source in fact verification due to their reliability and broad applicability. A KG consists of nodes and edges which makes it clear how concepts are linked together, allowing machines to reason over chains of topics. However, there are many challenges in understanding how these machine-readable concepts map to information in text. To enable the community to better use KGs, we introduce a new dataset, FactKG: Fact Verificationvia Reasoning on Knowledge Graphs. It consists of 108k natural language claims with five types of reasoning: One-hop, Conjunction, Existence, Multi-hop, and Negation. Furthermore, FactKG contains various linguistic patterns, including colloquial style claims as well as written style claims to increase practicality. Lastly, we develop a baseline approach and analyze FactKG over these reasoning types. We believe FactKG can advance both reliability and practicality in KG-based fact verification.

## introduction the wide spread risk of misinformation has increased the demand for fact-checking, that is, judging whether a claim is true or false based on evidence. accordingly, recent works on fact verification have been developed with various sources of evidence, such as text @xcite @xcite @xcite and tables @xcite @xcite . unfortunately, knowledge graphs (kg), one of the large-scale data forms, have not yet been fully utilized as a source of evidence. a kg is a valuable knowledge source due to two advantages. firstly, kg-based fact verification can provide more reliable reasoning: since the efficacy of realworld fact-checking hinges on this reliability, recent studies have focused on justifying the decisions of a fact verification system @xcite . in most existing works, the justification is based on the extractive summary of text evidence. therefore, the inferential links between the evidence and the verdict are not clear @xcite . compared to text and tables, a kg can simply represent reasoning process with logic rules on nodes and edges @xcite . this allows us to categorize common types of reasoning with the graphical structure, as shown in secondly, kg-based fact verification techniques have broad applicability beyond the domain of factchecking. for example, modern dialogue systems (e.g. amazon alexa (amazon staff, 2018), google assistant (kale and hewavitharana, 2018)) maintain and communicate with internal knowledge graphs, and it is crucial to make sure that their content is consistent with what the user says and otherwise update the knowledge graphs accordingly. if we model the user's utterance as a claim and the dialogue system's internal knowledge graph as a knowledge source, the process of checking their consistency can be seen as a form of kg-based fact verification task. more generally, kg-based fact verification techniques can be applied to cases which require checking the consistency between graphs and text. reflecting these advantages, we introduce a new dataset, factkg: fact verification via reasoning on knowledge graphs, consisting of 108k textual claims that can be verified against dbpedia @xcite and labeled as sup-ported or refuted. we generated the claims based on graph-text pairs from webnlg @xcite to incorporate various reasoning types. the claims in factkg are categorized into five reasoning types: one-hop, conjunction, existence, multi-hop, and negation. furthermore, factkg consists of claims in various styles including colloquial, making it potentially suitable for a wider range of applications, including dialogue systems. we conducted experiments on factkg to validate whether graph evidence had a positive effect for fact verification. our experiments indicate that the use of graphical evidence in our model resulted in superior performance when compared to baselines that did not incorporate such evidence. ## data construction our goal is to diversify the graph reasoning patterns and linguistic styles of the claims. to achieve this, we categorize five reasoning types of claims: one-hop, conjunction, existence, multi-hop, and negation. our claims are generated by transforming the sentences in s w , a subset of webnlg's text-graph pairs (section 3.1). 3 next, we also diversified the claims with colloquial style transfer and presupposition (section 3.2). ## conclusion in this paper, we present factkg, a new dataset for fact verification using knowledge graph. in order to reveal the relationship between fact verification and knowledge graph reasoning, we generated claims corresponding to a certain graph pattern. additionally, factkg also includes colloquialstyle claims that are applicable to the dialogue system. our analysis showed that the claims in our dataset are difficult to solve without reasoning over the knowledge graph. we expect the dataset to offer various research directions. one possible use of our dataset is as a benchmark for justification prediction. most research on this task generate a text passage as justification, yet this approach lacked a gold reference. on the contrary, the interpretability of the knowl-edge graph allows us to employ it as an explanation for the verdict, such as question answering in the medical domain where explainability is important. furthermore, using the kg structure for the claim generation allows us to generate a dataset with more complex multi-hop reasoning by design without relying on annotator creativity.