Decker: Double Check with Heterogeneous Knowledge for Commonsense Fact Verification (2023)

Abstract: Commonsense fact verification, as a challenging branch of commonsense question-answering (QA), aims to verify through facts whether a given commonsense claim is correct or not. Answering commonsense questions necessitates a combination of knowledge from various levels. However, existing studies primarily rest on grasping either unstructured evidence or potential reasoning paths from structured knowledge bases, yet failing to exploit the benefits of heterogeneous knowledge simultaneously. In light of this, we propose Decker, a commonsense fact verification model that is capable of bridging heterogeneous knowledge by uncovering latent relationships between structured and unstructured knowledge. Experimental results on two commonsense fact verification benchmark datasets, CSQA2.0 and CREAK demonstrate the effectiveness of our Decker and further analysis verifies its capability to seize more precious information through reasoning. The official implementation of Decker is available at https://github.com/Anni-Zou/Decker .

## introduction commonsense question answering is an essential task in question answering (qa), which requires models to answer questions that entail rich world knowledge and everyday information. the major challenge of commonsense qa is that it not only requires rich background knowledge about how the world works, but also demands the ability to conduct effective reasoning over knowledge of various types and levels @xcite . recently, there emerges a challenging branch of commonsense qa: commonsense fact verification, which aims to verify through facts whether a given commonsense claim is correct or not @xcite √√ figure 1 : an example from csqa2.0 @xcite . given the question, we perform a double check between the heterogeneous knowledge (i.e., kg and facts) and aim to derive the answer by seizing the valued information through reasoning. 2021; @xcite . different from previous multiple-choice settings which contain candidate answers @xcite , commonsense fact verification solely derives from the question itself and implements reasoning on top of it (figure 1 ). therefore, it poses a novel issue of how to effectively seize the useful and valuable knowledge to deal with commonsense fact verification. one of the typical methods is to make direct use of knowledge implicitly encoded in pre-trained language models (plms) @xcite @xcite , which have proved to be useable knowledge bases @xcite . the knowledge in plms is gained during the pre-training stage through mining large-scale collection of unstructured text corpora. nevertheless, the sore spot lies in that it is natural for human brains to project our prior world knowledge onto the answers facing the commonsense questions @xcite , whereas it is tough for plms to learn commonsense knowl-edge that is implicitly stated in plain texts from corpora @xcite . to strengthen plms to perform commonsense qa, there is a surging trend of methods equipping language models with different levels of external knowledge, encompassing structured knowledge such as knowledge graphs (kg) @xcite @xcite and unstructured knowledge such as text corpus @xcite . while the kg-based methods yield remarkable performances on commonsense qa recently, they are more suitable and adaptive for multiple-choice settings because they lay emphasis on discovering connected patterns between the question and candidate answers. for example, to answer a question crabs live in what sort of environment? with candidate answers saltwater, galapagos and fish market, the kg-based methods manage to capture the path crab-sea-saltwater in kg, leading to a correct prediction. nonetheless, they encounter a bottleneck when dealing with commonsense fact verification. figure 1 shows an example: when asked whether july always happens in the summer around the worlds, the kg-based methods have a tendency to detect a strong link between july and summer, which may persuade the model to deliver the wrong prediction. in general, there are two major limitations in previous studies. on one hand, structured knowledge abounds with structural information among the entities but suffers from sparsity and limited coverage. on the other hand, unstructured knowledge provides rich and broad context-aware information but undergoes noisy issues. these two kinds of knowledge can be naturally complementary to each other. however, most existing works focus on either structured or unstructured external knowledge but fail to exploit the benefits of heterogenous knowledge simultaneously. as the example in figure 1 shows: if we rely only on the structured knowledge in kg, we tend to derive that july and summer are strongly correlated, with an extremely weak relationship between summer and winter. similarly, if we focus only on the textual facts, we are more inclined to focus on the fact in grey, as it describes more information about summer in july. as a consequence, uncovering latent relationships among heterogeneous knowledge helps bridge the gap and yield more valuable and useful information. motivated by the above ideas, we propose decker, a commonsense fact verifier that bridges heterogeneous knowledge and performs a double check based on interactions between structured and unstructured knowledge. our proposed decker works in the following steps: (i) firstly, it retrieves heterogeneous knowledge including a kg subgraph and several relevant facts following prior works @xcite ; (ii) secondly, it constructs an integral graph with encoded question and facts and then employs relational graph convolutional networks (r-gcn) to reason and filter over the heterogenous knowledge; (iii) lastly, it adopts a multi-head attention pooling mechanism to obtain a final refinement of enriched knowledge representation and combines it with the question representation for downstream tasks. our contributions are summarized as follows: ## conclusion in this work, we propose decker, a commonsense fact verification model that bridges heterogeneous knowledge and performs a double check based on the interactions between structured and unstructured knowledge. our model not only uncovers latent relationships between heterogeneous knowledge but also conducts effective and fine-grained knowledge filtering of the knowledge. experiments on two commonsense fact verification benchmarks (csqa2.0 and creak) demonstrate the effectiveness of our approach. while most existing works focus on fusing one specific type of knowledge, we open up a novel perspective to bridge the gap between heterogeneous knowledge to gain more comprehensive and enriched knowledge in an intuitive and explicit way.