ACEN et: Attention Guided Commonsense Reasoning on Hybrid Knowledge Graph (2022)

Abstract: Augmenting pre-trained language models (PLMs) with knowledge graphs (KGs) has demonstrated superior performance on commonsense reasoning. Given a commonsense based QA context (question and multiple choices), existing approaches usually estimate the plausibility of candidate choices separately based on their respective retrieved KGs, without considering the interference among different choices. In this paper, we propose an Attention guided Commonsense rEasoning Network (ACENet) to endow the neural network with the capability of integrating hybrid knowledge. Specifically, our model applies the multi-layer interaction of answer choices to continually strengthen correct choice information and guide the message passing of GNN. In addition, we also design a mix attention mechanism of nodes and edges to iteratively select supporting evidence on hybrid knowledge graph. Experimental results demonstrate the effectiveness of our proposed model through considerable performance gains across CommonsenseQA and OpenbookQA datasets.

## introduction commonsense question answering (csqa) aims to answer questions based on the understanding of context and some background knowledge, which is the critical gap between the human intelligence and machine intelligence @xcite . this capability of owning prior knowledge and reasoning is a foundation for communication and interaction with the world. therefore, commonsense reasoning has become an important research task with various datasets and models proposed in this field @xcite @xcite @xcite . ## methodology the diagram of the proposed acenet is shown in figure 2 . we assume a setting where each example in our data set contains a question q and a set of answer choices {c 1 , c 2 , ..., c n }. we derive the gold answer from qa context and relevant commonsense knowledge. therefore, we retrieve a kg g as the source of commonsense knowledge following prior work @xcite . ## experiments in this section, we conducted experiments over two commonsense qa benchmarks by answering the following research questions. ## conclusions in this paper, we propose a multi-head attention knowledge interaction layer to enhance correct choice information and capture nuances in different choices. meanwhile, the mix attention mechanism of nodes and edges is introduced into message passing to iteratively select relevant knowledge in hybrid knowledge graph. experimental results on commonsenseqa and openbookqa demonstrate the superiority of acenet over other lm+kg methods and the strong performance in handling complex questions. in future work, we plan to further investigate augmenting effects of knowledge graph for reasoning, and integrate neural and symbolic reasoning system to achieve dual system cognitive intelligence.