Relation-Aware Question Answering for Heterogeneous Knowledge Graphs (2023)

Abstract: Multi-hop Knowledge Base Question Answering(KBQA) aims to find the answer entity in a knowledge graph (KG), which requires multiple steps of reasoning. Existing retrieval-based approaches solve this task by concentrating on the specific relation at different hops and predicting the intermediate entity within the reasoning path. However, these models fail to utilize information from head-tail entities and the semantic connection between relations to enhance the current relation representation, which undermines the information capturing of relations in KGs. To address this issue, we construct a dual relation graph where each node denotes a relation in the original KG ( primal entity graph ) and edges are constructed between relations sharing same head or tail entities. Then we iteratively do primal entity graph reasoning, dual relation graph information propagation, and interaction between these two graphs. In this way, the interaction between entity and relation is enhanced, and we derive better entity and relation representations. Experiments on two public datasets, WebQSP and CWQ, show that our approach achieves a significant performance gain over the prior state-of-the-art.

## introduction knowledge base question answering (kbqa) is a challenging task that aims to answer natural language questions with a knowledge graph. with the fast development of deep learning, researchers leverage end-to-end neural networks @xcite to solve this task. recently there has been more and more interest in solving complicated questions where multiple steps of reasoning are required to identify the answer entity @xcite . such a complicated task is referred to as multi-hop kbqa. there are two groups of methods in kbqa; information retrieval (ir) and semantic parsing (sp). sp-based methods need executable logical forms as supervision, which are hard to obtain in real scenes. ir-based approaches, which retrieves relevant entities and relations to establish the question specific subgraph and learn entity representation by reasoning over question-aware entity graph with gnn @xcite , are widely used to solve multihop kbqa @xcite @xcite . these systems are trained without logical forms, which is a more realistic setting as these are hard to obtain. we focus on the retriever-reasoner approach to multi-hop kbqa without the addition of logical form data. during the reasoning process of existing methods, the representation of relations are fixed but the initial relation representation may not be optimal. we claim the semantic connection between relations which is co-relation and the semantic information from head and tail (head-tail) entities to build relation representation is neglected. we take one case in webqsp dataset @xcite as an example, where the question is "through which countries of the sahel does the niger river flow?". in figure 1 , to answer the question, the model has to hop from the topic entity "niger river" by the reasoning path "con-tained_by". on the other hand, the candidate entity "niger"(country) should connect to "sahel" with relation "contains" to satisfy the constraint "of the sahel" to be the answer. so one crucial point for the model to retrieve the answer is to understand the meaning of relation "contained_by" and "contains". on one hand, the relation "con-tained_by" and "contains" have head-tail entities of type "location" like "niger river", "guinea", "africa" and so on. the semantic information of entity type narrows down the meaning scope of relation "contained_by" and "contains" into locative relations. on the other hand, the massive intersection between head-tail entity sets of relation"contained_by" and "contains" implies the high semantic connection of these two relations, and we assume the representation learning for one of relations can enhance the other. if the model discovers the reverse relationship between "con-tained_by" and "contains", the understanding of one relation can help construct the representation of the other. to utilize the semantic information from headtail entities and the co-relation among relations, we propose our framework of unified reasoning of primal entity graph and dual relation graph. we construct the dual relation graph by taking all relations in the primal entity graph as nodes and linking the relations which share the same head or tail entities. in each hop, first we reason over primal entity graph by attention mechanism under question instructions, and then we do information propagation on dual relation graph by gat @xcite to incorporate neighbor relation information into the current relation, at last we merge the semantic information of head-tail entities by the assumption of transe @xcite to enhance the relation representation. we conduct the iteration for multiple rounds to ensure that the representation of relations and entities can stimulate each other. we do experiments on two benchmark datasets in the field of kbqa and our approach outperforms the existing methods by 0.8% hits@1 score and 1.7% f1 score in webqsp, 1.5% hits@1 score and 5.3% f1 score in cwq @xcite , becoming the new state-of-the-art. our contributions can be summarized as three folds: ## related work over the last decade, various methods have been developed for the kbqa task. early works utilize machine-learned or hand-crafted modules like entity recognition and relation linking to find out the answer entity @xcite @xcite . with the popularity of neural networks, recent researchers utilize end-to-end neural networks to solve this task. they can be categorized into two groups: semantic parsing based methods (sp-based) @xcite @xcite and information retrieval based methods (ir-based) @xcite @xcite . spbased methods convert natural language questions into logic forms by learning a parser and predicting the query graph step by step. however, the predicted graph is dependent on the prediction of last step and if at one step the model inserts the incorrect intermediate entity into the query graph, the prediction afterward will be unreasonable. spbased methods need the ground-truth executable queries as supervision and the ground-truth executable query forms bring much more cost to annotate in practical. ir-based methods first retrieve relevant knowledge graph(kg) elements to construct the question-specific subgraph and reason over it to derive answer distribution by gnn. zhang et al. (2018) introduce variance reduction in retrieving answers from the knowledge graph. under the setting of supplemented corpus and incomplete knowledge graph, sun et al. (2019) propose pullnet to learn what to retrieve from a corpus. different from embedkgqa (saxena et al., 2020), which utilizes kg embeddings over interaction between the question and entities to reduce kg sparsity, we apply transe in incorporating the head-tail entities information to enhance the relation representation. shi et al. (2021) propose transfernet to support both label relations and text relations in a unified framework, for purpose of enhancing the supervision of intermediate entity distribution, he et al. (2021) adopt the teacher-student network in multihop kbqa. however, it ignores the information from head-tail entities and semantic connection with neighbor relations to build the representation of current relation. ## preliminary in this part, we introduce the concept of the knowledge graph and the definition of multi-hop knowledge base question answering task(kbqa). ## analysis in this section, we probe our performance on cases with different sizes and connectivity of dual relation graph and primal entity graph. then we explore the benefit on relation and entity representation by introducing dual-primal graph interaction along with dual graph propagation. furthermore, we do 3 ablation studies to evaluate different mod-ules in our approach and explore our performance with different reasoning steps n. ## conclusion in this paper, we propose a novel and efficient framework of iteratively primal entity graph reasoning, dual relation graph information propagation, and dual-primal graph interaction for multi-hop kbqa over heterogeneous graphs. our approach incorporates information from head-tail entities and semantic connection between relations to derive better relation and entity representations. we conduct experiments on two benchmark datasets in this field and our approach outperforms all the existing methods.