A Sequential Flow Control Framework for Multi-hop Knowledge Base Question Answering (2022)

Abstract: One of the key challenges of knowledge base question answering (KBQA) is the multi-hop reasoning. Since in different hops, one attends to different parts of question, it is important to dynamically represent the question semantics for each hop. Existing methods, however, (i) infer the dynamic question representation only through coarse-grained attention mechanisms, which may bring information loss, (ii) and have not effectively modeled the sequential logic, which is crucial for the multi-hop reasoning process in KBQA.To address these issues, we propose a sequential reasoning self-attention mechanism to capture the crucial reasoning information of each single hop in a more fine-grained way. Based on Gated Recurrent Unit (GRU) which is good at modeling sequential process, we propose a simple but effective GRU-inspired Flow Control (GFC) framework to model sequential logic in the whole multi-hop process.Extensive experiments on three popular benchmark datasets have demonstrated the superior effectiveness of our model. In particular, GFC achieves new state-of-the-art Hits@1 of 76.8% on WebQSP and is also effective when KB is incomplete. Our code and data are available at https://github.com/Xie-Minghui/GFC.

## introduction knowledge base question answering (kbqa) aims to answer questions from structured knowledge bases. in real application scenarios of kbqa, reasoning with multiple hops over knowledge graph (kg) is necessary for answering complex questions. therefore, how to perform multi-hop reasoning effectively becomes a key challenge for multi-hop kbqa task @xcite @xcite @xcite . existing methods for multi-hop kbqa have three main strands. the first is semantic parsing people tend to pay more attention to current relations while pay less attention to past relations. thus gfc is more consistent with human reasoning habit. however, existing methods still face some critical problems. first, path-based methods and some embedding-based methods usually leverage coarsegrained attention mechanisms to capture reasoning information of each hop. for example, kvmemnn @xcite adopts cross-attention between key-value memory and sentence-level question representation. @xcite uses the sentence-level question representation to eliminate relation embeddings of previous hop. some methods @xcite adopt crossattention between the sentence-level question representation and question tokens. however, compressing all the necessary information into the sentencelevel representation may lose some crucial information. although these methods have achieved good performance, there is still room for improvement. second, they lack modeling sequential logic effectively in the whole multi-hop process. humans often reason sequentially and consider past and present information comprehensively, which is a kind of sequential logic. however, the dynamic question representation of each hop is relatively independent @xcite . and they do not control information flow effectively in different hops. for example, in figure 1 , models need to inhibit past relations for getting the right answer. however, existing methods cannot do this well. in response, we propose a novel model for multihop kbqa, dubbed gfc. first, we design a sequential reasoning self-attention mechanism to obtain more fine-grained reasoning information of each hop. our update mechanism combines the self-attention mechanism with sequential logic in the reasoning scenario. it can capture more nuanced reasoning information to distinguish similar relations on kg. second, we design a simple but effective gru-inspired flow control framework to model the sequential logic in the whole multi-hop process more effectively. this framework controls reasoning information flow among different hops, which enables gfc to consider reasoning information of past and present comprehensively. besides, it tactfully integrates the proposed update mechanism into itself through our heuristic thinking about gru. inspired by the gating mechanism of gru, we also introduce a self-gate unit to filter out redundant past reasoning information. as integral parts of framework, these mechanisms further enhance the capability of the overall flow control framework. our key contributions are as follows: ## related work in this paper, we mainly focus on neural network based methods. ## methodology the diagram of our proposed model gfc is shown in figure 2 . the task of kbqa is to find the answer entities for natural language question q with the help of a relation graph g. the entities mentioned in a question are called topic entities. starting from topic entities, we derive the gold answer entities through the multi-hop reasoning on knowledge graph. our proposed model gfc adopts a sparsematrix reified kb proposed by refiedkb @xcite to represent symbolic knowledge base. this representation method enables our model to perform rapid calculations on large scale knowledge graphs and need no retraining for new entities. ## conclusions in this paper, we design (i) a sequential reasoning self-attention mechanism to extract the crucial reasoning information of each single hop in a more fine-grained way and (ii) a gru-inspired flow control framework to model sequential logic in the whole multi-hop process more effectively. experimental results show the superior performance of gfc. specially, gfc achieves new state-of-the-art hits@1 performance on webqsp. gfc also shows its high effectiveness when kb is incomplete. as a path-based method, gfc not only has better interpretability and extensibility, but also has better performance. in future work, we plan to investigate further on how to model the multi-hop reasoning process using the structures of language models.