Modelling Long-distance Node Relations for KBQA with Global Dynamic Graph (2020)

Abstract: The structural information of Knowledge Bases (KBs) has proven effective to Question Answering (QA). Previous studies rely on deep graph neural networks (GNNs) to capture rich structural information, which may not model node relations in particularly long distance due to oversmoothing issue. To address this challenge, we propose a novel framework GlobalGraph , which models long-distance node relations from two views: 1) Node type similarity: GlobalGraph assigns each node a global type label and models long-distance node relations through the global type label similarity; 2) Correlation between nodes and questions: we learn similarity scores between nodes and the question, and model long-distance node relations through the sum score of two nodes. We conduct extensive experiments on two widely used multi-hop KBQA datasets to prove the effectiveness of our method.

## introduction knowledge bases have become critical resources in a variety of natural language processing applications. a kb such as freebase @xcite always contains millions of facts which are composed of subject-predicate-object triples, also referred to as a relation between two entities. such rich structural information has proven effective in kb-based question answering (kbqa) tasks , which aim to find the answer entities to a factoid question using facts in the targeting kb @xcite . early studies on kbqa are mainly based on neural network models @xcite , which simulate the similarity between the factoid question and the entities in the kb. although these methods are effective, the structural information in the kb is not fully utilized, which is essential in the reasoning process @xcite . to address this limitation, recent studies @xcite focus on graph neural networks (gnns), which update nodes by aggregating their neighbor information in graphs. this updated pattern allows gnns to capture structural information. however, gnn is a special form of laplacian smoothing @xcite , stacking multiple gnn layers may oversmooth features of nodes and reduce the discriminative power of graph embedding. with this insufficiency, conventional gnns are poor at modeling long-distance node relations, which is essential for gnn reasoning. @xcite . in this paper, to address the above limitations, we propose a novel framework globalgraph, which models long-distance node relations from two views: 1) modeling node relations by predicting whether two nodes are of the same type label; 2) modeling node relations by predicting whether two nodes are all correlated with the question. for the 1st view, we assign global type labels for each node according to its neighbor relation information, and then model the long-distance node relations by their global label similarity. relations contain node label information, and the relation information around the same type nodes should be similar. for example, as shown in figure 1 , there are two triples: (n 3 , directed by, n 1 ) and (n 4 , directed by, n 5 ). based on the relation "directed by" in these two triples, we regard that n 1 is a person and the same as n 5 . type labels of the two nodes are the same, so we connect the two nodes. based on the new graph, gnn propagates information across long-distance nodes. for the 2nd view, only connect the same label nodes is insufficient, because a normal kb contains a huge number of long-distance node pairs with different labels, which are not utilized for gnn reasoning. for a specific node pair, although the node labels are not similar, if the two nodes are related to the question, the information propagation between them is also useful for reasoning. based on this, we dynamically select nodes related to the current question, and construct a dynamic graph to connect these nodes through full connection. finally, we implement gnn to perform information propagation and reasoning. by solving the two views, we model global node features through their long-distance nodes, and then combine them with the local node features of conventional gnns to perform answer prediction. the main contributions of this paper can be summarized as follows: ## conclusion in this paper, we propose a novel kbqa model based on graph neural network, which can capture long-distance node relations by modeling the relation features of each node and further judge the feature similarity. moreover, our model constructs a dynamic question-aware subgraph, retains the nodes related to the question, and propagates messages on these nodes to improve the reasoning ability. experiments based on two open datasets demonstrate our model's ability on performing answer prediction. ablation experiments prove the validity of each part of the model. case study demonstrates our model's ability to capture long-distance node relations. in the future,we will explore other ways to capture the relation between distant nodes and improve the current proposed model.