R easoning LM : Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph (2023)

Abstract: Question Answering over Knowledge Graph (KGQA) aims to seek answer entities for the natural language question from a large-scale Knowledge Graph (KG). To better perform reasoning on KG, recent work typically adopts a pre-trained language model (PLM) to model the question, and a graph neural network (GNN) based module to perform multi-hop reasoning on the KG. Despite the effectiveness, due to the divergence in model architecture, the PLM and GNN are not closely integrated, limiting the knowledge sharing and fine-grained feature interactions. To solve it, we aim to simplify the above two-module approach, and develop a more capable PLM that can directly support subgraph reasoning for KGQA, namely ReasoningLM. In our approach, we propose a subgraph-aware self-attention mechanism to imitate the GNN for performing structured reasoning, and also adopt an adaptation tuning strategy to adapt the model parameters with 20,000 subgraphs with synthesized questions. After adaptation, the PLM can be parameter-efficient fine-tuned on downstream tasks. Experiments show that ReasoningLM surpasses state-of-the-art models by a large margin, even with fewer updated parameters and less training data. Our codes and data are publicly available at https://github.com/RUCAIBox/ReasoningLM.

## introduction question answering over knowledge graph (kgqa) @xcite has garnered significant attention in recent years, which aims to find answers for the natural language questions based on knowledge graphs (kgs), e.g., freebase @xcite and @xcite . since a massive amount of world knowledge has been formatted into a structured form (e.g., a triple ⟨head, relation, tail⟩) in the * corresponding author. kg, we can develop kgqa methods by leveraging structural semantics of kg to more accurately infer the answer entities to complex factual questions. starting from the topic entities mentioned in the question, a typical kgqa approach @xcite is to perform the multi-hop reasoning along with relations on the kg, until finding a path that can reach the answer entities. to develop this approach, existing methods @xcite @xcite mostly incorporate a text encoder to produce the representation of the given question, and a reasoning module to perform multihop reasoning on the kg using the question representation. typically, recent work @xcite adopts the pre-trained language models (plm) (e.g., bert @xcite ) and graph neural networks (gnn) (e.g., gat @xcite ) to implement the text encoder and reasoning module respectively, which can better understand the semantic information in the question and structured knowledge from the kg, improving the final performance despite the effectiveness, there are two major shortcomings with the aforementioned approach that combines a plm encoder and a gnn reasoner. first, due to different model architectures, plm and gnn are often integrated in a loose way (e.g., relevance score sharing), which largely limits the knowledge sharing and fine-grained interaction between the question and kg (a subgraph spanned by related entities). second, the gnn based reasoner performs reasoning mainly based on subgraph structure, which lack rich semantic knowledge as that in plms, making the reasoning results likely to be less effective, especially for complex questions. in addition to the two shortcomings, such a approach also requires a complicated implementation in practice, since it involves in two different modules. to address these issues, we aim to simplify the above two-module approach and develop a more capable plm that can directly support structural subgraph reasoning for kgqa. our approach is inspired by the finding that the transformer architecture consisting of stacked self-attention modules can be explained as a fully-connected graph encoder @xcite in a mathematically equivalent way. therefore, transformer essentially has the potential to effectively model graph data and further performs graph reasoning, which has been shown in existing work @xcite . while, these attempts based on graph transformers either neglect the modeling in text semantics, or cannot capture fine-grained semantic interaction between question and kg subgraph, making them infeasible for kgqa tasks. to this end, in this paper, we propose a subgraph reasoning enhanced plm, called reasoninglm, enabling both effective question understanding and kg reasoning in a unified approach. as the major contribution, we propose a subgraph-aware selfattention mechanism, which can imitate the gnn to model the entities and their relations via attending to neighboring nodes on the kg. further, such a structural attention mechanism has been integrated into a constrained masking framework, to jointly model question attention, kg to question attention, and kg attention. in this way, we can not only perform the knowledge interaction and sharing between the question and kg, but also leverage plm to perform structural reasoning as gnn. however, since the plm is originally trained by learning from general-purpose natural language text, it is necessary to adapt it to the special input format and attention mechanism. thus, we propose an adaptation tuning strategy that utilizes 20,000 subgraphs with synthesized questions to adapt the parameters of the plm. after adaptation, the plm has been well adapted for subgraph reasoning, hence it can be fine-tuned on different downstream kgqa tasks in a parameter-efficient manner, achieving better performance with only a few parameters trained. to evaluate the effectiveness of our approach, we conduct extensive experiments on three kgqa datasets. experimental results demonstrate that our proposed approach can surpass existing state-ofthe-art models by a large margin, even with fewer updated parameters and less training data. our contributions can be summarized as follows: plm for knowledge graph reasoning. besides kgqa, plm has also been used for other knowledge graph reasoning tasks, such as commonsense reasoning @xcite or predicting missing facts @xcite . there are mainly two methods for plm to leverage the kg. several studies @xcite attempt to fuse the representation of the kg into the plm, which is modeled by the gnn. it can make the plm aware of the kg to some extent through the modeled representation. however, it's not easy to bridge the gap between the plm and gnn given the different model architecture and initialized parameters, leading to a sufficient understanding of kg for plm. in contrast, a more direct way is to linearize the kg as a sequence and input it to plms @xcite . in this way, the plm can directly utilize the kg to perform reasoning. despite its simplicity, such a way neglects the structure of kg, which is an important feature. in contrast, we propose to model the question and kg in a single plm, while capturing the structure information with subgraph-aware self-attention mechanism. ## preliminary in this section, we present the notations utilized throughout the paper, followed by a formal definition of the kgqa task. ## approach in this section, we present our proposed approach, i.e., reasoninglm, which enables both effective question understanding and kg reasoning in a single plm for better solving the kgqa task. ## conclusion in this work, we proposed a subgraph reasoning enhanced plm to support question understanding and kg reasoning in a single plm, namely reasoninglm. in our approach, we first adopted a bfs-based subgraph serialization to enable the plm to capture the structured information and then proposed a subgraph-aware self-attention mechanism to support graph reasoning within the plm based on the serialized subgraph. in order to adapt the plm to the special input format and attention mechanism, we further utilized an adaptation tuning strategy with a cheap data construction cost of 15 dollars by using chatgpt in an automatic way. finally, we applied the reasoninglm to solve downstream kgqa tasks with parameter-efficient fine-tuning. experimental results have shown that our approach can significantly improve the performance compared to existing strong baselines by updating only 1m parameters.