Learning Query Adaptive Anchor Representation for Inductive Relation Prediction (2023)

Abstract: Relation prediction on knowledge graphs (KGs) attempts to infer the missing links between entities. Most previous studies are limited to the transductive setting where all entities must be seen during the training, making them unable to perform reasoning on emerging entities. Recently, the inductive setting is proposed to handle the entities in the test phase to be unseen during training, However, it suffers from the inefficient reasoning under the enclosing subgraph extraction issue and the lack of effective entity-independent feature modeling. To this end, we propose a novel Query Adaptive Anchor Representation (QAAR) model for inductive relation prediction. First, we extract one opening subgraph and perform reasoning by one time for all candidate triples, which is more efficient when the number of candidate triples is large. Second, we define some query adaptive anchors which are independent on any specific entity. Based on these anchors, we take advantage of the transferable entity-independent features (relation-aware, structure-aware and distance features) that can be used to produce entity embeddings for emerging unseen entities. Such entity-independent features is modeled by a query-aware graph attention network on the opening subgraph. Experimental results demonstrate that our proposed QAAR outperforms state-of-the-art baselines in inductive relation prediction task.

## introduction knowledge graphs (kgs) store a large amount of facts regarding the real-world knowledge in the form of (h, r, t), where h, t are head and tail entities and r is the relation between h and t. they have played important roles in various downstream applications, including but not limited to question answering @xcite , semantic search @xcite and many more. nowadays many * corresponding author. the conventional relation prediction models, such as transe @xcite , distmult @xcite , rotate @xcite , tucker @xcite , compgcn @xcite , kbgat @xcite , reinceptione @xcite , dis-enkgat @xcite , pairre @xcite , fftatth @xcite , kgtuner @xcite have successively achieved satisfactory performance on relation prediction. they all follow a general strategy to represent the entities in the low-dimension vector space by using the kg structure and neighborhood information in the transductive setting, where all entities must be seen in the training phase. however, many kgs are evolving and new entities are emerging every day in the real world. these conventional relation prediction models in the transductive setings cannot effectively learn the embeddings for the new entities unless re-training the whole kgs from scrath. recently, researchers have focused on relation prediction in the inductive setting, where new entities are emerging while testing. compared to relation prediction in the transductive setting, the inductive setting is more challenging since the missing links can be inferred in kgs with unseen entities. currently, models on relation predictions in the inductive setting can be divided into two categories @xcite : rule-based methods and gcn-based methods. rule-based methods (e.g., neurallp @xcite , drum @xcite , rulen (meilicke et al., 2018) etc.) are able to handle the relation prediction with new/unseen entities since the rules are independent of any specific entities, their performance is usually insufficient due to the limitation of the flexible issue. gcn-based methods, such as grail @xcite , compile @xcite , meta-ikg @xcite , tact @xcite , conglr @xcite , require to extract a k-hop enclosing subgraph 1 for every candidate triple (h, r, t) and then perform reasoning on all these candidate subgraphs. however, these methods suffer from the inefficient issue since the number of candidate triples is usually large. figure 1 shows an example of the query (barack obama, nationality, ?). the subfigures (b), (c) and (d) are enclosing subgraphs extracted from graph (a) for three different candidate triples. though the subgraphs (b) and (c) have the same structure, these previous methods have to perform reasoning two times, which will spend more reasoning time. moreover, previous methods only use the distance as features and ignore rich local structure information in subgraphs, which leads to sub-optimal performance. as shown in figure 1 , the entity honolulu in subfigure (b) is labeled as "(1,1)", which is the same with the entity malik obama in subfigure (d). this leads to similar node labeling for the nodes in (b) and (d), although these two subgraphs are totally different. using these weak node features will result in poor performance since it is hard for the model to distinguish different subgraphs. 1 following @xcite , a k-hop enclosing subgraph between nodes h and t contains all nodes that appear on the graph from h to t with the maximum k-hop distance. based on the above analysis, we are motivated to address two research issues: (1) how to develop a model to perform inductive reasoning for all candidate triples with only one subgraph and one-time node labeling and reasoning. (2) how to extract adequate transferable entity-independent anchor foot_0 knowledge to produce more expressive entity embeddings for emerging entities. to address these issues, we propose a novel query adaptive anchor representation (qaar) model. specifically, for a given query (h, r, ?) where ? denotes a missing entity to be inferred, instead of extracting an enclosing subgraph for every candidate triple, we first extract a k-hop opening subgraph foot_1 surrounding the query entity h for all candidates. this enables our model to perform reasoning by one time to rank all candidates, which is more efficient when the number of candidates is large. then, we propose a dynamic anchor-based node labeling method to label each node with rich entity-independent information. specifically, we construct three kinds of anchors for each query (h, r, ?), including relation types, the center node of the subgraph and the one-hop neighbors of the center node which is adaptively selected according to the query entity h. based on these anchors, we can label each node in the subgraph with three kinds of features, namely relation-aware, structure-aware and distance features. these rich node features enable our model to learn expressive representation for subgraphs. finally, a query-aware graph attention neural network is proposed to perform reasoning on subgraphs by gathering query-related neighborhood features. experimental results on three benchmark datasets demonstrate that our proposed qaar significantly outperforms previous methods, and achieves the new state-of-the-art (sota) performance for inductive relation prediction.