Inductively Representing Out-of-Knowledge-Graph Entities by Optimal Estimation Under Translational Assumptions (2021)

Abstract: Conventional Knowledge Graph Completion (KGC) assumes that all test entities appear during training. However, in real-world scenarios, Knowledge Graphs (KG) evolve fast with out-of-knowledge-graph (OOKG) entities added frequently, and we need to efficiently represent these entities. Most existing Knowledge Graph Embedding (KGE) methods cannot represent OOKG entities without costly retraining on the whole KG. To enhance efficiency, we propose a simple and effective method that inductively represents OOKG entities by their optimal estimation under translational assumptions. Moreover, given pretrained embeddings of the in-knowledge-graph (IKG) entities, our method even needs no additional learning. Experimental results on two KGC tasks with OOKG entities show that our method outperforms the previous methods by a large margin with higher efficiency.

## introduction knowledge graphs (kg) play a pivotal role in various nlp tasks, but generally suffer from incompleteness. to address this problem, knowledge graph completion (kgc) aims to predict missing relations in a kg based on knowledge graph embeddings (kge). transductive kge methods, such as transe @xcite and rotate @xcite , achieve success in conventional kgc, which assumes that all test entities appear during training. however, in real-world scenarios, kgs evolve fast with out-of-knowledge-graph (ookg) entities added frequently. to represent these emerging ookg entities, transductive kge methods need to retrain on the whole kg frequently, which when an ookg entity "tenet" is added, we can represent it efficiently via information of its ikg neighbors to predict its missing relations with other entities. in recent years, some inductive methods have been proposed for ookg entities without retraining. @xcite ; @xcite ; @xcite ; @xcite adopt graph neural networks (gnn) to aggregate the ikg neighbors to represent the ookg entities. these methods are effective but require relatively complex calculations, which could be simplified for higher efficiency. @xcite ; @xcite utilize external resources such as entity descriptions or images to enrich the ookg entity embedding, thus avoiding retraining. how-ever, high-quality external resources are expensive to acquire, which may limit the feasibility. in this paper, we propose an inductive method that derives formulas from translational assumptions to estimate ookg entity embeddings. compared to existing methods for kgc with ookg entities, our method has simpler calculations and does not need external resources. for a triplet (h, r, t), translational assumptions of translational distance kge models suppose that embedding h can establish a connection with t via an r-specific operation. assuming that h is an ookg entity and t is an ikg entity, we show that if a translational assumption can derive a specific formula to compute h via pretrained t and r, then there will be no other candidate for h that better fits this translational assumption. therefore, the computed h is the optimal estimation of the ookg entity under this translational assumption. among existing typical kge models, we discover that translational assumptions of transe and rotate can derive these specific estimation formulas. therefore, based on them, we design two instances of our method called invtranse and invrotate, respectively. note that our estimation formulas have no trainable parameters, so our method needs no additional learning when given pretrained ikg embeddings. our contributions are summarized as follows: (1) we propose a simple and effective method to inductively represent ookg entities by their optimal estimation under translational assumptions. (2) our method needs no external resources. given pretrained ikg embeddings, our method even needs no additional learning. (3) we evaluate our method on two kgc tasks with ookg entities. experimental results show that our method outperforms the state-of-the-art methods by a large margin with higher efficiency, and maintains a robust performance even with higher ookg entity ratios. ## related work transductive kge methods map entities and relations to embeddings, and then use score functions to measure the triplet salience. transe @xcite pioneers translational distance methods and is widely-used. it derives a series of methods, such as transh @xcite , transr @xcite , and rotate @xcite . besides, semantic matching methods form another mainstream @xcite @xcite @xcite . these transductive kge methods achieve success in conventional kgc, but to improve efficiency, some inductive methods adopt gnn to aggregate ikg neighbors to produce embeddings for ookg entities @xcite @xcite . these methods are effective but need relatively complex calculations. other inductive methods incorporate external resources to enrich embeddings and represent ookg entities via only external resources @xcite @xcite . however, high-quality external resources are hard and expensive to acquire, which may limit the feasibility. ## conclusion this paper aims to efficiently represent ookg entities. we propose a simple and effective method that inductively represents ookg entities by their optimal estimation under translational assumptions. moreover, given pretrained ikg embeddings, our method needs no additional learning. evaluations on two kgc tasks show that our method outperforms the state-of-the-art methods by a large margin with higher efficiency, and maintains a robust performance with higher ookg entity ratios. from a geometric perspective, hw r hw r is the projection of h on the plane p . from the translational assumption, we can only deduce that the projection of h is equal to v. however, there exist infinitely many possible h that can satisfy this condition. therefore, the solution of h is not unique, and we cannot obtain a specific estimation formula from the translational assumption of transh.