R at E : Relation-Adaptive Translating Embedding for Knowledge Graph Completion (2020)

Abstract: Many graph embedding approaches have been proposed for knowledge graph completion via link prediction. Among those, translating embedding approaches enjoy the advantages of light-weight structure, high efficiency and great interpretability. Especially when extended to complex vector space, they show the capability in handling various relation patterns including symmetry, antisymmetry, inversion and composition. However, previous translating embedding approaches defined in complex vector space suffer from two main issues: 1) representing and modeling capacities of the model are limited by the translation function with rigorous multiplication of two complex numbers; and 2) embedding ambiguity caused by one-to-many relations is not explicitly alleviated. In this paper, we propose a relation-adaptive translation function built upon a novel weighted product in complex space, where the weights are learnable, relation-specific and independent to embedding size. The translation function only requires eight more scalar parameters each relation, but improves expressive power and alleviates embedding ambiguity problem. Based on the function, we then present our Relation-adaptive translating Embedding (RatE) approach to score each graph triple. Moreover, a novel negative sampling method is proposed to utilize both prior knowledge and self-adversarial learning for effective optimization. Experiments verify RatE achieves state-of-the-art performance on four link prediction benchmarks.

## introduction a knowledge graph refers to a collection of interlinked entities, which is usually formatted as a set of triples. a triple is represented as a head entity linked to a tail entity by a relation, which is written as @xcite or (h, r, t). large-scale knowledge graphs, such as freebase @xcite and wordnet @xcite , containing structured information, have been leveraged to support a broad spectrum of natural language processing (nlp) tasks, e.g., question answering @xcite , recommender system @xcite , relation extraction @xcite , etc. nonetheless, the human-curated, real-world knowledge graphs often suffer from incompleteness or sparseness problem @xcite , which inevitably hurts the performance of downstream tasks. hence, how to auto-complete knowledge graphs becomes a popular problem in both research and industry communities. for this purpose, many light-weight graph embedding approaches @xcite @xcite have been proposed. unlike costly graph neural networks (gnns) @xcite , these approaches use low-dimensional embeddings to represent the entities and relations, and capture their relationships via semantic matching or geometric distance. specifically, the approaches with semantic matching, e.g., distmult @xcite and quate @xcite , use a matching function f (h, r, t) that operates on whole triple to directly derive its plausibility score. in contrast, the approaches with geometric distance, e.g., transe @xcite and rotate @xcite , first apply a translation function to head entity and relation for a new embedding in latent space and then denotes the model is equipped with the corresponding property. "sym.", "antisym.", "inv." and "comp." are abbreviations of relation patterns of symmetry, antisymmetry, inversion and composition respectively. for a trans-based graph embedding model, "disambiguation" denotes whether the model explicitly handles embedding ambiguity problem as detailed in §2.5. and, recently, some trans-based graph embedding approaches, e.g., rotate @xcite , go beyond real vector space. they represent the entities and relations in complex vector space, and define the translation function on complex vectors. empowered by the properties of arithmetic operations (e.g., product) in complex space, the translation function can easily capture relation patterns of symmetry (e.g., marriage), antisymmetry (e.g., father), inversion(e.g., hypernym vs. hyponym) and composition (e.g., mother ∧ @xmath0. compared to those defined in real vector space, these approaches improve model's capability in handling a variety of relation patterns and achieve state-of-the-art performance. nevertheless, current trans-based graph embedding approaches with complex embeddings are vulnerable to the following two issues. on the one hand, although approaches solely in complex vector space are equipped with high interpretability for various relation patterns, they are limited by the expressive power of standard product/add of two complex numbers. to improve, quate @xcite introduces quaternion hypercomplex vector space with semantic matching, at the cost of both interpretability and computational overheads, but the improvement is still marginal. on the other hand, embedding ambiguity problem, which means different entities are assigned with similar embeddings, cannot be explicitly handled by existing trans-based approaches (e.g., transe and rotate). it is mainly caused by the propagation of applying a translation function to one-to-many relations for optimizing ∀@xmath1, r). to alleviate both issues above, we propose a novel relation-adaptive translating embedding (rate) approach for knowledge graph completion. as an extension of the trans-based embedding approach ro-tate, our proposed rate inherits the capability to handle various relation patterns, and further presents a light-weight yet effective relation-adaptive translation function. specifically, the function is composed of a novel element-wise weighted product defined in complex vector space, where the weights are learnable, relation-specific and independent to embedding dimension. rather than rigorous complex number product in rotate and quate, rate provides a more flexible way -either the resulting real or imaginary part is a weighted sum of the product on every pair of numbers respectively from the two complex number arguments (i.e., real or imaginary part). hence, rate only requires eight more scalar parameters each relation than baseline rotate, which is much less than the embedding dimension by one or two orders of magnitude. through relation-adaptive translation function, the proposed approach empirically promotes the capacity of modeling translation process and embedding ambiguity problem, while preserves most interpretability to handle various relation patterns. we also propose a novel local-cognitive negative sampling method, by integrating type-constraint training technique @xcite with self-adversarial learning @xcite . the former leverages prior knowledge in graph during training and samples negative head (tail) entities from relation-specific domain (range), which is limited by the hard sampling criterion and suffers from graph sparseness. by comparison, the latter scores a certain number of uniformly-sampled negative samples based on current model, and uses the normalized scores as weights for the loss function. it hence depends heavily on an incompletely-trained model. thus, we integrate them for their mutual benefits: besides using a self-adversarial loss, our method leverages prior knowledge to weaken the effect of current model. our main contributions are summarized in the following. ## proposed approach this section begins with a definition of link prediction task for knowledge graph completion, followed by an introduction to a baseline rotate. then, we propose a novel relation-adaptive translation function to compose the final relation-adaptive translating embedding approach. then, we present an efficient negative sampling method by integrating the merits of two previous sampling strategies. lastly, we demonstrate the capability of our proposed model in alleviating embedding ambiguity problem. ## related work unlike semantic matching graph embedding approaches @xcite @xcite require additional overheads to score a triple, this work is in line with trans-based graph embedding approaches that employ an efficient translation function defined in a latent space. transe @xcite is the most representative trans-based approach, which embeds entities/relations in real vector space and utilizes the relations as translations. it optimizes score function towards "h+@xmath2. several recent trans-based approaches @xcite @xcite ) can be viewed as extensions of transe. more recently, rotate @xcite , as a state-of-the-art trans-based approach, represents the entities/relations in complex vector space and formulates the translating process as a rotation in complex space. also related to this work, many negative sampling methods @xcite are proposed to effectively learn structured knowledge. kbgan @xcite uses knowledge graph embedding model as a negative sample generator to fool the main embedding model (i.e., the discriminator in gans). in contrast, self-adversarial learning @xcite scores a certain number of uniformly-sampled negative samples based on current model, and utilizes the scores to perform a weighted loss function. lastly, this work is also related to using prior knowledge in graphs for training. type-constraint method @xcite , which is based on local closed-world assumptions, corrupts heads (or tails) from relation-specific domain (or range). ## conclusion in this paper, we study a novel trans-based graph embedding approach, called relation-adaptive translating embedding (rate), for knowledge graph completion. it is based on the proposed relation-adaptive translation function with a novel weighted product in complex space, which not only improves represent-ing and modeling capacities but also alleviates embedding ambiguity problem caused by non-injective relations. therefore, rate achieves a better trade-off between interpretability and expressive power than previous trans-based approaches. moreover, a local-cognitive negative sampling method is also presented to seamlessly integrate prior knowledge with self-adversarial learning for effective learning. in experiments, rate achieves state-of-the-art performance on four link prediction benchmarks. extensive ablation study and analyses further provide comprehensive insights into rate.