3 D Rotation and Translation for Hyperbolic Knowledge Graph Embedding (2024)

Abstract: The main objective of Knowledge Graph (KG) embeddings is to learn low-dimensional representations of entities and relations, enabling the prediction of missing facts. A significant challenge in achieving better KG embeddings lies in capturing relation patterns, including symmetry, antisymmetry, inversion, commutative composition, non-commutative composition, hierarchy, and multiplicity. This study introduces a novel model called 3H-TH (3D Rotation and Translation in Hyperbolic space) that captures these relation patterns simultaneously. In contrast, previous attempts have not achieved satisfactory performance across all the mentioned properties at the same time. The experimental results demonstrate that the new model outperforms existing state-of-the-art models in terms of accuracy, hierarchy property, and other relation patterns in low-dimensional space, meanwhile performing similarly in high-dimensional space.

## introduction the components of a knowledge graph are collections of factual triples, where each triple (h, r, t) denotes a relation r between a head entity h and a tail entity t; toy examples are shown in fig. 1 . @xcite , yago @xcite , and wordnet @xcite are some examples of knowledge graphs used in the real world. meanwhile, applications such as questionanswering @xcite , information retrieval @xcite , recommender systems @xcite , and natural language processing @xcite ) may find significant value for knowledge graphs. therefore, knowledge graph research is receiving increasing attention in both the academic and business domains. our code is available at https://github.com/ yihuazhu111/3h-th . predicting missing links is a crucial aspect of knowledge graphs, given their typical incompleteness. in recent years, significant research efforts have focused on addressing this challenge through the utilization of knowledge graph embedding (kge) techniques, which involve learning lowdimensional representations of entities and relations @xcite . kge approaches have demonstrated scalability and efficiency in modeling and inferring knowledge graph entities and relations based on available facts. a major issue in kge research concerned several relation patterns, including symmetry, antisymmetry, inversion, composition (i.e., commutative and non-commutative composition), hierarchy, and multiplicity (see @xcite ). in fact, several current approaches have attempted to model one or more of the above relation patterns @xcite @xcite . the transe @xcite which models the antisymmetry, inversion, and composition patterns, represents relations as translations. the rotate @xcite represents the relation as a rotation and aims to model symmetry, antisymmetry, inversion, and composition. for some difficult patterns (see fig. 1 ), including noncommutative composition, hierarchy, and multiplicity, the atth @xcite embeds relation in hyperbolic space to enable relations to acquire hierarchy property. the duale @xcite attempts to combine translation and rotation operations to model multiple relations. such approaches, however, have failed to perform well on all the above relation patterns simultaneously as shown in here we present how our proposed method (3h-th) works for the difficult relation pattern examples in fig. 1 . by embedding the entities and relations in hyperbolic space, we can allow the kg model to acquire hierarchy properties so that we can more clearly distinguish between the different hierarchies of entities, for example, movie director, name, and actor. besides, to solve noncommutative problems, for example (see fig. 1 ), if the mother of a's father (b) is c while the father of a's mother (d) is e, then c and e are equal if the relations were commutative, we use the quaternion geometry property (non-commutative) to enable the model to obtain a non-commutative composition pattern. finally, we try to combine rotation and translation operations to obtain multiplicity properties, e.g. different relations exist between the same entities (e.g., award-winner, director). moreover, our study provides some important insights into developing several comparable methods to explore the impact of a combination of translation and rotation in euclidean or hyperbolic space, as well as both simultaneously. we evalu-ate the new model on three kge datasets including wn18rr @xcite , fb15k-237 @xcite , and fb15k @xcite . experimental results show that the new model outperforms existing state-of-theart models in terms of accuracy, hierarchy property, and other relation patterns in low-dimensional space, meanwhile performing similarly in highdimensional space, which indicates that the new model 3h-th can simultaneously model symmetry, antisymmetry, inversion, composition, hierarchy, and multiplicity relation patterns. ## related work knowledge graph embedding has received a lot of attention from researchers in recent years. one of the main kge directions has been led by translation-based and rotation-based approaches. another key area is hyperbolic kge, which enables models to acquire hierarchy property. in particular, our approach advances in both directions and acquires both advantages. translation-based approach. one of the widely adopted methods in kge is the translation-based approach, exemplified by transe @xcite , which represents relation vectors as translations in the vector space. in this approach, the relationship between the head and tail entities is approximated by adding the relation vector to the head entity vector, resulting in a representation that is expected to be close to the tail entity vector. after transe, there has been an increasing amount of literature on its extension. transh @xcite represents a relation as a hyperplane to help the model perform better on complex relations. by embedding entities and relations in separate spaces with a shared projection matrix, transr @xcite further creates a relation-specific space to obtain a more expressive model for different types of entities and relations. compared to transr, transd @xcite employs independent projection 2 vectors for each object and relation, which can reduce the amount of computation. although these methods are relatively simple and have only a few parameters, they do not effectively express crucial relation patterns such as symmetry, hierarchy, and multiplicity relations ( rotation-based approach. rotate @xcite introduced a new direction as rotationbased methods, which represents the relation vectors as rotation in complex vector space and can model various relation patterns, including symmetry, antisymmetry, inversion, and composition. quate @xcite substitutes 2d rotation with quaternion operation (3d rotation) in quaternion space, aiming to obtain a more expressive model than rotate. furthermore, the incorporation of 3d rotation enables the model to capture the non-commutative composition of relations, leveraging the geometric properties of quaternions (wherein two 3d rotations are known to be noncommutative). however, these rotation operations cannot solve hierarchy and multiplicity ( hyperbolic kge. one of the major challenges for kge is the hierarchy problem. hyperbolic geometry has been shown to provide an efficient approach to representing kg entities and relations in low-dimensional space while maintaining latent hierarchy properties. murp @xcite optimizes the hyperbolic distance between the projected head entity and the translational tail entity to achieve comparable results by using fewer dimensions than the previous methods. roth @xcite tries to substitute translation operations with rotation operations to obtain more relation patterns properties like rotate. however, there is still room for improvement in handling other relation patterns, particularly in terms of multiplicity and non-commutative composition properties. bique @xcite utilizes biquaternions, which encompass both circular rotations in euclidean space and hyperbolic rotations, aim to acquire hierarchy properties and rotate-based relation patterns, while this approach struggles to effectively capture the multiplicity property. our proposed model 3h-th leverages translation, 3d rotation, and hyperbolic embedding to offer a comprehensive and expressive representation of entities and relations, encompassing various relation patterns ( ## problem formulation and background we describe the kge problem and present some related methods before our approach part. ## our approach our proposed model aims to enhance the representation of entities and relations by incorporating various relation patterns, with a particular focus on non-commutative composition, multiplicity, and hierarchy. to achieve this, we leverage techniques such as translation, 3d rotation, and hyperbolic embedding, allowing for a more expressive and comprehensive representation. ## experiments we expect that the composite model 3h-th, which performs both 3d rotation and translation in hyper- ## conclusion in this study, we propose the 3h-th model for kge to address multiple relation patterns, including symmetry, antisymmetry, inversion, commutative composition, non-commutative composition, hierarchy, and multiplicity. by combining 3d rotation and translation in hyperbolic space, the model effectively represents entities and relations. experimental results demonstrate that the 3h-th model achieves excellent performance in low-dimensional space. moreover, the performance difference becomes smaller in high-dimensional space, although the model still performs well.