Enhancing Hyperbolic Knowledge Graph Embeddings via Lorentz Transformations (2024)

Abstract: Knowledge Graph Embedding (KGE) is a powerful technique for predicting missing links in Knowledge Graphs (KGs) by learning the entities and relations. Hyperbolic space has emerged as a promising embedding space for KGs due to its ability to represent hierarchical data. Nevertheless, most existing hyperbolic KGE methods rely on tangent approximation and are not fully hyperbolic, resulting in distortions and inaccuracies. To overcome this limitation, we propose LorentzKG, a fully hyperbolic KGE method that represents entities as points in the Lorentz model and represents relations as the intrinsic transformation—the Lorentz transformations between entities. We demonstrate that the Lorentz transformation, which can be decomposed into Lorentz rotation/reflection and Lorentz boost, captures various types of relations including hierarchical structures. Experimental results show that our LorentzKG achieves state-of-the-art performance.

## introduction knowledge graphs (kgs) @xcite , which comprise factual triples of the form (head entity, relation, tail entity), are essential in questionanswering @xcite , information extraction @xcite , and recommendations @xcite . recent advancements related to kgs centered around knowledge graph embedding (kge), which involves mapping entities and relations to some dedicated representation spaces, providing an effective tool for interpreting semantic meaning within kgs. while many existing models utilize euclidean space for modeling relation patterns in kgs @xcite @xcite , they face challenges in effectively modeling semantic hierarchies due to the limited expressive power of low-dimensional euclidean space. increasing the dimensions incurs high computational costs, especially given the large number of entities in kgs. recent work has explored hyperbolic space, a non-euclidean space with constant negative curvature, as a more effective and efficient embedding space for data with hierarchical structures @xcite @xcite @xcite . several hyperbolic kge models have been proposed to represent kgs @xcite @xcite @xcite , demonstrating the advantages of hyperbolic space, particularly in the low-dimensional settings. most existing hyperbolic kge models @xcite @xcite rely on tangent space approximations to perform vector space operations, such as matrix-vector multiplication and vector addition. tangent space is a vector space that approximates the manifold at a given base point. however, this approach is not fully hyperbolic since it involves lifting the data from hyperbolic space to tangent space for feature transformation and subsequently projecting it back to hyperbolic space. this projection introduces additional complexities and numerical instability, as it requires the calculation of a series of hyperbolic functions. moreover, the tangent approximation of data that is located away from the base point is distorted and inaccurate, compromising the expressibility of hyperbolic space @xcite . to address above issues, it is essential to explore fully hyperbolic approaches that bypass the reliance on tangent space approximations and instead work directly within hyperbolic space. in this paper, we address this challenge by utilizing the lorentz model @xcite of hyperbolic space, and its isometry group, the positive lorentz group @xcite , to develop a fully hyperbolic kge model, called lorentzkg. the property of the positive lorentz group acting transitively on the lorentz model enables the definition of matrix multiplication on the hyperbolic space as a linear operation-multiplying a group element of the lorentz group, the lorentz transformation, to a point on the lorentz model. this linear approach avoids tangent approximation, and furthermore, benefits from the numerical stability exhibited by the lorentz model @xcite , ensuring reliable and accurate calculations. our lorentzkg model follows a translationbased kge methodology, where relation-specific transformations using lorentz transformation are applied to the head and tail entities embedded in the lorentz model. the plausibility of a given fact is measured by the hyperbolic distance between the transformed entities. the highlights of our lorentzkg are: (i) the transitive nature of lorentz transformation on the lorentz model ensures that the transformation is an intrinsic linear operation and guarantees that the transformed entity embedding remains a point in the lorentz model; (ii) the nature of the lorentz transformation, allowing for decomposition into various types of transformations, enables us to effectively capture diverse relational patterns observed in kgs, including symmetry/anti-symmetry, inversion, composition, and hierarchy (sec 3.3); (iii) different relational transformations on head and tail entities enable modeling of relation cardinalities (such as 1to-1, 1-to-n, n-to-1, and n-to-n @xcite ) (sec 3.3); (iv) experimental results demonstrate the superiority of our lorentzkg compared to several euclidean and hyperbolic kge models. in addition, we point out connections with prior translation-based euclidean kge models (sec 3.4) and contrasts our lorentzkg with prevalent hyperbolic kge models (sec 3.2), underscoring our work's novelty. ## preliminaries hyperbolic space is a riemannian manifold with constant negative curvature. there are five isometric models of hyperbolic space @xcite , including the lorentz model l and the poincaré ball model b (see appendix b). we select the lorentz model as the representation space in lorentzkg due to its nature as a homogeneous space and its intrinsic characteristic of the positive lorentz group being its isometry group. in this section, we provide a brief overview of these concepts and important geometric notions in hyperbolic space that are relevant to our work. ## methodology here we present our proposed lorentzkg which uses the lorentz model for entity embeddings and lorentz transformation for relation embeddings in kgs. the score function measures triple plausibility using the squared lorentzian distance between transformed entity embeddings, along with a bias term. we also discuss lorentzkg's capability to model kg relations, highlighting its potential in capturing diverse relational patterns. ## experiments dataset. we evaluate our proposed model on two widely used knowledge graph datasets -wn18rr @xcite and fb15k-237 @xcite . some statistics of these two knowledge graph datasets are summarized in table . 1. ξ g is the global graph curvature that measures the level of hierarchy in graphs. the lower ξ g indicates the more tree-like the kg is @xcite . more experiments are included in sec g. ## conclusions here we introduce lorentzkg, a fully translationbased hyperbolic kge model that utilizes lorentz transformations. lorentzkg enables linear translation of entity embeddings and facilitates the modeling of various relation patterns in kgs. by applying different transformations to the head and tail entities, lorentzkg is able to capture relation cardinalities. experimental results show the superiority of lorentzkg over existing models, highlighting its enhanced expressivity and modeling capabilities, particularly in low-dimensional settings.