BiQUE : B iquaternionic Embeddings of Knowledge Graphs (2021)

Abstract: Knowledge graph embeddings (KGEs) compactly encode multi-relational knowledge graphs (KGs). Existing KGE models rely on geometric operations to model relational patterns. Euclidean (circular) rotation is useful for modeling patterns such as symmetry, but cannot represent hierarchical semantics. In contrast, hyperbolic models are effective at modeling hierarchical relations, but do not perform as well on patterns on which circular rotation excels. It is crucial for KGE models to unify multiple geometric transformations so as to fully cover the multifarious relations in KGs. To do so, we propose BiQUE, a novel model that employs biquaternions to integrate multiple geometric transformations, viz., scaling, translation, Euclidean rotation, and hyperbolic rotation. BiQUE makes the best trade-offs among geometric operators during training, picking the best one (or their best combination) for each relation. Experiments on five datasets show BiQUE’s effectiveness.

## introduction knowledge graphs (kgs) provide an efficient way to represent real-world entities and their intricate connections in the form of @xcite triples. each head/tail entity corresponds to a node in a kg, and each relation represents a directed edge between them. imbued with rich factual knowledge, kgs have demonstrated their effectiveness in a wide range of downstream applications @xcite . although problems due to incompleteness and noise continue to plague kgs, those issues have been ameliorated by knowledge graph embeddings (kges) that project entities and relations into lowdimensional dense vectors. current kge methods mainly focus on exploiting geometric transformations and embedding spaces to model relational patterns such as (anti)symmetry, inversion, and composition. transe @xcite represents each relation as a translation from a head entity to a tail entity. with translations alone, transe cannot model many relation types such as symmetric ones. in contrast, rotate @xcite represents each relation as a rotation in complex space. it proves that it can model (anti)symmetry, inversion, and composition patterns. quate @xcite extends rotate's complex number representation to a hypercomplex number representation. a drawback of rotation-based kge models is that their representations are entrenched in (euclidean) circular rotation, and hence they are unable to model hierarchical and tree-like structures (e.g. hypernym and part_of ). such hierarchical relations are common and even pervasive in some kgs. since, in circular rotation, all rotating points are constrained to be at the same distance from the center of a circle, it is hard to model relations whose semantics require that entities move at different distances from the nexus. to overcome this shortcoming, recent models project kgs into hyperbolic space @xcite . the hyperbolic models inevitably lose the basic properties of euclideanspace transformations, and thus cannot avail themselves of these useful operations. moreover, it is difficult to seamlessly integrate the hyperbolic models with extant non-hyperbolic models to create more powerful hybrids because the models' different geometric representations do not cohere. what we want is the best of best worlds, i.e., a model capable of effecting both circular rotations and hyperbolic transformations, in a coherent geometric representation. this allows the model to choose the best representation for each relation, e.g., circular rotations for symmetric/inversion relations and hyperbolic rotations for hierarchical patterns. in addition, for relations that exhibit both circular and hyperbolic characteristics (e.g., the in-teract_with relation in figure 1 ), the model would rely on the data to choose the sweet spot balancing both transformations. for relations that are best captured by the composition of circular and hyperbolic rotations (e.g., affects in figure 1 ), the model would learn the best composition of both representations jointly. lastly, by subsuming circular rotations, the model would inherit the representational prowess of circular-rotation models. in this paper, we propose precisely such a model named bique. bique employs a powerful algebraic system called biquaternions @xcite to represent kgs. most common number systems used by current kge methods (including real numbers, complex numbers, and real quaternions) are subsumed and systematically unified by biquaternions. further, the hamilton product of biquaternions, at the core of bique, imbues it with a strong geometric interpretation that combines both circular rotations and hyperbolic rotations. in sum, our contributions are as follows. ## related work we briefly survey kge methods that are most relevant to our approach. euclidean models. these models represent entities and relations by real vectors, and can be categorized into translation-based models @xcite @xcite , semantic-matching models @xcite , and neural models @xcite . euclidean models typically cannot represent all relation types in a kg, e.g., transe cannot model symmetric patterns, and distmult cannot model antisymmetric/inversion patterns @xcite . euclidean models typically need large embedding dimensions for good empirical performance. complex-valued models. complex @xcite uses complex-valued tensor factorization and complex conjugation to model antisymmetric relations. rotate @xcite models each relation as a 2d circular rotation in a complex vector space. it can model several relation types, but falls short when modeling hierarchical patterns. quate (zhang et al., 2019) extends ro-tate's complex representation to a hypercomplex one via quaternion embeddings, and represents rotations in a four-dimensional real-number space. however, it shares the weakness of circular-rotation methods in being deficient when modeling hierarchical semantics. duale (cao et al., 2021) incorporates dual numbers into quaternions, and thereby unifies circular rotation with translation. this geometric combination of duale differs from that of our bique model (section 4), which unifies circular rotation with hyperbolic rotation. another difference is that bique models translations via a different mechanism in the form of a relationspecific biquaternion. ## background biquaternions, endowed with rich algebraic properties, have been widely used in quantum mechanics, general relativity, and signal processing @xcite , but have yet to make inroads into knowledge graph embeddings. a biquaternion is defined on a four-dimensional vector space over the field c of complex numbers. we denote a complex number @xmath0, c @xmath1, and i is the usual imaginary unit (i 2 = -1). definition 1. the basic algebraic forms of a biquaternion q are we denote the scalar and vector parts of q respectively as s(q) = w and v(q) = xi+yj+zk. a pure biquaternion q is one with s(q) = 0. a quaternion (hamilton, 1844) is a restricted biquaternion, in which w, x, y, @xmath2.g., q r and q i in equation 2 are quaternions). complex numbers and real numbers are both special cases of biquaternions. the biquaternion @xmath3: (a) as the vector v(q) = [w, x, y, z] t ; (b) as ||q||(cos θ+u sin θ) (where θ = cos -1 (w/||q||), θ ∈ c, @xmath4, and ||q|| = w 2 +x 2 +y 2 +z 2 ); and (c) as the matrix next we present basic operations on a biquaternion @xmath5. the conjugate of q is denoted as @xmath6. the complex conjugate of q is denoted as q * = w * +x * i+y * j+z * k (c * = c r -c i i is the standard complex conjugate of a complex number @xmath7. two biquaternions q 1 = w 1 +x 1 i+y 1 j+z 1 k and q 2 = w 2 +x 2 i+y 2 j+z 2 k are added and subtracted (in the obvious manner) as q 1 ±q 2 = (w 1 ±w 2 )+(x 1 ±x 2 )i+(y 1 ±y 2 )j+(z 1 ±z 2 )k. the multiplication q 1 q 2 between q 1 and q 2 can be obtained via standard algebraic distributivity (while obeying the properties in equations 3, and following the normal multiplication rule of complex numbers for the products between coefficients) as equation 5 is termed the hamilton product between q 1 amd q 2 . alternatively, the multiplication can be equivalently represented as a matrix-vector product equations 6 and 7 can be easily verified by substituting in equations 4 and 5, and using normal matrix multiplication. the set of biquaternions and the set of quaternions are both closed under multiplication, and multiplication is associative but not commutative. also note that q 1 q 2 = q 2 q 1 . by using equation 4, we can easily verify that m(q) = m(q) t and m(q * ) = m(q) * (where m(•) * refers to the complex conjugation of each element in the matrix). the norm of q is given by ||q|| = √ q @xmath8: ||q|| = ||q||). a unit biquaternion is one with unit norm, i.e., ||q|| = 1. ## experiments to validate bique's effectiveness, we conduct extensive experiments on the knowledge graph completion (kgc) task. we use three standard knowledge graph datasets, viz., wn18rr, fb15k-237, and yago3-10. in addition, to demonstrate bique's scalability, we run it on two huge commonsense knowledge graph datasets, viz., con-cept100k and atomic. our codes and datasets are publicly available at https://github.com/ guojiapub/bique . ## conclusion in this paper, we propose bique, a novel model that uses biquaternionic algebra for kges, and combines multiple geometric transformations in a coherent representation. our experimental results and detailed empirical analysis demonstrate the effectiveness, scalability, and advantages of our model. as future work, we will extend bique to work on knowledge hypergraphs.