Knowledge Graph Embeddings in Geometric Algebras (2020)

Abstract: Knowledge graph (KG) embedding aims at embedding entities and relations in a KG into a low dimensional latent representation space. Existing KG embedding approaches model entities and relations in a KG by utilizing real-valued , complex-valued, or hypercomplex-valued (Quaternion or Octonion) representations, all of which are subsumed into a geometric algebra. In this work, we introduce a novel geometric algebra-based KG embedding framework, GeomE, which utilizes multivector representations and the geometric product to model entities and relations. Our framework subsumes several state-of-the-art KG embedding approaches and is advantageous with its ability of modeling various key relation patterns, including (anti-)symmetry, inversion and composition, rich expressiveness with higher degree of freedom as well as good generalization capacity. Experimental results on multiple benchmark knowledge graphs show that the proposed approach outperforms existing state-of-the-art models for link prediction.

## introduction knowledge graphs (kgs) are directed graphs where nodes represent entities and (labeled) edges represent the types of relationships among entities. this can be represented as a collection of triples (h, r, t), each representing a relation r between a "head-entity" h and an "tail-entity" t. some real-world knowledge graphs include freebase @xcite , wordnet @xcite , @xcite , and dbpedia @xcite . however, most existing kgs are incomplete. the task of link prediction alleviates this drawback by inferring missing facts based on the known facts in a kg and thus has gained growing interest. embedding kgs into a low-dimensional space and learning latent representations of entities and relations in kgs is an effective solution for this task. in general, most existing kg embedding models learn to embed kgs by optimizing a scoring function which assigns higher scores to true facts than invalid ones. recently, learning kg embeddings in the complex or hypercomplex spaces has been proven to be a highly effective inductive bias. complex @xcite , rotate, protate @xcite , and quate @xcite achieved the state-of-the-art results on link prediction, due to their abilities of capturing various relations (i.e., modeling symmetry and anti-symmetry). they both use the asymmetrical hermitian product to score relational triples where the components of entity/relation embeddings are complex numbers or quaternions. complex numbers and quaternions can be described by the various components within a clifford multivector @xcite . in other words, the geometric algebra of clifford (1882) provides an elegant and efficient rotation representation in terms of multivector which is more general than hamilton (1844)'s unit quaternion. in this paper, we propose a novel kg embedding approach, geome, which is based on clifford multivectors and the geometric product. concretely, we utilize n multivector embeddings of n grades (@xmath0, 3) to represent entity and relation. each component of an entity/relation embedding is a multivector in a geometric algebra of n grades, g n , with scalars, vectors and bivectors, as well as trivectors (for @xmath1. in terms of a triple (h, r, t), we use an asymmetrical geometric product which involves the conjugation of the embedding of the tail entity to multiply the embeddings of e s , r, e o , and obtain the final score of the triple from the product embedding. the advantages of our formulas include the following points: experimental results demonstrate that our approach achieves state-of-the-art results on four wellknown kg benchmarks, i.e., wn18, fb15k, wn18rr, and fb15k-237. ## related work most kg embedding models can be classified as distance-based or semantic matching based, according to their scoring functions. distance-based scoring functions aim to learn embeddings by representing relations as translations from head entities to tail entities. @xcite proposed transe by assuming that the added embedding of s and r should be close to the embedding of o. since that, many variants and extensions of transe have been proposed. for example, transh @xcite projects entities and relations into a hyperplane. transr @xcite introduces separate projection spaces for entities and relations. transd @xcite uses independent projection vectors for each entity and relation and can reduce the amount of calculation compared to transr. toruse @xcite defines embeddings and distance function in a compact lie group, torus. the recent distance-based kg embedding models, rotate and protate @xcite , propose a rotation-based distance scoring functions with complexvalued embeddings. likewise, transcomplex @xcite also maps entities and relations into a complex-valued vector space. on the other hand, semantic matching models include rescal @xcite , distmult @xcite , complex @xcite , simple (kazemi and poole, 2018) and quate @xcite . in rescal, each relation is represented with a square matrix, while distmult replaces it with a diagonal matrix in order to reduce the complexity. simple is also a simple yet effective bilinear approach for knowledge graph embedding. complex embeds entities and relations in a complex space and utilizes an asymmetric hermitian product to score triples, which is immensely helpful in modeling various relation patterns. quate extends complex in a hypercomplex space and replaces the hermitian product with the hamilton product which provides a greater extent of expressiveness. in addition, neural network based kg embedding models have also been proposed, e.g., ntn @xcite , conve @xcite , @xcite and iteracte @xcite . our proposed approach, geome, subsumes complex, protate and quate in the geometric algebras. in addition to the inheritance of the attractive properties of these existing kg embedding models, our approach takes advantages of the multivectors, e.g., the rich geometric meanings, the excellent representation ability and the generalization ability in the geometric algebras of different grades. due to the above merits of the geometric algebras and multivectors, they have also been widely applied in computer vision and neurocomputing @xcite . ## geometric algebra and multivectors leaning on the earlier concepts of grassmann (1844)'s exterior algebra and hamilton (1844)'s quaternions, clifford (1882) intended his geometric algebra to describe the geometric properties of scalars, vectors and eventually higher dimensional objects. in addition to the well known scalar and vector elements, there are bivectors, trivectors, n-vectors and multivectors which are higher dimensional generalisations of vectors. an n -dimensional vector space r n can be embedded in a geometric algebra of n grades , g n . in this section, we take g 2 and g 3 as examples to introduce multivectors and some corresponding operators. hence an arbitrary 3-grade multivector @xmath2, where a 0 , a 1 , a 2 , a 3 , a 12 , a 23 , a 13 , a 123 are all real numbers. each element of a multivector, e.g., a scalar, a vector, or an n-vector, is called as a blade. a 2-grade multivector @xmath3, two vectors and one bivector. ## conclusion we propose a new gemetric algebra-based approach for kg embedding, geome, which utilizes multivector representations to model entities and relations in a kg with the geometric product. our approach subsumes several state-of-the-art kg embedding models, and takes advantages of the flexibility and representational power of geometric algebras to enhance its generalization capacity, enrich its expressiveness with higher degree of freedom and enable its ability of modeling various relation patterns. experimental results show that our approach achieves the state-of-the-art results on four well-known benchmarks. let assume that the matrix m at(m r i ) is householder. therefore, it has two eigenvalues {-1, 1}. the eigenvectors corresponding to -1 are orthogonal to the eigenvalues corresponding to 1. there are two conditions the abovementioned equation equals to zero if for either vec(m h i ) or vec(m t i ), the elements (coefficents of vector parts in this case) corresponding to the negative sign (-1) of 1 will be zero. the abovementioned conditions hold for each multivector element m i of the k-dimensional embeddings m, i.e. @xmath4, . . . , k. therefore, there are 2 k possible options (capacity of model) to have φ(h, r, t)φ(t, r, h) = 0 (modeling symmetric pattern). inversion given two relations r 1 , r 2 which form inverse pattern i.e. r 1 = r -1 2 (e.g. r 1 =sonof, r 2 =fatherof ), we show that geome models inverse pattern by φ(h, r 1 , t) -φ(t, r 2 , h) = 0. this is proved as follows let assume that the matrices of m at(m r 1i ) and m at(m r 2i ) have same eigenvalues λ 1i = λ 2i . therefore, we have since for n-grade multivector, there are 2 n variables in the vector, the corrsponding matrices m at(m r 1i ), m at(m r 2i ) are 2 n × 2 n dimensional. since a 2 n × 2 n matrix has at most 2 n distinct