S peed E : E uclidean Geometric Knowledge Graph Embedding Strikes Back (2024)

Abstract: Geometric knowledge graph embedding models (gKGEs) have shown great potential for knowledge graph completion (KGC), i.e., automatically predicting missing triples. However, contemporary gKGEs require high embedding dimensionalities or complex embedding spaces for good KGC performance, drastically limiting their space and time efficiency. Facing these challenges, we propose SpeedE, a lightweight Euclidean gKGE that (1) provides strong inference capabilities, (2) is competitive with state-of-the-art gKGEs, even significantly outperforming them on YAGO3-10 and WN18RR, and (3) dramatically increases their efficiency, in particular, needing solely a fifth of the training time and a fourth of the parameters of the state-of-the-art ExpressivE model on WN18RR to reach the same KGC performance.

## introduction geometric knowledge graph embedding models (gkges) represent entities and relations of a knowledge graph (kg) as geometric shapes in the semantic vector space. gkges achieved promising performance on knowledge graph completion (kgc) and knowledge-driven applications @xcite ; while allowing for an intuitive geometric interpretation of their captured patterns @xcite . efficiency problem. recently, increasingly more complex embedding spaces were explored to boost the kgc performance of gkges @xcite @xcite . however, more complex embedding spaces typically require more costly operations or more parameters, lowering their time and space efficiency compared to euclidean gkges @xcite . even more, most gkges require high-dimensional embeddings to reach good kgc performance, increasing their time and space requirements @xcite . thus, the need for (1) complex embedding spaces and (2) high-dimensional embeddings lowers the efficiency of gkges, hindering their application in resource-constrained environments, especially in mobile smart devices @xcite @xcite . challenge and methodology. although there has been much work on scalable gkges, any such work has focused exclusively on either reducing the embedding dimensionality @xcite @xcite or using simpler embedding spaces @xcite @xcite , thus addressing only one side of the efficiency problem. facing these challenges, this work aims to design a euclidean gkge that performs well on kgc under low-dimensional conditions, reducing its storage space, inference, and training times. to reach this goal, we analyze expressive @xcite , a euclidean gkge that has shown promising performance on kgc under highdimensional conditions. ## knowledge graph completion this section discusses the kgc problem and its empirical evaluation @xcite . first, we introduce the triple vocabulary, consisting of a finite set of relations r and entities e. we use this vocabulary to define triples, i.e., expressions of the form r j (e h , e t ), where r @xmath0, e h , e @xmath1, and where we call e h the triple's head and e t its tail. a finite set of triples over the triple vocabulary is called a knowledge graph g. kgc describes the problem of predicting missing triples of g. ## related work the main focus of our work lies on gkges, i.e., knowledge graph embedding models (kges) that allow for a geometric interpretation of their captured inference patterns. thus, we have excluded neural kges as they are typically less interpretable @xcite @xcite . gkges are commonly classified by how they embed relations: bilinear gkges embed relations as matrices, allowing them to factorize a graph's adjacency matrix by computing the bilinear product of entity and relation embeddings. the pioneering bilinear model is rescal @xcite , embedding relations with full-rank d × d matrices and entities with d-dimensional vectors. however, its parameter size grows quadratically with its dimensionality d, lim-iting rescal's scalability @xcite . thus, more scalable bilinear gkges were proposed, such as distmult @xcite , complex @xcite , tucker @xcite , simple (kazemi and poole, 2018), quate @xcite , and dualquate @xcite . although these enhanced models could capture increasingly more inference patterns, none of them can capture composition patterns. spatial gkges embed relations as semantic regions within the embedding space. boxe @xcite is the pioneering spatial gkge, embedding relations with two bounded hyper-rectangles. this allows boxe to capture most of the core inference patterns. however, purely spatial models cannot capture composition patterns. functional gkges embed relations as functions f r i : k @xmath2. the pioneering functional model is transe @xcite , which embeds relations as translations from head to tail entity embeddings. however, representing relations as translations limits transe from capturing inference patterns, such as symmetry or hierarchy. thus, many extensions were proposed, solving some of these limitations, such as rotate @xcite , murp @xcite , roth @xcite , @xcite and cone @xcite . however, none of these models can capture hierarchy patterns. spatio-functional gkges. recently, @xcite proposed expressive, a spatio-functional gkge that combines the advantages of both spatial and functional models by embedding relations as hyper-parallelograms. thereby, it can capture all core inference patterns simultaneously. ## the expressive model this section reviews expressive @xcite , a euclidean gkge with strong kgc performance under high dimensionalities. ## experiments this section empirically evaluates speede: section 6.1 describes the experimental setup. section 6.2 studies speede's kgc performance, finding that it is competitive with sota gkges on fb15k-237 and even significantly outperforms them on yago3-10 and wn18rr. section 6.3 studies speede's space and time efficiency, finding that on wn18rr, speede needs a quarter of expressive's parameters to reach the same kgc performance while training five times faster than it. ## conclusion although there has been much work on resourceefficient gkges, any such work has focused exclusively on reducing the embedding dimensionality @xcite @xcite or using simpler embedding spaces @xcite @xcite , thus addressing only one side of the efficiency problem. in this work, we address the embedding space and dimensionality side jointly by introducing speede, a lightweight gkge that (1) provides strong inference capabilities, (2) is competitive with sota gkges, even significantly outperforming them on yago3-10 and wn18rr, and (3) dramatically increases the efficiency of current gkges, needing solely a fifth of the training time and a fourth of the number of parameters of the sota expres-sive model on wn18rr to reach the same kgc performance.