Improving Knowledge Graph Embedding Using Affine Transformations of Entities Corresponding to Each Relation (2021)

Abstract: To find a suitable embedding for a knowledge graph remains a big challenge nowadays. By using previous knowledge graph embedding methods, every entity in a knowledge graph is usually represented as a k-dimensional vector. As we know, an affine transformation can be expressed in the form of a matrix multiplication followed by a translation vector. In this paper, we firstly utilize a set of affine transformations related to each relation to operate on entity vectors, and then these transformed vectors are used for performing embedding with previous methods. The main advantage of using affine transformations is their good geometry properties with interpretability. Our experimental results demonstrate that the proposed intuitive design with affine transformations provides a statistically significant increase in performance with adding a few extra processing steps or adding a limited number of additional variables. Taking TransE as an example, we employ the scale transformation (the special case of an affine transformation), and only introduce k additional variables for each relation. Surprisingly, it even outperforms RotatE to some extent on various data sets. We also introduce affine transformations into RotatE, Distmult and ComplEx, respectively, and each one outperforms its original method.

## introduction knowledge graphs are usually collections of factual triples-(head entity, relation, tail entity) also known as (subject, predicate, object), which represent human knowledge of the real world in a structured way. there are some outstanding knowledge graphs, such as wordnet @xcite , freebase @xcite , dbpedia @xcite , @xcite . they * corresponding author have gained widespread attention from their successful usage in various applications, e.g., question answering @xcite , natural language processing @xcite , recommendation systems @xcite ,etc. although millions of entities and billions of facts exist in the large-scale knowledge graphs, they still suffer from the incompleteness problem. therefore, knowledge graph completion also known as link prediction which aims to predict missing links among entities based on the known triples has attracted much attention gradually. recently, extensive studies have been done concerning knowledge graph embedding @xcite @xcite . these methods represent entities and relations as low-dimensional vectors (or matrices, tensors, etc.), which not only preserve the semantic information of the knowledge graph, but also represent entities and relations in a fixed structure which is easier for machines' further processing. therefore, apart from the link prediction task, knowledge graph embedding can also be used in various downstream tasks, such as triple classification @xcite , search personalization @xcite ) and so on. the success of existing knowledge graph embedding models heavily relies on their ability to model different types of the relations, such as symmetry/antisymmetry and composition. for example, transe @xcite , which represent relations as translations, can model the composition paterns. distmult @xcite , which forces all relation embeddings to be diagonal matrices in bilinear model, can model the symmetry pattern. however, most models ignore the difference between single-relational and multi-relational triples. multi-relational triples are ubiquitous phenomena in knowledge graphs. for instance, word-net @xcite contains the entity {de-partment_of_justice} with relations {_hypernym, _synset_domain_topic_of, _has_part}. freebase @xcite contains the entity {bryan singer} with relations {/film/director/film, /people/person/profession, /people/person/nationality, /people/person/place_of_birth and so on}. different relations lead entities to different identities or concerns. figure 1 briefly shows that multiple relations may have effects on the optimization of knowledge graph embedding models. we try to do some spatial transformations to make the entities contain the corresponding relation information, and help to distinguish the scenes of different relations. although there exists similar works that project entities with each relation @xcite , they often require complex projection matrices, which lead to a large amount of calculation and are difficult to apply to other models. in addition, other than transe series models, we also apply this transformation method to bilinear models similar to rescal @xcite , and improve their performance obviously. in this paper, we firstly utilize a set of affine transformations related to each relation to operate on entity vectors, and then these transformed vectors are used for performing embedding with previous methods like transe @xcite , ro-tate @xcite , distmult @xcite and complex @xcite . all of these applications are correspondingly simplified based on different model structures. our experimental results demonstrate that the proposed intuitive design with affine transformations provides a statistically significant increase in performance with adding a few extra processing steps or adding a limited number of additional variables. taking transe as an example, we employ the scale transformation (the special case of an affine transformation), and only introduce additional variables for each relation. surprisingly, it even outperforms rotate to some extent on various data sets. the application in other models also shows better results than their original models. especially for distmult and complex, experiments on three benchmark data sets show that the proposed affine-transformation-based algorithms outperform several other state-of-the-art algorithms. notations. throughout this paper, we use lowercase letters , ℎ, , and to represent entities, head entities, relations, and tail entities, respectively. the triplet (ℎ, , ) denotes a fact in knowledge graphs. the corresponding boldface lower-case letters h, r and t denote the embeddings (vectors) of head entities, relations, and tail entities. and are the dimensionality of entity and relation embedding space, respectively (@xmath0. ## related work in this section, we briefly review the related work. roughly speaking, the existing knowledge graph embedding models are mainly divided into three categories: translational models, bilinear models and deep learning models. translational models. transe is the first link prediction model to propose translation distance constraints, which supposes that entities and relations satisfy h + r ≈ t, where h, r, @xmath1, and defines the score function as (h, t) = -h+r-t 1/2 . transh @xcite is proposed to compensate for the shortcomings of transe. they find that transe cannot handle 1-n, n-1, n-n relations well. transh projects entities onto relationspecific hyperplanes with h ⊥ = hw hw and t ⊥ = tw tw , and the score function is defined as (h, t) =h ⊥ + rt ⊥ 2 . moreover, rotate @xcite defines each relation as a rotation from the source entity to the target entity in a complex vector space, which is able to represent various relation patterns including symmetry/asymmetry, inversion and composition. then quate @xcite represents entities and relations with quaternion; @xcite considers the hierarchical of relations, and both of them achieved impressive results. bilinear models. rescal @xcite represents each relation as a full rank matrix and defines a bilinear function as score function (h, t) = h m t . although the embedded relations have a large number of parameters, rescal can still get good results through some of the latest training methods @xcite . subsequently, distmult @xcite forces all relation embeddings m to be diagonal matrices, which can reduce the space of parameters and result in an easier model to be trained. however, distmult assumes that all relations are symmetric, and is not friendly to other types of relations, such as antisymmetry, composition. to solve this problem, complex @xcite extends distmult to complex space: h, r, @xmath2, and uses conjugatetranspose t to model asymmetric relations. deep learning models. mlp (dong et al., ## embedding with affine transformation in this section, we briefly introduce affine transformation at first. then we introduce our proposed method which utilizes affine transformation in transe, rotate, distmult and complex, respectively. ## experiments this section is organized as follows: firstly, we introduce the experimental settings in detail. secondly, we show the effectiveness of our proposed model on three benchmark datasets. finally, we analyze the embeddings generated by transe + at, rotate + at, dismult + at and complex + at, and show the results of ablation studies and visualize some parameters of models. ## conclusion we propose a novel knowledge graph embedding approach which firstly introduces a parametric mapping that projects entity vectors into a new space by an affine transformation corresponding to each relation, and then employs previous embedding methods that map the entities and relations into the embedding space. this algorithm enforces the embedding to be approximately uniformly distributed around the original entity vectors by adjusting the scaling and translation parameters of the affine transformation, which requires considerably less additional computational effort. extensive experimental results show that the affine-transformationbased algorithms outperform the original transe, rotate, distmult and complex, respectively. experiments on three benchmark data sets also show that the proposed affine-transformation-based algorithms outperform several other state-of-the-art algorithms in some quality indexes. we believe that knowledge graph embedding based on affine transformations is very promising and has the potential of being used for many applications. however, more comparison with other embedding methods are needed to fully understand its advantages and disadvantages.