T e R o: A Time-aware Knowledge Graph Embedding via Temporal Rotation (2020)

Abstract: In the last few years, there has been a surge of interest in learning representations of entities and relations in knowledge graph (KG). However, the recent availability of temporal knowledge graphs (TKGs) that contain time information for each fact created the need for reasoning over time in such TKGs. In this regard, we present a new approach of TKG embedding, TeRo, which defines the temporal evolution of entity embedding as a rotation from the initial time to the current time in the complex vector space. Specially, for facts involving time intervals, each relation is represented as a pair of dual complex embeddings to handle the beginning and the end of the relation, respectively. We show our proposed model overcomes the limitations of the existing KG embedding models and TKG embedding models and has the ability of learning and inferring various relation patterns over time. Experimental results on three different TKGs show that TeRo significantly outperforms existing state-of-the-art models for link prediction. In addition, we analyze the effect of time granularity on link prediction over TKGs, which as far as we know has not been investigated in previous literature.

## introduction in recent years, a number of sizable knowledge graphs (kgs) have been constructed, including dbpedia @xcite , yago @xcite , nell @xcite and freebase @xcite . in these kgs, a fact is represented as a triple (s, r ,o), where s (subject) and o (object) are entities (nodes), and r (relation) is the relation (edge) between them. several kg embedding (kge) models are developed to perform learning and inference over these kgs @xcite @xcite @xcite . the most common learning task for these models is link prediction, which is to complete a fact with the missing entity. for instance, one can use a kge model to perform an object query like @xcite . in this case, there are several valid answers to this question, regardless of the time factor. obviously, the inclusion of time information can make this query more specific, e.g., @xcite . some temporal kgs (tkgs) including icews @xcite , gdelt @xcite , yago3 @xcite and wikidata @xcite store billions of time-aware facts as quadruples @xcite where t is the time annotation, e.g., (barack obama, visits, ukraine, 2014-07-08). the availability of these tkgs that exhibits complex temporal dynamics in addition to its multi-relational nature has created the need for approaches that can characterize and reason over them. traditional kge models disregard time information, leading to an ineffectiveness of performing link prediction on tkgs involving temporary relations (e.g., visits, live in, etc.). to tackle this problem, tkg embedding (tkge) models encode time information in their embeddings. such tkge models @xcite @xcite @xcite were shown to have better performances on link prediction over tkgs than traditional kge models. however, most of the existing tkge models are the extensions of transe @xcite and distmult @xcite , and thus are not fully expressive for some relation patterns @xcite . in this paper, we propose a novel approach for tkges, tero, which defines the temporal evolution of an entity embedding as a rotation from the initial time to the current time in the complex vector space. we show the limitation of the existing tkge models and the advantage of our proposed model on learning various relation patterns over time. specially, for facts involving time intervals, each relation is represented as a pair of dual complex embeddings which are used to handle the beginning and the end of the relation, respectively. in this way, tero can adapt well to datasets where time annotations are represented in the various forms: time points, beginning or end time, time intervals. most of previous tkge-related works as far as we know use specific time granularities for various tkgs. for example, the time granularity of the icews datasets is fixed as 24 hours in @xcite . in this work, we adopt various time-division approaches for different tkg datasets and investigate the effect of the length of the time steps on the performance of our model. to verify our approach, we compare the performance of our proposed models on link prediction and time prediction tasks over four different tkgs with the state-of-the-art kge models and the existing tkge models. the experimental results demonstrate that our proposed model outperforms other baseline models significantly by inferring various relation patterns and encoding time information. distance-based models measure the plausibility of a fact as the distance between the two entities, usually after a translation or rotation carried out by the relation. a typical example of distance-based models is transe @xcite . transe exhibits deficiencies when learning 1-n relations. thus, various extensions of transe @xcite @xcite , were proposed to tackle this problem. they use different mapping methods to project entities from entity space to relation space. specially, rotate @xcite defines each relation as a rotation from the subject to the object. nevertheless, these distance-based distance models are still unable to capture reflexive relations which can hold, i.e. for a particular relation r each entity is related to itself via r. in distance-based models, the values or the phases of vectors for all reflexive relations are enforced to be 0, which does not allow to fully express the semantic characteristics of these relations. semantic matching models measure plausibility of facts by matching latent semantics of entities and relations embodied in their embedding representations. a few examples of such models include rescal @xcite , distmult @xcite , complex @xcite , quate @xcite and geome @xcite . rescal and distmult cannot capture asymmetric relations since the score of the triple (s, r, o) is always equal to the score of its symmetric triple (o, r, s). complex, quate and geome have been proven to be able to capture various relation patterns for static kgs, but cannot model temporary relations in tkgs due to their ignorance of time information. recent research illustrated that the performances of kge models can be further improved by incorporating time information in tkgs. some tkge models are extended from transe, e.g., ttranse (leblay and chekol, 2018), ta-transe @xcite and atise @xcite . another part of tkge models are temporal extensions of distmult, e.g., know-evolve @xcite , tdistmult @xcite and ta-distmult @xcite . similar to transe and distmult, these tkge models have issues with capturing reflexive relations or asymmetric relations. specially, de-simple @xcite ## a novel tkge approach based on temporal rotation although various kge models have been developed to learn multi-relational interactions between entities, all of them have problems with inferring temporary relations which are only valid for a certain time point or a last for a certain time period. to illustrate this by example, assume we are given a quadruple (barack obama, visits, france, 2014-02-12) as a training sample, where the relation visits is a temporary relation. if we query (barack obama, visits, ?, 2014-07-08), a trained static kge model probably returns the incorrect answer france due to the validness of the triple barack obama, visits, france, while the correct answer is ukraine considering the given time constraint. on the other hand, most of the existing tkge models, which were extended from transe @xcite and distmult @xcite , incorporate time information in the embedding space, but have limitations on learning transitive relations or asymmetric relations as discussed in section 2. to overcome the limitations of these existing kge and tkge models on learning and inferring over tkgs, we propose a new tkge model, tero, which defines the temporal evolution of an entity embedding as a rotation in the complex vector space. let e denote the set of entities, r denote the set of relations, and t denote the set of time steps. then a tkg is a collection of factual quadruples (s, r, o, t), where s, @xmath0, @xmath1, t denotes the actual time when the fact occurs. for any time t, we have a time step τ ∈ t representing this actual time. we map s, r, o to their complex embeddings, i.e., s, r, @xmath2; then we define the functional mapping induced by each time step τ as an element-wise rotation from the time-independent entity embeddings s and o to the time-specific entity embeddings s t and o t . the mapping function is defined as follows: ## conclusion in this work, we introduce tero, a new tkge model which represents entities or relations as single or dual complex embeddings and temporal changes as rotations of entity embeddings in the complex vector space. our model is advantageous with its capability in modelling several key relation patterns and handling time annotations in various forms. experimental results show that tero remarkably outperforms the existing state-of-the-art kge models and tkge models on link prediction over four well-established tkg datasets. specially, we adopt two different time-division approaches for various datasets and investigate the effect of the time granularity on the performance of our model.