R otate QVS : Representing Temporal Information as Rotations in Quaternion Vector Space for Temporal Knowledge Graph Completion (2022)

Abstract: Temporal factors are tied to the growth of facts in realistic applications, such as the progress of diseases and the development of political situation, therefore, research on Temporal Knowledge Graph (TKG) attracks much attention. In TKG, relation patterns inherent with temporality are required to be studied for representation learning and reasoning across temporal facts. However, existing methods can hardly model temporal relation patterns, nor can capture the intrinsic connections between relations when evolving over time, lacking of interpretability. In this paper, we propose a novel temporal modeling method which represents temporal entities as Rotations in Quaternion Vector Space (RotateQVS) and relations as complex vectors in Hamilton’s quaternion space. We demonstrate our method can model key patterns of relations in TKG, such as symmetry, asymmetry, inverse, and can capture time-evolved relations by theory. And empirically, we show that our method can boost the performance of link prediction tasks over four temporal knowledge graph benchmarks.

## introduction knowledge graphs (kgs) have been widely adopted to represent informative knowledge or facts in real-world applications @xcite @xcite . however, as known facts are usually sparse, kgs are far from completeness. thus, knowledge graph completion (kgc) methods are proposed to predict missing facts, i.e. links between entities @xcite @xcite . furthermore, in real world, many facts are bonded with a particular time by nature. for example, barack obama is the president of usa is only valid for the time period 2009 -2017. to model such time-sensitive facts, temporal knowledge graphs (tkgs) have * corresponding author. tkg embedding (tkge) methods @xcite @xcite were proposed to represent entities and relations with temporal features in tkgs @xcite . but how to present them with temporal interpretability remains a challenge for state-of-the-art tkge models. further, it is crucial for tkg completion (tkgc) to leverage the learned temporal information. previous static kgc works @xcite @xcite learn explainable embeddings of various relation patterns, so that symmetric pattern (e.g. "co-author"), asymmetric pattern (e.g. "affiliation"), inverse pattern (e.g. "buyer" vs. "seller") and complex composition pattern (e.g. "father's wife (mother)" vs. "wife's father (father in law)") can be captured in static kgs. however, in tkgs, there are inherent connections between entities and their relations along with time-evolving. for example, the relation between kit harington and rose leslie is in love in 2012, becomes engaged in 2017, and then turns into married in 2018. to the best of our knowledge, very few of the existing tkge methods can capture them. to address this problem, we take inspirations from hamilton's quaternion number system @xcite @xcite and propose a novel method based on quaternion. to be specific, we encode both entities and relations as quaternion embeddings, and then temporal entity embeddings can be represented as rotations in quaternion vector space (rotate-qvs). theoretically, we show the limitations of previous methods and demonstrate that performing quaternion embeddings can model symmetric, asymmetric, and inverse relation patterns. meanwhile, we prove our method is capable of cap-turing time-evolving information in tkg explicably. we empirically evaluate our method over four tkgc benchmarks and report state-of-theart performance consistently. further, we perform analysis of the learned quaternion embeddings and show the abilities of our rotateqvs for modeling various relation patterns, including temporal evolution. we summarize our main contributions as follows: ## preliminaries on hamilton's quaternions quaternion number system (hamilton, 1844) is an extension of traditional complex numbers. recently, quaternion has been applied in static knowledge graph embedding @xcite . for readers better understanding our method in section 3, we introduce the definition and basic operations of quaternion in this section. ## proposed method in this section, we introduce a novel temporal modeling approach for tkg by representing temporal information as rotations in quaternion vector space (rotateqvs). ## related work models working on static knowledge graph have been well studied @xcite @xcite ) with semantic and structure information. translation based methods, e.g. transe @xcite and transr @xcite , formalise the factual distance between a head entity s and a tail entity o with the translation carried out by the relation. adopting tensor factorization with a bilinear transformation, semantic matching models, e.g. rescal @xcite and dist-mult @xcite , capture the semantic relevance of entities. recently, more attention were paid to study various relation patterns. rotate @xcite treat each relation as a rotation so that symmetric/asymmetric, inversion and composition patterns can be inferred to predict missing links. further, quaternion number system (hamil-ton, 1844) is applied to model more complex composition patterns in 3d space, such as rotate3d @xcite and quate @xcite . many aforementioned methods @xcite @xcite @xcite are extended from static static kgs to tkgs. they integrate time information into previous static methods as independent features. others study the dynamic evolution of tkg. atise @xcite regards the temporal evolution of entity and relation embeddings as combinations of trend component, seasonal component and random component. cygnet @xcite proposes a time-aware copy-generation mechanism leveraging known facts in the past to predict unknown facts in the future. tero @xcite defines the temporal evolution of entity embedding as a rotation in the complex vector space. inspired by tero, our rotateqvs further represents temporal entities as rotations in quaternion vector space and obtains more advantages. 12 modeling various temporal relation patterns @xcite , especially the temporal-evolution patterns, is crucial for tkge and the following tkgc. zhang et al. mentions the time-evolution property, but does not make a systematic research on it. it remains an open research question with few researches. our work (rotateqvs) takes inspirations from the idea of rotation and generalises it into the quaternion number system to model the complex temporal-evolution pattern that tero can hardly do. ## conclusion in this paper, we introduce a novel tkgc method rotateqvs which represents temporal information of tkgs as rotations in quaternion vector space. targeting temporal interpretability, we theoretically analyse that rotateqvs can model various relation patterns and demonstrate it with extensive experiments. compared to previous methods, rotate-qvs has made significant improvements on link prediction tasks over four benchmark datasets. furthermore, we show our rotateqvs has great advantages in modeling various relation patterns with temporal evolution.