Venue: NAACL 2025

# Mitigating Heterogeneity among Factor Tensors via Lie Group Manifolds

for Tensor Decomposition Based Temporal Knowledge Graph Embedding

Jiang Li\({}^{1,2,3}\) , Xiangdong Su\({}^{1,2,3}\)*, Guanglai Gao\({}^{1,2,3}\)

\({}^{1}\) College of Computer Science, Inner Mongolia University, China

\({}^{2}\) National & Local Joint Engineering Research Center of Intelligent Information Processing Technology for Mongolian, China

\({}^{3}\) Inner Mongolia Key Laboratory of Multilingual Artifcial Intelligence Technology, China lijiangimu@gmail.com, cssxd@imu.edu.cn

Corresponding Author

###### Abstract

Recent studies have highlighted the effectiveness of tensor decomposition methods in the Temporal Knowledge Graphs Embedding (TKGE) task. However, we found that inherent heterogeneity among factor tensors in tensor decomposition significantly hinders the tensor fusion process and further limits the performance of link prediction. To overcome this limitation, we introduce a novel method that maps factor tensors onto a unified smooth Lie group manifold to make the distribution of factor tensors approximating homogeneous in tensor decomposition. We provide the theoretical proof of our motivation that homogeneous tensors are more effective than heterogeneous tensors in tensor fusion and approximating the target for tensor decomposition based TKGE methods. The proposed method can be directly integrated into existing tensor decomposition based TKGE methods without introducing extra parameters. Extensive experiments demonstrate the effectiveness of our method in mitigating the heterogeneity and in enhancing the tensor decomposition based TKGE models1.

Footnote 1: Our code is available at [https://github.com/dellixx/tkbc-lie](https://github.com/dellixx/tkbc-lie)

+
Footnote †: _Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 2161–2172

## 1 Introduction

Knowledge graphs (KGs) are data structures that encapsulate knowledge triples of real-world entities and their interrelationships, and are widely used to improve information retrieval Liang et al. (2023), reasoning Xu et al. (2023), Q&A Hu et al. (2021), etc. Temporal knowledge graphs (TKGs) extend this paradigm by introducing timestamps into knowledge triplets to reflect the validity of facts over time and provide a deeper understanding and analysis of dynamic changes in the facts. Due to the data incompleteness in both KGs and TKGs, researchers propose many KG embedding (KGE) and TKG embedding (TKGE) methods to predict the missing facts, thereby enhancing the richness and accuracy of the KGs and TKGs. This work mainly focuses on TKGE.

As the interest in TKG grows, researchers proposed many TKGE methods and greatly promoted the development of TKG. Concerning the success of tensor decomposition in KGE Nickel et al. (2016); Trouillon et al. (2016); Lacroix et al. (2018), recent works Lacroix et al. (2020); Xu et al. (2021); Li et al. (2023) further extended tensor decomposition into TKGE and obtained very excellent performance. These works demonstrated that tensor decomposition can guarantee full expressiveness under specific embedding dimensionality bounds in TKG, thus enhancing the link prediction.

However, existing TKGE methods based on tensor decomposition suffer from inherent heterogeneity among factor tensors. Recent research Wu et al. (2020); Li et al. (2021) also highlights the intrinsic heterogeneity in TKGs, specifically in terms of entity and temporal heterogeneity. According to our analysis, the heterogeneity of entity, relation and timestamp originates from their semantic roles within the knowledge graphs. That is, the entities represent the static components of the graph, the relations delineate the interactions among the enti

Figure 1: (a) illustrates the heterogeneity in the distribution of entities, relations and timestamps within TKGs, as evidenced by the differing distribution curves. (b) illustrates the homogeneous distribution curves of entities, relations and timestamps when using our method to mitigate the heterogeneity among these three elements.



ties, and the timestamp characterizes the temporal aspects of these interactions, specifying when they occur and their duration. This heterogeneity leads to the learned factor tensor expliciting different distributions in TKGE, as shown in Figure 1(a). This further limits the tensor fusion in TKGE models and lowers the link prediction accuracy. More discussion about heterogeneity can be found in **Appendix A**.

Therefore, it is necessary to address the heterogeneity for tensor decomposition-based TKGE methods to enhance link prediction. To this target, we propose to map the factor tensors onto a unified smooth Lie group manifold to make the distribution of factor tensors approximating homogeneous in tensor decomposition, as shown in Figure 1(b). Since the manifold in Lie group looks the same at every point and all tangent spaces at any point are alike Sola et al. (2018), the factor tensors mapped by the Lie group have a smooth and unified distribution, which mitigates the heterogeneity among the factor tensors. We provide the theoretical proof of our motivation that homogeneous factor tensors are more effective in approximating the target compared to heterogeneous factor tensors in TKGE models in Sec. 4.1. We integrate the proposed method into several existing tensor decomposition based TKGE models and conduct extensive experiments to evaluate its effectiveness. The experimental results present the heterogeneity among factor tensors in TKGE methods and illustrate that the proposed method brings significant performance improvement. This confirms the effectiveness of our method in alleviating the heterogeneity. Our contributions are summarized as follows:

* To the best of our knowledge, we are the first to investigate the negative effect of the heterogeneity among the factor tensors for tensor decomposition based TKGE models and propose to enhance these models by diminishing the heterogeneity via Lie group manifold.
* We provide the theoretical proof of our motivation that homogeneous factor tensors are more effective than heterogeneous factor tensors in approximating the target in TKGE.
* Our proposed method can be directly integrated into the tensor decomposition based TKGE models without introducing any additional parameters, and extensive experiments on several TKGE models demonstrate its effectiveness and generalization.

## 2 Related Work

### Static Knowledge Graph Embedding

Drawing inspiration from the concept of translation invariance featured in word2vec Mikolov et al. (2013), TransE assesses the relations between entities and their links by calculating the distance from \({\mathbf{e}_{s}+\mathbf{e}_{\hat{r}}}\) to \({\mathbf{e}_{o}}\) using standard \(l_{1}\) or \(l_{2}\) norms, where \({\mathbf{e}_{s}}\) and \({\mathbf{e}_{o}}\) are the vectors that represent the starting and ending entities, and \({\mathbf{e}_{\hat{r}}}\) represents the linking relation. Following TransE, TransH Wang et al. (2014), TransR Lin et al. (2015), and TransD Ji et al. (2015) introduce various mapping ways and thus refine these embeddings for better KGE representation. ComplEx Trouillon et al. (2016) employs 3-th order tensor decomposition to capture the interactions within KGs. TorusE Ebisu and Ichise (2018) utilizes a torus (a donut-shaped manifold) for its embeddings. TorusE introduces a torus, which is a compact Abelian Lie group, and defines distance functions on the torus. The torus can be considered as a collection of multiple Lie groups. Instead, we map the factor tensors to the Lie group space, thus mitigating the distributional heterogeneity among them.

### Temporal Knowledge Graph Embedding

In TKGE models, the temporal information is added, and the scoring function is calculated for the quadruples to assess their reasonableness. Therefore, most TKGE models use existing KGE models as a foundation. TTransE Leblay and Chekol (2018) extends TransE and encodes time stamps \(\tau\) as translations same as relations. Hence, the score function of TTransE is denoted as \(\phi(s,\hat{r},o,\tau)=||\mathbf{s}+\mathbf{\hat{r}}+\mathbf{\tau}-\mathbf{o}||_{p}\). Furthermore, TA-TransE Garcia-Duran et al. (2018) encode timestamps based on TransE. RotateQVS Chen et al. (2022) uses quaternion embeddings to represent both entities and relations. Recently, BoxTE Messner et al. (2022) models the TKGE based on a box embedding model BoxE Abboud et al. (2020). TCompoundE Ying et al. (2024) employs relation-specific and time-specific compound geometric operations to enhance the modeling of temporal dynamics and relational patterns 



### Tensor Decomposition Based Temporal Knowledge Graph Embedding

TComplEx Lacroix et al. (2020) and TNTComplEx Lacroix et al. (2020) expand upon the ComplEx model by executing a fourth-order tensor decomposition in temporal knowledge graphs (TKGs). This method offers a more nuanced understanding of the temporal dimensions in knowledge graphs. TeLM Xu et al. (2021) utilizes the asymmetric geometric product, a method that allows for a more sophisticated and expressive representation of temporal relationships and entities. TeAST Li et al. (2023) maps relations onto Archimedean spiral timelines, and ensures that relations occurring simultaneously are placed on the same timeline, with all relations evolving over time. The above works are all based on tensor decomposition to optimize the TKGE representation. In this paper, we focus on exploring the problem of the heterogeneity of factor tensors in tensor decomposition based TKGE models.

## 3 Background and Notation

### TKGE Task

Given a TKG, let denote the set of entities, denote the set of relations, and denote the set of timestamps. A TKG can be defined as a collection of quadruples \((s,{\hat{r}},o,\tau)\), where \(s\in\mathcal{E}\), \({\hat{r}}\in\mathcal{R}\), \(o\in\mathcal{E}\) and \(\tau\in\mathcal{T}\) denote the subject entity, relation, object entity and timestamp, respectively. The TKGE task aims to accurately learn embedded representations of entities, relations and timestamps to facilitate predictions of missing entities in TKGs. Specifically, it involves predicting the object entity \(o\) given a tuple \((s,{\hat{r}},?,\tau)\), or conversely, predicting the subject entity \(s\) for a tuple \((?,{\hat{r}},o,\tau)\), thereby capturing the dynamic nature of relationships over time. In this paper, we denote the relation in TKG as \({\hat{r}}\) and the rank in mathematics as \(r\).

### Tensor Decomposition for TKGE

In the existing tensor decomposition based TKGE methods, each relation quadruple is represented by a \(\{0,1\}\)-valued 4-th order tensor \(\mathcal{Y}\in\{0,1\}\)Lacroix et al. (2020). This representation allows each element \(\mathcal{Y}_{s,{\hat{r}},o,\tau}=1\) to indicate that at a specific time \(\tau\), there is a relationship \({\hat{r}}\) between entities \(s\) and \(o\). In link prediction, tensor decomposition algorithms learn to infer a predicted tensor \(\mathcal{X}\) that approximates the ground truth \(\mathcal{Y}\), as

\[\mathcal{Y}\sim\mathcal{X}=\sum_{r=1}^{R}\mathbf{u}_{r}\otimes\mathbf{v}_{r} \otimes\mathbf{w}_{r}\otimes\mathbf{t}_{r}, \tag{1}\]

where rank \(r\in\{1,...,R\}\), \(\otimes\) denotes the tensor product. \(\mathbf{u}_{r}\), \(\mathbf{v}_{r}\), \(\mathbf{w}_{r}\) and \(\mathbf{t}_{r}\) denote the subject entity, relation, object entity and timestamp factor tensors. Tensor decomposition based TKGE methods aim to optimize the factor tensors to make \(\mathcal{X}\) as close as possible to the tensor \(\mathcal{Y}\) and thus achieve more accurate link prediction.

## 4 Methodology

This paper employs Lie group manifold to diminish the heterogeneity of factor tensors in tensor decomposition based TKGE models and thus improves the performance of these models. **In Sec. 4.1**, we provide the theoretical proof of our motivation that homogeneous tensors are more effective than heterogeneous tensors in approximating the target for tensor decomposition based TKGE methods. **In Sec. 4.2**, we explain why Lie groups can mitigate the heterogeneity among factor tensors, and describe how to map the factor tensors to the Lie group space. **In Sec. 4.3**, we introduce a Logarithmic Mapping '\(log(f(\cdot))\)' operation and alleviate the heterogeneity among factors by minimizing the difference between the original factor tensors and the mapped factor tensors in the Lie group through the N3 regularization in the loss function.

### Theoretical Analysis of Homogeneous _vs_. Heterogeneous Factor Tensors

Proposition: Homogeneous factor tensors (\(\mathbf{u}_{r}\), \(\mathbf{v}_{r}\), \(\mathbf{w}_{r}\), \(\mathbf{t}_{r}\)) with a low rank can effectively approximated \(\mathcal{Y}\) while heterogeneous factor tensors (\(\mathbf{u}_{r}\), \(\mathbf{v}_{r}\), \(\mathbf{w}_{r}\), \(\mathbf{t}_{r}\)) require a higher rank to approximate \(\mathcal{Y}\) in TKGE. \({}_{\blacksquare}\)

Proof: Given a 4th-order tensor decomposition in TKGE

\[\mathcal{Y}\approx\sum_{r=1}^{R}\mathbf{u}_{r}\otimes\mathbf{v}_{r}\otimes \mathbf{w}_{r}\otimes\mathbf{t}_{r}, \tag{2}\]

where \(R\) is the rank of the decomposition, and \((\mathbf{u}_{r},\mathbf{v}_{r},\mathbf{w}_{r},\mathbf{t}_{r})\in\mathbb{R}\) are factor tensors.

If the factor tensors (\(\mathbf{u}_{r}\), \(\mathbf{v}_{r}\), \(\mathbf{w}_{r}\), \(\mathbf{t}_{r}\)) are homogenous, there are a common set of basis vectors \(\mathbf{B}=\{\mathbf{b}_{1},\mathbf{b}_{2},\ldots,\mathbf{b}_{m}\}\) between these factor tensors. Based on this homogeneity, the column

vectors of each factor matrix can be expressed as linear combinations of the basis vectors in \(\mathbf{B}\).

\[\begin{split}&\mathbf{u}_{r}=\mathbf{B}\cdot\boldsymbol{\alpha}_{ r},\quad\mathbf{v}_{r}=\mathbf{B}\cdot\boldsymbol{\beta}_{r},\\ &\mathbf{w}_{r}=\mathbf{B}\cdot\boldsymbol{\gamma}_{r},\quad \mathbf{t}_{r}=\mathbf{B}\cdot\boldsymbol{\delta}_{r},\end{split} \tag{3}\]

where \(r=1,\ldots,R\), and \(\boldsymbol{\alpha}_{r}\), \(\boldsymbol{\beta}_{r}\), \(\boldsymbol{\gamma}_{r}\), and \(\boldsymbol{\delta}_{r}\) are the coefficient vectors for the \(r\)-th component in their respective factor matrices. Consequently, we can get

\[\mathcal{Y}\approx\sum_{r=1}^{R}(\mathbf{B}\cdot\boldsymbol{\alpha}_{r}) \otimes(\mathbf{B}\cdot\boldsymbol{\beta}_{r})\otimes(\mathbf{B}\cdot \boldsymbol{\gamma}_{r})\otimes(\mathbf{B}\cdot\boldsymbol{\delta}_{r}). \tag{4}\]

Given the homogeneity among factor tensors, we can further obtain the representation as follows

\[\mathcal{Y}\approx\sum_{j=1}^{m}\mathbf{b}_{j}\otimes\mathbf{b}_{j}\otimes \mathbf{b}_{j}\otimes\mathbf{b}_{j}\cdot\sum_{r=1}^{R}\alpha_{jp}\beta_{jr} \gamma_{jp}\delta_{jn}, \tag{5}\]

where \(\alpha_{jp}\), \(\beta_{jr}\), \(\gamma_{jr}\), and \(\delta_{jr}\) are the scalar coefficients corresponding to the projection of the \(r\)-th component onto the \(j\)-th basis vector in their respective factor matrices. The set of basis vectors \(\{\mathbf{b}_{1},\mathbf{b}_{2},\dots,\mathbf{b}_{m}\}\) are orthogonal to each other, ensuring that each dimension represented by these vectors is independent. \(\lambda_{j}=\sum_{r=1}^{R}\alpha_{jp}\beta_{jr}\gamma_{jr}\delta_{jr}\) can be considered as scaling constants. Thus, we can get a reduced-rank tensor \(\mathcal{Y}^{\prime}\) as

\[\mathcal{Y}\approx\mathcal{Y}^{\prime}=\sum_{j=1}^{m}\lambda_{j}\mathbf{b}_{ j}\otimes\mathbf{b}_{j}\otimes\mathbf{b}_{j}\otimes\mathbf{b}_{j}, \tag{6}\]

which captures the essence of the homogeneity within the factor matrices. The rank \(m\) of \(\mathcal{Y}^{\prime}\) is less than the original rank \(R\) (\(m<R\)). Thus, homogeneous factor tensors (\(\mathbf{u}_{r}\), \(\mathbf{v}_{r}\), \(\mathbf{w}_{r}\), \(\mathbf{t}_{r}\)) with a low rank can effectively approximate the target quadruple tensor \(\mathcal{Y}\).

In contrast, if the factor tensors (\(\mathbf{u}_{r}\), \(\mathbf{v}_{r}\), \(\mathbf{w}_{r}\), \(\mathbf{t}_{r}\)) are highly heterogeneous, they exhibit distinct semantics and distributions characteristics. This implies that these factor tensors cannot be effectively approximated by a simple, small number of rank-1 tensors. Each rank-1 tensor can be viewed as a representation of a specific pattern or feature. For these heterogeneous tensors, the patterns they capture, or the semantics they represent within the data necessitate a larger number of rank-1 tensors to capture their diverse characteristics individually. Hence, the heterogeneous factor tensors (\(\mathbf{u}_{r}\), \(\mathbf{v}_{r}\), \(\mathbf{w}_{r}\), \(\mathbf{t}_{r}\)) require a higher rank or even full rank to approximate \(\mathcal{Y}\) in TKGE. Since higher rank means more parameters to estimate and more computation, homogeneous factor tensors are more effective than heterogeneous factor tensors in approximating the target for tensor decomposition based TKGE methods.

### Mitigating Heterogeneity via Lie Group

In TKGE, the underlying reason why highly heterogeneous factor tensors require a higher rank to approximate the target vector is that the heterogeneity among factor tensors can limit the fusion process of subsequent computations, which can be analogous to the multimodal fusion process Chen and Zhang (2020). Therefore, it is crucial to mitigate the heterogeneity among factor tensors to approximate the target tensor more efficiently. Our motivations for choosing Lie groups to mitigate heterogeneity in TKGE are as follows.

Firstly, Lie groups are adept at maintaining structural integrity and handling data's dynamic nature over time, making them a suitable choice for TKGE. The application of Lie groups in fields like robotics Sola et al. (2018), machine learning Sommer et al. (2020); Lee and Civera (2022), and computer vision Teed and Deng (2021) underscores its capability to model complex geometric transformations effectively. Secondly, as shown in Figure 2, Lie group is a mathematical structure that simultaneously satisfies the axioms of a group and the properties of a smooth manifold. It is like a curved, smooth hyper-surface, with no edges or spikes, embedded in a space of higher dimension. The smoothness of the manifold implies the existence of a unique tangent space at each point. In

Figure 2: An illustration of the relation between the Lie group and the Lie algebra. The Lie algebra \(\mathfrak{so}(n)\) is the tangent space to the Lie group’s manifold \(S(n)\).

 

a Lie group, the manifold looks the same at every point, and therefore all tangent spaces at any point are alike. Thus, the factor tensors mapped by the Lie group have a smooth and unified distribution, which further mitigates the heterogeneity among the factor tensors.

In this study, we map the factor tensors to the same Lie group space and make the factor tensors have a unified distribution to mitigate the heterogeneity among them. To facilitate the description of our method, we employ factor tensor of rank 4 in the subsequent discussions. Given a factor tensor \({\mathbf{e}}\) of rank 4 and map it to the Lie group \(SO(2)\) space, we get

\[f(\cdot):\mathbb{R}^{n}\to SO(2);~{}~{}~{}{\mathbf{e}}\mapsto{\bf R} \tag{7}\]

where \(f(\cdot)\) denots Lie group mapping operation. \({\bf R_{e}}\) is a rotation matrix in \(SO(2)\), denoted as

\[{\bf R_{e}}=\begin{pmatrix}\cos{\mathbf{e}}&-\sin{\mathbf{e}}\\ \sin{\mathbf{e}}&\cos{\mathbf{e}}\end{pmatrix}. \tag{8}\]

Accordingly, given a quadruple (\(s\), \(\hat{r}\), \(o\), \(\tau\)) in TKG, its corresponding factor tensors are (**u**, **v**, **w**, **t**) in the tensor decomposition based TKGE models. We map these four factor tensors onto Lie group space, and we get the rotation matrices

\[{\bf R_{u}}=\begin{pmatrix}\cos{\mathbf{u}}&-\sin{\mathbf{u}}\\ \sin{\mathbf{u}}&\cos{\mathbf{u}}\end{pmatrix},{\bf R_{v}}=\begin{pmatrix} \cos{\mathbf{v}}&-\sin{\mathbf{v}}\\ \sin{\mathbf{v}}&\cos{\mathbf{v}}\end{pmatrix}, \tag{9}\] \[{\bf R_{w}}=\begin{pmatrix}\cos{\mathbf{w}}&-\sin{\mathbf{w}}\\ \sin{\mathbf{w}}&\cos{\mathbf{w}}\end{pmatrix},{\bf R_{t}}=\begin{pmatrix} \cos{\mathbf{t}}&-\sin{\mathbf{t}}\\ \sin{\mathbf{t}}&\cos{\mathbf{t}}\end{pmatrix}.\]

When generalized to \(n\) rank, \(SO(\sqrt{n})\) can be denoted as a \(Givens\) rotation matrix. In an \(n\)-dimensional space, a Givens rotation is performed by fixing \(\sqrt{n}-2\) dimensions and applying a rotation transformation within the plane formed by the remaining two dimensions. This effectively isolates the rotation to a specific two-dimensional subspace. The Givens rotation matrix \(G(i,j,{\mathbf{e}})\) for rotating the \(i\)-th and \(j\)-th coordinates in \(\sqrt{n}\)-dimensional space is given by:

\[G(i,j,{\mathbf{e}})=\begin{bmatrix}1&\cdots&0&\cdots&0&\cdots&0\\ \vdots&\ddots&\vdots&&\vdots&&\vdots\\ 0&\cdots&\cos({\mathbf{e}})&\cdots&-\sin({\mathbf{e}})&\cdots&0\\ \vdots&&\vdots&\ddots&\vdots&&\vdots\\ 0&\cdots&\sin({\mathbf{e}})&\cdots&\cos({\mathbf{e}})&\cdots&0\\ \vdots&&&\vdots&&\vdots&\ddots&\vdots\\ 0&\cdots&0&\ldots&0&\cdots&1\end{bmatrix}, \tag{10}\]

where the rotation occurs in the plane spanned by the \(i\)-th and \(j\)-th basis vectors. The matrix is an identity matrix except for the four elements \(G_{ii}\), \(G_{ij}\), \(G_{ji}\), and \(G_{jj}\), which form the \(2\times 2\) rotation block within the larger matrix. Hence, we focus on the \(2\times 2\) base rotation matrices in the code implementation.

### Logarithmic Mapping from \(So(n)\) to \(so(n)\)

Our training goal is to diminish the heterogeneity among the factor tensors in TKGE model training and thus improve the link prediction performance. Due to the Lie group \(SO(n)\) residing on a non-Euclidean manifold, we introduce a variant of Logarithmic Mapping operation Huang et al. (2017) (denoted as \(log(\cdot)\)) on the Lie group space and converts the rotation matrices into the usual skew-symmetric matrices which are situated in the Euclidean space, as

\[log(\cdot):SO(n)\rightarrow\mathfrak{so}(n),~{}~{}~{}{\bf R}\mapsto log({\bf R}). \tag{11}\]

The logarithm mapping \(log({\bf R})\) is

\[log({\bf R})=\left\{\begin{aligned} & 0,&\text{if}~{}~{}~{}~{}\theta({\bf R})=0,\\ &\frac{\theta({\bf R})}{2\sin(\theta({\bf R}))}({\bf R}-{\bf R}^{T}),& \text{otherwise},\end{aligned}\right. \tag{12}\]

where \(\theta({\bf R})\) is the angle of \({\bf R}\), as

\[\theta({\bf R})=\arccos\left(\frac{trace({\bf R})-1}{2}\right). \tag{13}\]

Here, \(trace(\cdot)\) is a square matrix is the sum of its diagonal elements.

The mathematical derivation of logarithmic mappings references this work Sola et al. (2018). After logarithmic mapping, we get \(log(f({\bf u}_{r}))\), \(log(f({\bf v}_{r}))\), \(log(f({\bf w}_{r}))\), \(log(f({\bf t}_{r}))\). Then, we calculate the differences between the original tensors and their corresponding mapped tensors on the Lie group

\[\begin{split}{\bf u}_{r}^{\prime}&={\bf u}_{r}-log( f({\bf u}_{r})),\\ {\bf v}_{r}^{\prime}&={\bf v}_{r}-log(f({\bf v}_{ r})),\\ {\bf w}_{r}^{\prime}&={\bf w}_{r}-log(f({\bf w}_{r})),\\ {\bf t}_{r}^{\prime}&={\bf t}_{r}-log(f({\bf t}_{r})). \end{split} \tag{14}\]

Finally, we perform the standard tensor decomposition with \({\bf u}_{r}^{\prime}\), \({\bf v}_{r}^{\prime}\), \({\bf w}_{r}^{\prime}\), \({\bf t}_{r}^{\prime}\), as

\[\mathcal{Y}\sim\mathcal{X}=\sum_{r=1}^{R}\lambda_{r}{\bf u}_{r}^{\prime}\otimes{ \bf v}_{r}^{\prime}\otimes{\bf w}_{r}^{\prime}\otimes{\bf t}_{r}^{\prime}.\] (15) 

Following previous works (Lacroix et al., 2018; Xu et al., 2021; Li et al., 2023), we use the full multiclass log-softmax loss function and N3 regularization to optimize the factor tensors, which are defined as follows:

\[\mathcal{L}=-\log(\frac{\exp(\phi(s,\hat{r},o,\tau))}{\sum_{s^{\prime}\in \mathcal{E}}\exp(\phi(s^{\prime},\hat{r},o,\tau))})\]

\[-\log(\frac{\exp(\phi(o,\hat{r}^{-1},s,\tau))}{\sum_{o^{\prime}\in\mathcal{E}} \exp(\phi(o^{\prime},\hat{r}^{-1},s,\tau))})\]

\[+\lambda_{\mu}\sum_{i=1}^{R}(\|\mathbf{u}_{r}^{\prime}\|_{3}^{3}+\|\mathbf{v}_ {r}^{\prime}\|_{3}^{3}+\|\mathbf{w}_{r}^{\prime}\|_{3}^{3}+\|\mathbf{t}_{r}^{ \prime}\|_{3}^{3}), \tag{16}\]

where \(\lambda_{\mu}\) denotes N3 regularization weight, \(\hat{r}^{-1}\) is the inverse relation. We use \(\|\mathbf{t}_{r}^{\prime}\|_{3}^{3}\) to represent the temporal regularizer for simplicity, which is computed in N3 regularization way.

By using the N3 regularization, we minimize the \(\mathbf{u}_{r}^{\prime}\), \(\mathbf{v}_{r}^{\prime}\), \(\mathbf{w}_{r}^{\prime}\), \(\mathbf{t}_{r}^{\prime}\). That is, we drive the \(\mathbf{u}_{r}\), \(\mathbf{v}_{r}\), \(\mathbf{w}_{r}\), \(\mathbf{t}_{r}\) to be homogeneous in Euclidean space, since the \(log(f(\mathbf{u}_{r}))\), \(log(f(\mathbf{v}_{r}))\), \(log(f(\mathbf{w}_{r}))\), \(log(f(\mathbf{t}_{r}))\) are tend to be homogeneous in Lie group space. Therefore, the proposed method can mitigate the heterogeneity among factor tensors in tensor decomposition based TKGE methods.

## 5 Experiments

### Datasets

To evaluate the effectiveness of the proposed method, we evaluate our method on two popular TKGE benchmark datasets. ICEWS14 and ICEWS05-15 (Garcia-Duran et al., 2018) are both extracted from the _Integrated Crisis Early Warning System (ICEWS)_ dataset (Lautenschlager et al., 2015), which consists of temporal sociopolitical facts starting from 1995. ICEWS14 consists of sociopolitical events in 2014 and ICEWS05-15 involves events occurring from 2005 to 2015. ICEWS14 is a fine temporal granularity dataset, while ICEWS05-15 has a wider temporal granularity relative to ICEWS14. See **Appendix B** for summary statistics of the dataset and more discussion of the dataset.

### Evaluation Protocol

In this research, we follow the previous works (Lacroix et al., 2020; Xu et al., 2021; Li et al., 2023) to evaluate our method. Specifically, to evaluate the quality of the ranking for each test quadruples, we calculate all possible substitutions for the subject and object entities, denoted as \((s^{\prime},\hat{r},o,\tau)\) and \((s,\hat{r},o^{\prime},\tau)\), where \(s^{\prime}\), \(o^{\prime}\in\mathcal{E}\). After that, we sort the score of candidate quadruples under the time-wise filtered settings (Lacroix et al., 2020; Xu et al., 2021; Li et al., 2023). The performance is evaluated using standard evaluation metrics, including Mean Reciprocal Rank (MRR) and Hits@\(n\). The Hits@\(n\) metric measures the percentage of correct entities in the top \(n\) predictions. Higher values of MRR and Hits@\(n\) indicate better performance. Hits ratio with cut-off values \(n=1,3,10\). In this paper, we utilize H@\(n\) to denote Hits@\(n\) for convenience.

### Experimental Setup

We implement our method based on the existing training framework2. All experiments are trained on a single NVIDIA Tesla A100. The hyperparameters used in the experiment are consistent with the optimal hyperparameters of the original paper report. The best models are selected by early stopping (threshold of 10) on the validation datasets. The max epoch is 200. We report the average results on the test set for five runs. To ensure a fair validation of the effectiveness of our method, we employ the same hyperparameter configuration in both the before and after comparison experiments.

Footnote 2: [https://github.com/facebookresearch/tkbc](https://github.com/facebookresearch/tkbc)

According to Lie group mapping described in Sec. 4.2, it is essential to ensure that the rank \(r\) of the matrix can be satisfied by the square root of \((2\times r)\) is an integer in our method. This requirement arises from the implementation of the matrix logarithm map for TcomplEx, TNTcomplEx and TeAST. The rank of TeLM needs to be satisfied by the square root of \((4\times r)\) is an integer.

## 6 Results and Analysis

### Main Results

In our experiments, we validate the effectiveness of our proposed method for dealing with TKGs heterogeneity in tensor decomposition on ICEWS14 and ICEWS05-15 datasets. The improvements are marked in red in Table 1, highlighting the advancements over the baselines. When our method is applied to different tensor decomposition based TKGE models, they all achieve meaningful improvements in different metrics. This significant improvement confirms our Proposition 1, in which homogeneous factor tensors can be effectively approximated \(\mathcal{Y}\) with a low rank. Additionally, the 

ICEWS05-15 dataset validates the effectiveness of our method in mitigating data heterogeneity, with TeAST notably exhibiting an average improvement of 10.3 points.

In conclusion, the experimental results provide robust evidence supporting Proposition 1. The experimental outcomes validate the theoretical framework of our study and demonstrate that our novel method effectively alleviates data heterogeneity in tensor decomposition and enhances the link prediction performance of these models. More experiments on the large TKG dataset GDELT can be found in Appendix C.

### Quantitative Analysis on Heterogeneity

In this section, we perform a quantitative analysis to prove the effectiveness of our proposed method. For any factor tensors \({\mathbf{e}_{x}}\) and \({\mathbf{e}_{y}}\), the skew-symmetric matrices in \(\mathfrak{so}(n)\) are given by

\[A_{x}=log(f({\mathbf{e}_{x}})),\quad A_{y}=log(f({\mathbf{e}_{y}})). \tag{17}\]

We define the difference between \({\mathbf{e}_{x}}\) and \({\mathbf{e}_{y}}\) in \(\mathfrak{so}(n)\) to be denoted as \(d(A_{x},A_{y})\). The relationship between the set of skew-symmetric matrices \(\{A_{x}\}\) obtained from the mapping of a set of vectors \({\mathbf{e}_{x}}\) can be described using the operations in the Lie algebra, such as computing their Lie brackets. This process of mapping vectors to \(SO(n)\) and then to \(\mathfrak{so}(n)\) transforms them into elements with a unified algebraic structure, mitigating the differences in structural distribution between them.

To quantify the structural differences between the skew-symmetric matrices \(A_{x}\) and \(A_{y}\) in \(\mathfrak{so}(n)\), we consider the Frobenius norm of their difference

\[\|A_{x}-A_{y}\|_{F}=\sqrt{\text{trace}((A_{x}-A_{y})^{T}(A_{x}-A_{ y}))}. \tag{18}\]

This norm provides a measure of the difference between the corresponding matrices.

Based on the above quantitative formulas, we evaluate on the representative models TComplEx and TNTComplEx. Specifically, we calculate the difference between entity and relation, entity and timestamp, and relation and timestamp. As shown in Table 2, we calculate their average distance difference on ICEWS14.

As shown in Table 2, for the standard TComplEx model, the average difference in distance between entities and relations is 15.71, between entities and timestamps is 7.61, and betwee

\begin{table}
\begin{tabular}{l r l c c c c c c c c c c} \hline \hline  & \multicolumn{9}{c}{**ICEWS14**} & \multicolumn{9}{c}{**ICEWS05-15**} \\ \cline{2-13}  & \multicolumn{1}{c}{_rank_} & \multicolumn{1}{c}{_Para._} & MRR & H@1 & H@3 & H@10 & _rank_ & _Para._ & MRR & H@1 & H@3 & H@10 \\ \hline \multicolumn{13}{l}{_Tensor Decomposition Based TKGE Models_} \\ \hline TCompIEx\({}^{\heartsuit}\) & 128 & 2.04M & 55.3 & 46.3 & 60.7 & 71.5 & 128 & 3.84M & 58 & 49 & 64 & 76 \\ TNTComplEx\({}^{\heartsuit}\) & 128 & 2.15M & 55.7 & 46.3 & 61.5 & 73.0 & 128 & 3.97M & 60 & 50 & 65 & 78 \\ TeLM & 121 & 3.85M & 50.6 & 42.1 & 55.0 & 67.1 & 121 & 7.26M & 56.8 & 48.7 & 61.1 & 72.0 \\ TeAST & 128 & 2.13M & 53.4 & 43.9 & 58.8 & 70.9 & 128 & 4.87M & 48.8 & 38.4 & 54.7 & 68.3 \\ \hline \multicolumn{13}{l}{_Tensor Decomposition Based TKGE Models_} \\ \hline TCompIEx & 128 & 2.04M & **56.2** & **46.8** & **61.6** & **73.5** & & & & & **59.6** & **50.2** & **65.3** & **77.2** \\  & (+0.9) & (+0.5) & (+0.9) & (+0.9) & (+0.9) & & & & (+1.6) & (+0.3) & (+1.3) & (+1.2) \\ \hline \multirow{2}{*}{TNTComplEx} & \multirow{2}{*}{128} & \multirow{2}{*}{2.15M} & **56.3** & **46.7** & **61.8** & **74.1** & & & & & **60.2** & **50.8** & **65.9** & **78.1** \\  & & (+0.6) & (+0.4) & (+0.3) & (+1.1) & & & & & (+0.2) & (+0.8) & (+0.9) & (+0.1) \\ \hline \multirow{2}{*}{TeLM} & \multirow{2}{*}{121} & \multirow{2}{*}{3.85M} & **54.5** & **45.5** & **59.5** & **71.7** & & & & & & **59.0** & **50.6** & **63.8** & **74.7** \\  & & (+3.9) & (+3.4) & (+4.5) & (+3.6) & & & & & (+2.2) & (+1.9) & (+2.7) & (+2.7) \\ \hline \multirow{2}{*}{TeAST} & \multirow{2}{*}{128} & \multirow{2}{*}{2.13M} & **56.1** & **47.3** & **61.4** & **72.3** & & & & & **59.2** & **50.5** & **64.8** & **76.9** \\  & & (+2.7) & (+3.4) & (+2.6) & (+1.4) & & & & (10.4) & (+12.1) & (+10.1) & (+8.6) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Link prediction results on ICEWS14 and ICEWS05-15. The results of \({}^{\heartsuit}\) are taken from Lacroix et al. (2020). Other results are obtained from our experiments. \(log(f(\cdot))\) indicates that our proposed method.

 

timestamps is 15.72. These results demonstrate significant differences in quantification between different types of embeddings.

Further, when our method \(log(f(\cdot))\) is employed in the TComplEx model, we observe a reduction in all three distance difference. This significant improvement points to the effectiveness of our method in mitigating the heterogeneity among different factor tensors. Similarly, the results of the TNTComplEx model support the above statement.

### Visualisation Analysis

To further verify that our method can effectively mitigate the heterogeneity between factor embeddings, we utilize t-SNE Van der Maaten and Hinton (2008) to visualize the learned entity, relation, and timestamp embeddings. As shown in Figure 3, we can observe that the learned factor embeddings through our method exhibit a trend towards homogeneity. This further demonstrates the inherent heterogeneity present among different types of embeddings in TKGE based on tensor decomposition. Our method effectively mitigates this issue, demonstrating that meaningful performance improvements can be achieved.

### Effect of Rank

In this work, we compare the performance of the standard TeAST Li et al. (2023) model with our proposed TeAST model enhanced by \(+\log(f(\cdot))\) across different rank values on ICEWS14. As shown in Figure 4, we observe that the performance of both models improves with the increase of rank values. However, after the rank value reaches 800, the pace of performance improvement slows down. This is because the representation capacity of the model reaches saturation at a certain level, beyond which the marginal benefits of increasing rank diminish. Additionally, higher rank values might lead to overfitting, especially in cases of sparse data, negatively affecting the model's generalization ability on unseen data.

## 7 Conclusion

In this study, we are the first to introduce methods to mitigate heterogeneity in factor tensors within tensor decomposition-based TKGE models. We reveal that the heterogeneity primarily stems from diverse semantic content among elements (entities, relation and timestamps), which impedes the effective fusion of factor tensors and limits link prediction accuracy. We prove that homogeneous tensors are more effective than heterogeneous tensors in tensor fusion and approximating the target for tensor decomposition based TKGE methods. Our method maps factor tensors onto a smooth Lie group manifold to standardize their distribution and

Figure 4: Results of TeAST and TeAST\(+log(f(\cdot))\) with different rank on ICEWS14.

Figure 3: Visualisations of the learned entity, relation and timestamp embeddings on ICEWS14.

 

mitigate heterogeneity without increasing model complexity. Our experimental results demonstrate the effectiveness of this method in mitigating tensor heterogeneity and enhancing performance. We hope that this work can offer fresh insights for research in the field of TKGE.

## Limitations

In this paper, we investigate the effect of the heterogeneity among factor tensors on link prediction in tensor decomposition based TKGE models. We mainly focus on addressing the issue of heterogeneity among elements within TKGs, which is recognized as a key challenge in this domain. Moreover, similar to the majority of TKGE models, our method is unable to process new entities that are not present in the training data.

## Acknowledgments

This work was funded by National Natural Science Foundation of China (Grant No. 62366036), National Education Science Planning Project (Grant No. BIX230343), Key R&D and Achievement Transformation Program of Inner Mongolia Autonomous Region Grant No. 2022YFHHH0077), The Central Government Fund for Promoting Local Scientific and Technological Development (Grant No. 2022ZY0198), Program for Young Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region (Grant No. NJYT24033), Inner Mongolia Autonomous Region Science and Technology Planning Project (Grant No. 2023YFSH0017), Science and Technology Program of the Joint Fund of Scientific Research for the Public Hospitals of Inner Mongolia Academy of Medical Sciences (Grant No.2023GLLH0035).

## References

* Abboud et al. (2020) Ralph Abboud, Ismail Ilkan Ceylan, Thomas Lukasiewicz, and Tommaso Salvatori. 2020. Boxe: A box embedding model for knowledge base completion. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. Cited by: SS1.
* Cai et al. (2022) WEBAN: a survey on machine learning with applications to document analysis. In Proceedings of the 2022 ACM SIGKDD international conference on Knowledge discovery & data mining, pp. 1295-1305. External Links: Document Cited by: SS1.
* K. Chen, Y. Wang, Y. Li, and A. Li (2022)Rotateqvs: representing temporal information as rotations in quaternion vector space for temporal knowledge graph completion. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 5843-5857. External Links: Link, Document Cited by: SS1.
* T. Ebisu and R. Ichise (2018)Toruse: knowledge graph embedding on a lie group. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pp. 1819-1826. External Links: Link, Document Cited by: SS1.
* November 4, 2018, pp. 4816-4821. External Links: Link, Document Cited by: SS1.
* Zikun Hu, Y. Cao, L. Huang, and T. Chua (2021)How knowledge graph and attention help? a qualitative analysis into bag-level relation extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pp. 4662-4671. External Links: Link, Document Cited by: SS1.
* Z. Huang, C. Wan, T. Probst, and L. Van Gool (2017)Deep learning on lie groups for skeleton-based action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: SS1.
* G. Ji, S. He, L. Xu, K. Liu, and J. Zhao (2015)Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers, H. Larocia, P. Obozinski, and N. Usunier (Eds.), Berlin, Heidelberg, Germany, pp. 687-696. External Links: ISBN 978-3-319-3891-3, Link, Document Cited by: SS1.
* T. Lacroix, G. Obozinski, and N. Usunier (2020)Tensor decompositions for temporal knowledge base completion. In 8th International Conference on Learning Representations, ICLR 2020, 

Addis Ababa, Ethiopia, April 26-30, 2020_. OpenRevieww.net.
* Lacroix et al. (2018) Timothee Lacroix, Nicolas Usunier, and Guillaume Obozinski. 2018. Canonical tensor decomposition for knowledge base completion. In _Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmassan, Stockholm, Sweden, July 10-15, 2018_, volume 80 of _Proceedings of Machine Learning Research_, pages 2869-2878. PMLR.
* Lautenschlager et al. (2015) Jennifer Lautenschlager, Steve Shellman, and Michael Ward. 2015. Icews event aggregations.
* Leblay and Chekol (2018) Julien Leblay and Melisachew Wudage Chekol. 2018. Deriving validity time in knowledge graph. In _Companion of the The Web Conference 2018 on The Web Conference 2018, WWW 2018, Lyon , France, April 23-27, 2018_, pages 1771-1776. ACM.
* Lee and Civera (2022) Seong Hun Lee and Javier Civera. 2022. Hara: A hierarchical approach for robust rotation averaging. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 15777-15786.
* Li et al. (2023) Jiang Li, Xiangdong Su, and Guanglai Gao. 2023. TeAST: Temporal knowledge graph embedding via archimedean spiral timeline. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 15460-15474, Toronto, Canada. Association for Computational Linguistics.
* Li et al. (2021) Zhifei Li, Hai Liu, Zhaoli Zhang, Tingting Liu, and Neal N Xiong. 2021. Learning knowledge graph embedding with heterogeneous relation attention networks. _IEEE Transactions on Neural Networks and Learning Systems_, 33(8):3961-3973.
* Liang et al. (2023) Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, and Xinwang Liu. 2023. Learn from relational correlations and periodic events for temporal knowledge graph reasoning. In _Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval_, pages 1559-1568.
* Lin et al. (2015) Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In _Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA_, pages 2181-2187. AAAI Press.
* March 1, 2022_, pages 7779-7787. AAAI Press.
* Mikolov et al. (2013) Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In _Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States_, pages 3111-3119.
* Nickel et al. (2016) Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. 2016. A review of relational machine learning for knowledge graphs. _Proceedings of the IEEE_, 1(104):11-33.
* Park et al. (2022) Namyong Park, Fuchen Liu, Purvanshi Mehta, Dana Cristofor, Christos Faloutsos, and Yuxiao Dong. 2022. EVokg: Jointly modeling event time and network structure for reasoning over temporal knowledge graphs. In _Proceedings of the fifteenth ACM international conference on web search and data mining_, pages 794-803.
* Sola et al. (2018) Joan Sola, Jeremie Deray, and Dinesh Atchuthan. 2018. A micro lie theory for state estimation in robotics. _CoRR_, abs/1812.01537.
* Sommer et al. (2020) Christiane Sommer, Vladyslav Usenko, David Schubert, Nikolaus Demmel, and Daniel Cremers. 2020. Efficient derivative computation for cumulative b-splines on lie groups. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 11148-11156.
* Seed and Deng (2021) Zachary Teed and Jia Deng. 2021. Tangent space backpropagation for 3d transformation groups. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_, pages 10338-10347.
* Trouillon et al. (2016) Theo Trouillon, Johannes Welbl, Sebastian Riedel, Eric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In _Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016_, volume 48 of _JMLR Workshop and Conference Proceedings_, pages 2071-2080. JMLR.org.
* Van der Maaten and Hinton (2008) Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. _Journal of machine learning research_, 9(11).
* Wang et al. (2014) Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In _Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, July 27 -31, 2014, Quebec City, Quebec, Canada_, pages 1112-1119. AAAI Press.
* Wu et al. (2020) Jiapeng Wu, Meng Cao, Jackie Chi Kit Cheung, and William L Hamilton. 2020. Temp: Temporal message passing for temporal knowledge graph completion. _arXiv preprint arXiv:2010.03526_.
* Xu et al. (2021) Chengjin Xu, Yong-Yu Chen, Mojtaba Nayyeri, and Jens Lehmann. 2021. Temporal knowledge graph completion using a linear temporal regularizer and 

multivector embeddings. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021_, pages 2569-2578. Association for Computational Linguistics.
* Xu et al. (2023) Yi Xu, Junjie Ou, Hui Xu, and Luoyi Fu. 2023. Temporal knowledge graph reasoning with historical contrastive learning. In _Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023_, pages 4765-4773. AAAI Press.
* Ying et al. (2024) Rui Ying, Mengting Hu, Jianfeng Wu, Yalan Xie, Xiaoyi Liu, Zhunheng Wang, Ming Jiang, Hang Gao, Linlin Zhang, and Renhong Cheng. 2024. Simple but effective compound geometric operations for temporal knowledge graph completion. In _Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 11074-11086, Bangkok, Thailand. Association for Computational Linguistics.
* Zhang et al. (2019) Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. 2019. Heterogeneous graph neural network. In _Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining_, pages 793-803.

## Appendix A Definition and Discussion of Heterogeneity in TKG

In KGs, 'heterogeneity' refers to the semantic difference of the entity and relation. Similarly, in TKGs, 'heterogeneity' refers to the semantic difference of the entity, relation and timestamp. The heterogeneity among entities, relations, and time is as follows: (1) The heterogeneity between entities and relations is reflected in their structural roles within the graph, with entities existing as nodes and relations represented as edges between nodes. (2) The heterogeneity between entities and time is reflected in the fact that entities represent the static components of the graph, while time affects the changes in entity attributes and their relations. (3) The heterogeneity between relations and time is reflected in that relations delineate the interactions among entities, while time characterizes the temporal aspects of these interactions, specifying when they occur and their duration. Recent work Zhang et al. (2019); Li et al. (2021); Cai et al. (2022) also indicates that KGs have an intrinsic property of heterogeneity, which contains various types of entities and relations. Since TKG extends the KG paradigm, they inherently exhibit this heterogeneity as well. Additionally, TKGs incorporate temporal information, which further contributes to time heterogeneity.

The heterogeneity in TKG leads to the learned factor tensor expliciting different distributions in tensor decomposition based TKGE methods. Unlike previous works Wu et al. (2020); Park et al. (2022), we do not propose a model for modeling heterogeneous TKGs, but rather a unified approach for mitigating heterogeneity among entities, relations and timestamps via Lie group.

## Appendix B Statistics of Datasets

Statistics of all the datasets used in this work are listed in Table 3. \(\mathcal{E}\) denotes the set of entities, \(\mathcal{R}\) denotes the set of relations, and \(\mathcal{T}\) denotes the set of timestamps.

The ICEWS14 and ICEWS05-15 datasets exhibit heterogeneity across multiple dimensions, encompassing a wide array of entities, relations, and temporal variations. These datasets include diverse entities such as countries, governmental bodies, individuals, and organizations, each with unique attributes and patterns of behavior. The relations captured within these datasets are equally varied, detailing interactions ranging from diplomatic engagements to military conflicts, each bearing distinct characteristics and impacts. Furthermore, the chronological recording of events introduces a dynamic aspect to the data, with entities and their interrelations evolving over different times.

## Appendix C Results on Larger Dataset

The above experiments have validated that our proposed method can improve the TKGE performance on the high-heterogeneity KGs by mitigating heterogeneity among factor tensors for tensor decomposition based methods. To further validate the

\begin{table}
\begin{tabular}{c|c c c} \hline \hline  & **ICEWS14** & **ICEWS05-15** & **GDELT** \\ \hline \(\mathcal{E}\) & 7,128 & 10,488 & 500 \\ \(\mathcal{R}\) & 230 & 251 & 20 \\ \(\mathcal{T}\) & 365 & 4017 & 366 \\ \hline \#Train & 72,826 & 386,962 & 2,735,685 \\ \#Vaidl & 8,963 & 46,092 & 341,961 \\ \#Test & 8,941 & 46,275 & 341,961 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Statistics of ICEWS14, ICEWS05-15 and GDELT datasets in the experiment.

 

effectiveness, we conduct experiments on a larger and more challenging TKG dataset GDELT. The GDELT covers only 500 most common entities and 20 most frequent relations, while the number of quadruples achieves 2M. This is reflected in the denser relations between entities in KGs. Hence, the GDELT dataset is a challenging large-scale TKG.

We chose TComplEx and TNTComplEx models as the backbone model in the experiment on GDELT. The results are shown in Table 4. From Table 4, we observe that there is significant performance improvement in terms of H@1, H@3, and H@10. This proves that the proposed method can effectively diminish the heterogeneity among the factor tensors in TKGE. It exemplifies the potential of our method in handling large-scale TKGs.

## Appendix D Method Efficiency Comparison

Since we implement logarithmic mapping in our implementation of Lie group mapping using the following method, our method is theoretically a linear operation with \(O(n)\) time complexity. The following table shows the training time required for our method on the ICEWS05-15 dataset compared to other methods, confirming the linear operation of our method without significantly increasing the computational cost. As shown in Table 5, we can observe that the training time for models using the \(log(f(\cdot))\) method shows a slight increase. However, considering the potential performance improvements brought by our method, this additional time cost is acceptable.