Mitigating Heterogeneity among Factor Tensors via Lie Group Manifolds for Tensor Decomposition Based Temporal Knowledge Graph Embedding (2025)

Abstract: Recent studies have highlighted the effectiveness of tensor decomposition methods in the Temporal Knowledge Graphs Embedding (TKGE) task. However, we found that inherent heterogeneity among factor tensors in tensor decomposition significantly hinders the tensor fusion process and further limits the performance of link prediction. To overcome this limitation, we introduce a novel method that maps factor tensors onto a unified smooth Lie group manifold to make the distribution of factor tensors approximating homogeneous in tensor decomposition. We provide the theoretical proof of our motivation that homogeneous tensors are more effective than heterogeneous tensors in tensor fusion and approximating the target for tensor decomposition based TKGE methods. The proposed method can be directly integrated into existing tensor decomposition based TKGE methods without introducing extra parameters. Extensive experiments demonstrate the effectiveness of our method in mitigating the heterogeneity and in enhancing the tensor decomposition based TKGE models.

## introduction knowledge graphs (kgs) are data structures that encapsulate knowledge triples of real-world entities and their interrelationships, and are widely used to improve information retrieval @xcite , reasoning @xcite , q&a @xcite , etc. temporal knowledge graphs (tkgs) extend this paradigm by introducing timestamps into knowledge triplets to reflect the validity of facts over time and provide a deeper understanding and analysis of dynamic changes in the facts. due to the data incompleteness in both kgs and tkgs, researchers propose many kg embedding (kge) and tkg embedding (tkge) methods to predict the missing facts, thereby enhancing the richness and accuracy of the kgs and tkgs. this work mainly focuses on tkge. as the interest in tkg grows, researchers proposed many tkge methods and greatly promoted the development of tkg. concerning the success of tensor decomposition in kge @xcite @xcite , recent works @xcite @xcite further extended tensor decomposition into tkge and obtained very excellent performance. these works demonstrated that tensor decomposition can guarantee full expressiveness under specific embedding dimensionality bounds in tkg, thus enhancing the link prediction. however, existing tkge methods based on tensor decomposition suffer from inherent heterogeneity among factor tensors. recent research @xcite also highlights the intrinsic heterogeneity in tkgs, specifically in terms of entity and temporal heterogeneity. according to our analysis, the heterogeneity of entity, relation and timestamp originates from their semantic roles within the knowledge graphs. that is, the entities represent the static components of the graph, the relations delineate the interactions among the enti-ties, and the timestamp characterizes the temporal aspects of these interactions, specifying when they occur and their duration. this heterogeneity leads to the learned factor tensor expliciting different distributions in tkge, as shown in figure 1 @xcite . this further limits the tensor fusion in tkge models and lowers the link prediction accuracy. more discussion about heterogeneity can be found in appendix a. therefore, it is necessary to address the heterogeneity for tensor decomposition-based tkge methods to enhance link prediction. to this target, we propose to map the factor tensors onto a unified smooth lie group manifold to make the distribution of factor tensors approximating homogeneous in tensor decomposition, as shown in figure 1(b ). since the manifold in lie group looks the same at every point and all tangent spaces at any point are alike @xcite , the factor tensors mapped by the lie group have a smooth and unified distribution, which mitigates the heterogeneity among the factor tensors. we provide the theoretical proof of our motivation that homogeneous factor tensors are more effective in approximating the target compared to heterogeneous factor tensors in tkge models in sec. 4.1. we integrate the proposed method into several existing tensor decomposition based tkge models and conduct extensive experiments to evaluate its effectiveness. the experimental results present the heterogeneity among factor tensors in tkge methods and illustrate that the proposed method brings significant performance improvement. this confirms the effectiveness of our method in alleviating the heterogeneity. our contributions are summarized as follows: ## methodology this paper employs lie group manifold to diminish the heterogeneity of factor tensors in tensor decomposition based tkge models and thus improves the performance of these models. in sec. 4.1, we provide the theoretical proof of our motivation that homogeneous tensors are more effective than heterogeneous tensors in approximating the target for tensor decomposition based tkge methods. in sec. 4.2, we explain why lie groups can mitigate the heterogeneity among factor tensors, and describe how to map the factor tensors to the lie group space. in sec. 4.3, we introduce a logarithmic mapping 'log(f (•))' operation and alleviate the heterogeneity among factors by minimizing the difference between the original factor tensors and the mapped factor tensors in the lie group through the n3 regularization in the loss function. ## conclusion in this study, we are the first to introduce methods to mitigate heterogeneity in factor tensors within tensor decomposition-based tkge models. we reveal that the heterogeneity primarily stems from diverse semantic content among elements (entities, relation and timestamps), which impedes the effective fusion of factor tensors and limits link prediction accuracy. we prove that homogeneous tensors are more effective than heterogeneous tensors in tensor fusion and approximating the target for tensor decomposition based tkge methods. our method maps factor tensors onto a smooth lie group manifold to standardize their distribution and mitigate heterogeneity without increasing model complexity. our experimental results demonstrate the effectiveness of this method in mitigating tensor heterogeneity and enhancing performance. we hope that this work can offer fresh insights for research in the field of tkge.