DECRL: A Deep Evolutionary Clustering Jointed Temporal Knowledge Graph Representation Learning Approach
Authors: Qian Chen
Ling Chen
Venue: NeurIPS 2024 poster (2024)
Keywords: Event Prediction
Temporal Knowledge Graphs
Representation Learning
Evolutionary Clustering
Graph Neural Networks
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Temporal Knowledge Graph (TKG) representation learning aims to map temporal evolving entities and relations to embedded representations in a continuous low-dimensional vector space. However, existing approaches cannot capture the temporal evolution of high-order correlations in TKGs. To this end, we propose a **D**eep **E**volutionary **C**lustering jointed temporal knowledge graph **R**epresentation **L**earning approach (**DECRL**). Specifically, a deep evolutionary clustering module is proposed to capture the temporal evolution of high-order correlations among entities. Furthermore, a cluster-aware unsupervised alignment mechanism is introduced to ensure the precise one-to-one alignment of soft overlapping clusters across timestamps, thereby maintaining the temporal smoothness of clusters. In addition, an implicit correlation encoder is introduced to capture latent correlations between any pair of clusters under the guidance of a global graph. Extensive experiments on seven real-world datasets demonstrate that DECRL achieves the state-of-the-art performances, outperforming the best baseline by an average of 9.53\%, 12.98\%, 10.42\%, and 14.68\% in MRR, Hits@1, Hits@3, and Hits@10, respectively.

DECRL: A Deep Evolutionary Clustering Jointed Temporal Knowledge Graph Representation Learning Approach
Qian Chen, Ling Chen'
State Key Laboratory of Blockchain and Data Security
College of Computer Science and Technology Zhejiang University
(qianchencs, lingchenn (iipcchenns,,nnccenee
0cs zju. edu. zdu.edu. cn
Abstract
Temporal Knowledge Graph (TKG) representation learning aims map tempo- ral evolving entities and relations to embedded representations in continuous low-dimensional vector space. However, existing approaches cannot capture the temporal evolution of high-order correlations in TKGs. To this end, we propose a Deep Evolutionary Clustering jointed temporal knowledge graph Representation Learning approach (DECRL). Specifically, a deep evolutionary clustering module is proposed to capture the temporal evolution high-order correlations among entities. Furthermore, cluster-aware unsupervised alignment mechanism is intro- duced to ensure the precise one-to-one alignment soft overlapping clusters across timestamps, thereby maintaining the temporal smoothness of clusters. In addition, an implicit correlation encoder is introduced to capture latent correlations between any pair of clusters under the guidance of global graph. Extensive experiments on seven real-world datasets demonstrate that DECRL. achieves the state-of-the-an performances, outperforming the best baseline by average of 9.53%, 12.98% 10. 10.42%, and 14.68% in MRR, Hits 1, Hits 3, and Hits 10, respectively.
Introduction
Temporal Knowledge Graphs (TKGs) collections of human temporal evolving knowledge [35]. which are widely utilized various fields, e.g., information retrieval [99] natural language under- standing [3) and recommendation systems 32 A TKG represents events in the form of quadruples (s, r. o, t), where s and o denote the subject and object entities, respectively. r denotes the relation between and o, and t represents the timestamp [35]. Event prediction in TKGs is an important task that predicts future events according to historical events [17]. TKG representation learning continuous low-dimensional vector space. Due to the complex temporal dynamics and multi-relations within TKGs, TKG representation learning poses great challenges the research community.
aims map temporal evolving entities and relations embedded representations
In recent years, many TKG representation learning approaches have used graph neural networks 33 (GNNs) to model pairwise entity correlations [14] D. Furthermore, some researchers have leveraged derived structures, e.g., communities [39], entity groups (28). and hypergraphs [26] 29 to model oigh-order correlations among entities, i.e., the simultaneous correlations among three more entities.
These approaches, however, lack the capability to capture the temporal evolution of high- correlations TKGs. As kind of dynamic graph, TKGs inherently feature the temporal evolution
"Corresponding author: Ling Chen
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

of high-order correlations, For example, within TKGs focused solely on countries, a country entity may be affiliated with different international political organizations different timestamps, with these organizations experiencing membership adjustment over time. In addition, the membership adjustment in international political organizations does not shift suddenly. Typically, numerous events occur before the formal adjustment of their membership, gradually pushing memberships away from or drawing them closer to international political organizations. For instance, the UK's official departure from the European Union (EU) was preceded by numerous events that incrementally estranged from the EU.
To address the aforementioned deficiencies, Deep Evolutionary Clustering jointed temporal knowl- edge graph Representation Learning approach (DECRL) is proposed for event prediction in TKGs. To the best of our knowledge, DECRL is the first work that integrates deep evolutionary clustering approaches into TKGs, which jointly optimizes TKG representation learning with evolutionary clustering to capture the temporal evolution of high-order correlations. Our main contributions are outlined as follows:
We propose a deep evolutionary clustering module to capture the temporal evolution of high-order correlations among entities, where clusters represent the high-order correlations between multiple entities. Furthermore, a cluster-aware unsupervised alignment mechanism is introduced ensure precise one-to-one alignment of soft overlapping clusters across timestamps, maintaining the temporal smoothness of clusters over successive timestamps.
We propose implicit correlation encoder to capture latent correlations between any pair clusters, which defines the interaction intensities between clusters to form a cluster graph. In addition, a global graph, constructed from all events of training set, is introduced to guide the assignment of different interaction intensities to different cluster pairs.
We evaluate DECRL on seven real-world datasets. The experimental results demonstrate that DECRL achieves the oftth- (SOTA) performance. It outperforms the best baseline by an average of 9.53%, 12.98%, 10.42%, and 14.68% in MRR, Hits@ 1, Hits@3, and Hits@ 10, respectively.
2 Related work
2.1 TKG representation learning approach
GNNs and variants of recurrent neural networks (RNNs) are commonly integrated to model graph
structural information and temporal dependency within TKGs, respectively [ 34 16 [4]. For example, TeMP 3 and RE-GCN 16) both utilize relation-aware GCN 24 to model the influence
of neighbor entities and apply GRUs to model the temporal dependency. TiRGN [4 employs multi- relational GNNs to model graph structural information, and utilizes GRUs to capture the temporal dependency across sequential timestamps. However, these approaches often resort to stacking multiple layers to model the influence of distant neighbors, which may lead to the over-smoothing problem.
Recent research advancements have introduced paths [5 g 18 to enhance the modeling capability of the latent pairwise correlations between entities. For example, xERTE [8] utilizes time-aware
graph attention mechanism to obtain and exploit local subgraphs. TLogie [8] employs a time-
based random walk strategy to acquire logical paths. Furthermore, some researchers have leveraged derived structures, c.g., communities [39], entity groups (28), and hypergraphs [29], to model high- order correlations among entities. For example, EvoExplore (39 utilizes a temporal point process enhanced by a hierarchical attention mechanism to establish dynamic communities for modeling latent correlations among entities. DHyper (29) utilizes hypergraph neural networks to model high-order correlations among entities and among relations.
While the promising results of introducing derived structures, existing approaches lack the capability to capture the temporal evolution of high-order correlations in TKGs. To address this gap, we propose DECRL, which is the first work to integrate deep evolutionary clustering approaches into TKGs. jointl optimizes TKG representation learning with evolutionary clustering to capture the temporal evolution of high-order conrlltionss
2

---

2.2
Deep evolutionary clustering approach
Existing deep evolutionary clustering approaches employ different strategies for discovering stable clusters 21 36 20 38 371. For example, DYNMOGA [7] employs a deep evolutionary clustering identify evolving communities within dynamic networks, which treats the process as multi-objective optimization problem aimed at cluster stability. sE-NMF [2]] implements linear fusion of adjacency matrices across timestamps to enhance the temporal stability of clusters. Both DNETC [36] and CDNE (20] use constructed temporal smoothness loss functions to ensure consistent and stable evolution of clustering results through successive timestamps. TRNNGCN (37] introduces decay matrix to assess the influence of historical clustering results, thereby stabilizing the current clustering configuration.
approach designed
These approaches straightforwardly incorporate prior clustering results with current timestamp results over time under the presumption that clustering results are relatively stable across different timestamps. However, the nuances between similar clusters are difficult to recognize in this way. In necessary to discern the subtle differences in high-order correlations. To bridge this gap, in our work, cluster-aware unsupervised alignment mechanism is introduced, which ensures the precise one-to-one alignment of soft overlapping clusters across timestamps, thereby maintaining the temporal smoothness of clusters over successive timestamps.
TKG representation learning,
3 Preliminaries
Definition 1 (TKG). A TKG is a set of events formalized as = ((s,r,o,) 8, E E where E, R. and T denote the set of entities, rela- tions, and timestamps, respectively. G denotes the set of events at timestamp r.
Definition (Entity Graph). The entity graph constructed based on G which is multi-relational graph, and can be denoted as G = (Vo E where Ve and Eg denote the set of nodes and edges of the entity graph at timestamp 6, respectively. The nodes in Ve represent entities, while the edges in Eg represent relations between entities at timestamp t.
Definition 3 (Cluster Graph). The cluster graph is constructed based on G which is fully connected graph, and can be denoted as G = (V F where V and F denote the set nodes and edges of the cluster graph timestamp i, respectively. The nodes and edges of cluster graphs represent clusters and the latent correlations between them, respectively, where each cluster represents the high-order correlations among entities.
Task (Event Prediction). Given a query (s,?, o, t), the event prediction task in TKGs aims to predict the conditional probability of all relations with the subject entity 8, the object entity o, and the historical event sets G Gl:T-1, which can be denoted as p(F o, ol:T.-11 =1), where T denotes the number of historical timestamps.
4 Methodology
4.1
Overview
The proposed DECRL approach is illustrated in Figure] T At each timestamp, entity and relation representations updated by the cluster graph message passing module are merged with input represen- tations from the previous timestamp using a time residual gate, serving the input for the current timestamp. The multi-relational interactions among entities are modeled by the relation-aware GCN, The deep evolutionary clustering module captures the temporal evolution of high-order correlations among entities, maintaining the temporal smoothness of clusters over successive timestamps through an unsupervised alignment mechanism. The implicit correlation encoder captures the latent correla- tions between any pair of clusters, enabling message passing within the cluster graph to update entity representations. Finally, the attentive temporal encoder captures temporal dependencies among entity and relation representations across timestamps, integrating them with temporal information for future event prediction. The computational complexity of DECRL can be found in Appendix A.
4.2 Evolutionary clustering module
The entity graph G (Vi, F!) is constructed based on the historical events at timestamp I, which can be used to capture the structural dependency among concurrent events. Since the entity graph
3

---

Rensbal Coey
Breieal
Rabskon GCN
Telatie Away GCN
Tounin
Gragk
Ivventuemrr Nookie
Cibbel
Steellrity Mazza
Lapiiu
Cleatr Cimph
Figure 1: The framework of DECRL.
is multi-relational graph, relation-aware GCN which is formulated as:
24
is utilized to model the structural dependency,
h 11141 RRRLU
Σ
Wi(h + + 22h
(1)
do
where h. h. and h.d denote the tth layer representations of subject entity object entity and relation timestamp i, respectively. W and Wa learnable parameters for aggregating neighbor entity representations and self-loop, respectively. d, denotes the in-degree of object entity o. Initially, entities are assigned random representations based on their in-degree, such that entities with the same in-degree have the same initial representations, thereby efficiently accelerating the update process of entity representations. Relations are given with the random initial representations. For simplicity, the subscript 1 of variables is omitted in the following sections without causing ambiguity.
The representation of the relation timestamp t is influenced the r-related entity representations at timestamp and its representation at timestamp & - 1. The updating of the relation representation x.-1y where [:] denotes the concatenation operation. pooling
formulated as h = pooling j h denotes the mean pooling operation. e/y, denotes r-related entity representations at timestamp where V = (i](i, o, or (s, E g').
A country entity may be affiliated with different international political organizations at different timestamps within TKGs. Thus, the fuzzy e-means clustering method (23 utilized obtain the soft overlapping clusters, which outputs the membership matrix between entities and clusters by optimizing the object function as follows:
N₂
N
(e), (A)
N.
8.1.
1,0
No
where No and N: are the number of entities and clusters, respectively. vi_ denotes the membership degree of the entity representation e to the cluster centroid μ² at timestamp t. denotes the fuzzy

---

smoothing parameer,,,,,r, which is typically set to value greater than 1. (-) denotes the dot product.
denotes the norm of vector. Then, the representation of clusters is formulated as:
[eetillas
cj
Σ cVEV
No Σ
(ef |elll]
where cj denotes the representation of cluster at timestamp t, weighted by the membership degree of different entities to the cluster.
The maximum weight matching can be found in polynomial time using the Hungarian algorithm II Herein, cluster-aware Hungarian matching algorithm is introduced ensure smooth one-to-one aligmment of clusters across timestamps. The cluster-aware Hungarian matching algorithm quantifies the similarity between clusters of consecutive timestamps, i.c., t and which creates an affinity matrix A, where each element aj represents the similarity between the cluster j at timestamp and the cluster & at timestamp t: jj,k cos(c))- ...)
With the affinity matrix established, the cluster-aware Hungarian algorithm seeks an optimal as- signment that maximizes the sum of similarities across matched clusters. It is formulated an optimization problem: max, Σj= aj,z(i)+ where is the permutation function representing the one-to-one alignment between clusters consecutive timestamps. Therefore, x(j) denotes the index of the cluster at timestamp / matched with the cluster at timestamp /
Subsequently, the fused cluster representations are computed by taking weighted combination of the cluster representations at consecutive timestamps, which is formulated as:
cj 3 + (1- 3)
(5)
where 3 is the fusion weight, which indicates the relative contribution of each cluster representation consecutive timestamps to the fused cluster representation.
What's more, it is essential to ensure the temporal smoothness of these cluster representations across consecutive timestamps. To this end, temporal smoothness loss is introduced to penalize significant deviations of cluster representations across consecutive timestamps, which is formulated as:
T-1
No
(c)
Cremporal
(6)
clll
1=1 j=1
4.3
Cluster graph message passing module
Due to the intricate relationships and alliances formed among international political organizations, crucial capture the latent correlations between clusters for event prediction. Herein, an implicit correlation encoder is proposed to capture the latent correlations between any pair of clusters. The cluster graph is designed be fully connected graph to avoid missing any relevant information between clusters. The representations of the latent correlations are formulated as:
$ = RcLU (v(()), cf))
where of and cf are representations of the cluster and at timestamp t, respectively. yp is the transformation function, which is implemented by the multi-layer perceptron.
worth noting that clusters exhibit varying degrees of interaction, characterized by different intensities of latent correlations. Thus, we quantify the intensity of latent correlations between clusters, which formulated as:
d = (Conv(s(,))
where a is the sigmoid function, which ensures the resulting correlation intensity bounded between and 1. Conv(-) represents the convolution operation.
The intensity the latent correlations between clusters varies the short term and intensifies over time; higher frequency of interactions indicates stronger latent correlation. To this end,
5

---

construct a global graph, which is static graph composed of all events from the training set. The spectral clustering approach is then applied to this global graph to obtain global clusters cf glohal The similarity between global clusters is used to enhance the intensity of latent correlations between cluster pairs, which is formulated as:
gglobaly "*())
d.. dh.s x miy. m
|| (1)
III|
glohal 7*
calobal where (1) and @alobal (²))) are global clusters matched with the cluster and j at timestamp t, respec- tively. m is the similarity between the global cluster i and timestamp t. F is the enhanced intensity of latent correlations, which also integrates structural information from the global graph.
In the message passing process of the cluster graph, the stronger the intensity of the latent correlation, the more critical the latent correlation is, and vice versa. The aggregation operation is formulated as:
(10)
where F. and sf,, denote the enhanced intensity and the representation of latent correlation calculated by Equation I and Equation 7, respectively. Then, the representation of clusters is updated through:
c q p(u', ct)
(11)
To implement information transfer from clusters individual entities, from Section 4.2 Entity representations are updated based on their associated cluster representations, which are formulated as ex" = Σ u C₁,, where N₂ denotes the number of clusters.
the membership matrix
j=
4.4 Time residual gate
The time residual gate is introduced combine updated entity and relation representations with input representations of the current timestamp through weighted mechanism. This approach preserves inherent characteristics while capturing the temporal evolution of entities and relations. For simplicity, the subscripts i and j of variables are omitted in the following sections without causing ambiguity. The final updated representations at timestamp f are formulated as:
H' X' & H' + (1) & H' X' o (WWH' b)
(12)
where H'J denotes the entity relation representations updated by the cluster graph message passing module at timestamp t. " denotes the output of entity relation representations at timestamp t 1, and also is the input of timestamp t. The time residual gate, i.e., X determines the inherent characteristics to be preserved, where o is the sigmoid function. Wa and b are learnable parameters.
4.5 Attentive temporal encoder
In the context of sequence modeling, the attentive temporal encoder is introduced capture the temporal dependency among the final updated representations across timestamps. The position- enhanced representations of entity and relation are formulated as:
(t)) - v [oss(w1)))
:
(13)
where H' is the final updated representation of entity relation at timestamp t. [:] denotes the concatenation operation. ((L) is the time position encoder. d denotes the value of the representation dimension. wil wa are learnable parameters. 7' is the timestamp. Then, the temporal dependency is captured by a position-enhanncee
self-attention mechanism based on representation sequence 1 {z x² =T-1j, For simplicity, the subscript of variables omitted in the following sections. The integrated entity or relation representation is formulated as:
3mn (W W 2
exp(3m,n)
Vd
Om,n
Fum
Om mm
(14)
exp(3m,s)
n=1
where W, 9 and Wx k are learnable parameters. Omn, is the attention weight. hm the integrated representation of entity relation, which integrates the temporal information.

---

4.6 Event prediction
ConvTransE 25 is employed as the decoder of DECRL, which predicts the probability of each relation between an entity pair, which is formulated as:
p(r|s, o, g. = a H, FTF,CCnTTaann CoonTrranB((e, Ew))
(15)
where f is the probability vector of relations. is the sigmoid function. H, is the relation representa- tion matrix, each row of which corresponds to an integrated relation representation T. Conv (onvTransE(())) contains -ie-dimnniioaal convolution layer and a fully connected layer.
The event prediction training objective is minimizing the cross-entropy loss, which formulated as:
1 LTKG - ΣΣ (i log + (1 Vi.s) 1og(1_)t,)}
(16)
where S and N, denote the numbers of samples in the training set and relations, respectively, Vi. denotes the label of relation j for sample i, of which the element is 1 if the event occurs, otherwise 0. Pi.j is the predicted probability of relation j for sample i, calculated by Equation [5]
Finally, the total loss for DECRL is formulated as:
== (1 X)CTкG ACremporal
(17)
where hyper-parameter that controls the trade-off between event prediction and temporal smoothness, which is bounded between 0 and 1.
5 Experiments
In this section, we evaluate the performance of DECRL through comprehensive experiments. We first describe the datasets and experimental settings, followed by comparison with 12 SOTA TKG representation learning approaches. Next, we present an ablation study and syper-parameter sensitivity analysis (see Appendix A..5). Finally, offer case studies showcasing the effectiveness of DECRL.
5.1 Datasets and experimental settings
We evaluate DECRL seven real-world datasets: ICEWS14 330 ICEWSI8 2 ICEWS14C 29
ICEWSISC 129,, GDELT [3, WIKI [2], and YAGO 9. The ICEWS14 and ICEWS18 datasets span January 1, 2014, to December 31, 2014, and January 1, 2018, to October 31, 2018, respectively, We derive ICEWS14C 14C and ICEWSI8C datasets by filtering the original datasets to focus exclusively events involving countries. The GDELT dataset spans January 1, 2018, January 31, 2018. The WIKI and YAGO datasets are subsets the Wikipedia history and YAGO3 (22). respectively. Dataset statistics and experimental settings are detailed in Appendices A.2 and A.3
5.2 Comparison with baselines
DECRL is compared against 12 SOTA TKG representation learning approaches, categorized into shallow encoder-based approaches, i.e., TTransE [12 and HyTE [5]. DNN-based approaches, i.e.,
RE-NET 2 Glean G,, TeMP 34 RE-GCN 16 DACHA
4 and TRRG [4), and derived
structure- based approaches, i.e., TTTer (77. EvoExplore
39 GTRL. (28. and DHyper (29]. All
baselines are evaluated following consistent experimental settings with well-tuned hyper-parameters.
The detailed descriptions of compared baselines can be found in Appendix A.4
Tables T and 3 display the MRR and Hits @ 1/3/10 results of event prediction on ICEWS14, ICEWST8 8, ICEWS14C ICEWS18C, and GDELT datasets. "*" denotes the statistical superiority of DECRL over compared approaches based on pairwise t-tests 95% confidence level. The best results are highlighted in bold, while the second-bes results are underlined. Notably, DECRL consistently exhibits superior performance, yielding average improvements of 13.24%, 12.98%, 10.42%. and 14.68% in MRR and Hits 1/3/10 metrics, respectively. These findings affirm the efficacy integrating deep evolutionary clustering for capturing the temporal evolution of high-order correlations among entities. Furthermore, it is evident that approaches leveraging derived structures
7

---

Table 1: The performance DECRL and the compared approaches on ICEWS14 and ICEWS14C
ICEWS14
ICEWS14C
Approach
MRR
Hits@1
Hits@3
Hits@ 10
MRR
Hits@I
Hits@3
Hits 10
TTransE (WWW 2018)
23.79*
14.24*
29.17*
34.56*
11.79*
13.24*
19.97*
24.88*
HYTE (EMNLP 2018)
25.12*
18.15
30.15*
45.37*
22.17
18.15*
27.28*
35.37*
RE-NET (EMNIP 2020)
45.77*
37.98*
49.07*
58.87*
43.27
36.97*
47.08*
55.19*
Glean (KDD 2020)
42.20*
36.86*
47.68*
52.39*
40.24*
34.62*
45.48*
50.09*
TeMP (EMNLP 2020)
46.04
39.07*
49.84*
59.74*
44.17*
37.37*
47.78*
55.66*
RE-GCN (SIGIR 2021)
45.56*
38.09*
50.37*
62.44*
41.76*
36.67*
45.37*
51.74"
DACHA (TKDD 2022)
4
37.88*
49.47*
58.69*
44.26*
37.59*
44.18*
53.19*
TiRGN (JCCAI 2022)
46.07
39.83*
52.17*
63.95
44.73*
38.13*
49.77*
60.91
TiTer (EMNLP 2021)
46.12*
39.08*
50.76*
60.39"
44.86*
39.37*
48.84*
55.79*
EvoExplore (KBS 2022)
47.71*
40.68
52.37*
65.94*
49.77
40.12*
54.37*
65.83*
GTRL (TKDE 2023)
46.25*
40.11
51.09*
65.79*
50.95*
40.31
52.09*
64.89
DHyper (TOIS 2024)
56.15*
43.76
65.46*
85.89
54.16
41.45*
62.03*
75.35"
DECRL
62.61
48.73
70.57
93.03
58.55
44.62
66.52
82.06
Improvement
11.50%
11.36%
7.81% %
8.31%
8.11 %
7.65 %
7.24 %
8.91
Table 2: The performance of DECRL and the compared approaches on ICEWS18 and ICEWS18C
ICEWS18
ICEWS18C
Approach
MRR
Hits@1
Hits@3
Hits@ 10
MRR
Hits@
Hits@3
Hits@ 10
TTransE (WWW 2018)
11.96*
13.97
12.79*
24.33*
9.84*
10,29*
11.04
18.89*
HyTE (EMNLE 2018)
21.85*
16.86*
25.64*
41.86*
22.23*
16.27*
25.68*
33.39*
RE-NET (EMNLP 2020)
42.25*
33.81"
44.98*
52.72*
41.05*
32.87*
42.78
50.43*
Glean (KDD 2020)
37.11*
34.15%
42.56*
47.35*
35.58*
32.26*
40.44
46.49*
TcMP (EMNLPP 2020)
43.24*
38.77*
45.04*
55.94*
43.08*
36.07*
43.18*
53.03
RE-GCN (SIGIR 2021)
41.56*
37.59"
44.34
57.42*
40.27*
36.35*
41.75
49.25*
DACHA (TKDD 2022)
43.87*
37.11
47.47*
57.69*
40.11
36.11*
46.17*
52,37*
TiRGN (UCAA 2022)
44.27*
38.13*
50.66*
62.90*
43.57
37.23*
47.67*
54,44
TITer (EMNLP 2021)
45.44*
39.78
48.77*
58.73*
44.07*
38.85*
46.44
49,79*
EvoExplore (KBS 2022)
46.65*
40.05
50.07*
58.35
47.33*
38.96*
49.37
56.15*
GTRL. (TKDE 2023)
46.35*
40.95*
51.19*
60.18*
49.33*
40.15*
53.39*
60.74*
DHyper (TOIS 2024)
54,22*
42.16*
63.26*
75.38
52.11
41.04
60.03*
73.22*
DECRL
63.30
50.13
70.72
90.82
61.37
46,28
67.01
86.79
Improvement
16.75%
18.90%
11.79%
20.48 %
17.77% %
12.76%
11.63%
18.53 %
Table The performance of DECRL and the compared approaches on GDELT. "OOM" and "TLE" indicate of memory and single epoch exceeded 24 hours
Approach
MRR
Hits@1
Hits@3
Hits@
TTransE (WWW 2018)
8.62*
7.73*
11.03*
23.34*
HyTE (EMNLP P 2018)
10.63*
8.39*
14.23*
28.79*
RE-NET (EMNLP 2020)
17.55*
11.73*
18.14*
35.52*
Glean (KDD 2020)
15.60*
10.35*
17.61*
37.40*
TeMP (EMNLP 2020)
19.19*
11.07*
19.84*
40,52*
RE-GCN (SIGIR 2021)
20.84*
10.80*
21.09*
43.65*
DACHA (TKDD 2022)
21.91
11.27*
17.49*
47.13*
TiRGN (UCAAI 2022)
24.61*
13.78*
25,66*
49.02*
TITer (EMNLP 2021)
TLE
TLE
TLE
TLE
EvoExplore (KBS 2022)
18.53*
10.74*
19.45*
42,07*
GTRL (TKDE 2023)
22.44*
12.48
18.03*
50.82*
DHyper (TOIS 2024)
OOM
OOM
OOM
OOM
DECRI.
27.58
15.74
29.16
59.54
Improvement
12.07 %
14.22%
13.64%
17.16%
Table 4: The performance of DECRL and the compared approaches on WIKI and YAGO with MRR
Approach
WIKI
YAGO
TiRGN (ICAI 2022)
99.04
99,30
DHyper (TOIS 2024)
99.38
99.31
DECRL
99.67
99.56
Improvement
0.29%
0.25 %

---

Table 5: The performance of DECRL and the variants on ICEWS14
Method
MRR
Hits@1
Hits@3
Hits@ 10
DECRI -
wooaaigmmmnt
59.16 (-3.5)
43.92 (-4.8)
67.57 (-3.0) 92.31 (-0.7)
DOCRL-Woofuree
57.98 (-4.3)
41.90 (-6.8)
66.97 (-3.6)
92.00
DECRL-Wor woo-ICE
60.53 (-2.8)
46.33 (-2.4)
68.29 (-2.3)
(-1.9
DECRL- -woo-glohal-ggapll
58.67 (-3.9)
43.45 43.45
69.46 (-1.1) 91.07 (-2.0)
DECRL-WO-Ciempe
58.74 (-3.9)
44.43 (4.3)
65.33 (-5.2)
89.85 (-3.2)
DECRL
62.61
48.73
70.57
93.03
generally outshine those relying on DNNs, which in turn surpass approaches employing shallow encoders.
Since most approaches do not report event prediction results on WIKI and YAGO datasets, we compare DECRL with TiRGN and DHyper, both of which provide results these datasets, as shown in Table Furthermore, TiRGN and DHyper are the SOTA DNN-based approach and the SOTA derived structure-based approach, respectively. As a result, DECRL shows improvements of 0.29% and 0.25% over the runner-up, DHyper, on WIKI and YAGO datasets, respectively, underscoring its effectiveness. Detailed entity prediction results are presented in Appendix due to space limitation.
5.3 Ablation study
To dissect the contributions of each module within DECRL, we conduct the ablation study on ICEWS14. The detailed descriptions of variants are as follows:
DECRRL-wwoallgmmeent Removing unsupervised alignment mechanism.
DECRL-w/o-lusion Removing fusion operation between clusters across timestamps.
DECRL-w/O-ICE: Removing implicit correlation encoder, resulting in fully connected graph where all edges are assigned uniform weights.
DECRL-wW--lobbalgraaht Removing the guidance of the global graph on the assignment of different interaction intensities to different cluster pairs.
DECRL--WO--Gemppooll Removing the temporal smoothness loss term.
3 the MRR and Hits 1/3/10 results of DECRL and the variants on ICEWS14. From the ablation study, several conclusions be drawn: Firstly, the most substantial performance degradation is observed in the DECRL-w/O-fusion variant, which indicates the effectiveness of capturing the temporal evolution of high-order correlations among entities. Moreover, the perfor- mance degradation of the DECRL-w/l-llbbal-raap variant justifies the effectiveness of leveraging structural information from the global graph for guiding the assignment of different interaction intensities to different cluster pairs. Lastly, the performance degradation of DECRL-w/O-liggnme and DECRL-w/O-CE L/-ee-poo-- variants shows the importance of preserving the temporal smoothness of clusters over successive timestamps.
5.4
Case study
To demonstrate the efficacy of DECRL, we conducted case study with two variants: DECRL-W/o- alignment and DECRI--WOO-fusion both of which are elaborated in Sub-section [5.3] In Figure 2 we utilize t-SNE (31) for the visualization of entity representations on ICEWS14C. where red dots represent individual entities in the TKGs, with their groupings indicating entity clusters. Notably, the entities marked in Figure 2 i.c., China, Thailand, Vietnam, Laos, Malaysia, Philippines, and Cambodia, are countries in East Asia participating in the Belt and Road Initiative. "Middle" and "Final" denote the entity representations obtained after training at the penultimate epoch and the final epoch, respectively, for different variants.
By comparing the visualizations in the first row of sub-figures Figure 2 we observe that DECRL exhibits the best entity clustering phenomena during the mid-training phase. By comparing the visualizations in the second row of sub-figures in Figure 2 Figure 2 (Final DECRL) showcases pronounced clustering phenomena, and inter-cluster distances indicate significant differences in

---

Viean
Fislegnne Carbodie
-10
-30
-20
303
9
343
.108
(a) Middle DECRL
(b) Middle DECRL-w/O-alieen wo-alignment
(c) Middle DECRL-W/-fusion
143
200
163
Cunbedoo
Nand
eve 26%
103
-153
300
-20
302
-400
-301
-108
(d) Final DECRL
(e) Final DECRL-w/o-algameen
(I) Final DECRL-WoO-fusion
Figure 2: The visualization of entity representations ICEWS14C.
clusters, i.e., international political organizations. Figure 2e (Final DECRL-Woo-lligmment demon- strates moderate clustering phenomena among countries, but inter-cluster distances are significant enough. This observation implies that the lack of precise one-to-one alignment may lead to proximity between different clusters. Figure 2i (Final DECRL-w/O-fusion. which only models high-order correlations without temporal evolution) exhibits increased inter-cluster distances, with less distinct clustering phenomena. The comparison between Figure 2d (Final DECRL) and Figure 2 (Final HECRL-w/O-fusion shows that capturing temporal evolution leads to better entity representations, as indicated by larger inter-cluster distances and tighter intra-cluster groupings. Comparing the first and third rows of Figure 2 further highlights the training progression, where modeling temporal evolution gradually enhances cluster separation and tightens the grouping of entities within clusters. This demonstrates that the capability of DECRL to model the temporal evolution of high-order correlations significantly enhances its ability to capture more nuanced cluster representations. Additional case study details provided in Appendix A.7
6 Conclusions and limitations
In this paper, a Deep Evolutionary Clustering jointed temporal knowledge graph Representation Learning approach (DECRL) is proposed for event prediction in TKGs, which jointly optimizes TKG representation learning with evolutionary clustering to capture the temporal evolution of high-order correlations. Comprehensive experiments are conducted on seven real-world datasets, including the comparison with baselines, ablation study, hyper-parameter sensitivity analysis, and case studies, which demonstrate the superior performance of DECRL. However, this study overlooks the continuous temporal evolution of diverse high-order correlations. Currently. the implicit correlation encoder assumes uniform correlations across all clusters, which may not reflect the complexity of real- world interactions between political organizations. In future work, we will address this issue by developing a multi relation-aware inter-cluster correlation encoder. Furthermore, DECRL currently models the temporal evolution of high-order correlations by considering only the previous and current timestamps. Future work will focus on explicitly modeling the continuous influence of high-order correlations from different past timestamps on the current timestamp.
10

---

Acknowledgement
This work was supported by the Science Foundation of Donghai Laboratory (Grant No. DH- 2022ZY0013).
References
[1] Luyi Bai, Mingcheng Zhang, Han Zhang, and Heng Zhang. FTMF: Few-shot temporal knowl- edge graph completion based on meta-optimization and fault-tolerant mechanism. World Wide Web, 26(3):1243-122 2023.
[2] James Bergstra, Brent Komer, Chris Eliasmith, Dan Yamins, and David Cox. Hyperopt: A python library for model selection and hyperparameter optimization. Computational Science & Discovery, 8(1):014008, 2015.
[3] Jiaao Chen, Jianshu Chen, and Zhou Yu. Incorporating structured commonsense knowledge in story completion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6244-6251, 2019.
[4] Ling Chen, Xing Tang, Weiqi Chen, Yuntao Qian, Yansheng Li, and Yongjun Zhang. DACHA: dual graph convolution based temporal knowledge graph representation learning method using historical relation. ACM Transactions Knowledge Discovery from Data, 16(3):1-18, 2021.
[5] Shib Sankar Dasgupta, Swayambhu Nath Ray, and Partha Talukdar. HyTE: Hyperplane- based temporally aware knowledge graph embedding. In Proceedings of the Conference Empirical Methods Natural Language Processing, pages 2001-2011, 2018.
[6] Songgaojun Deng, Huzefa Rangwala, and Yue Ning. Dynamic knowledge graph based multi- event forecasting. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery Data Mining, pages 1585-1595, 2020.
[7] Francesco Folino and Clara Pizzuti. An evolutionary multiobjective approach for community discovery in dynamic networks. IEEE Transactions on Knowledge and Data Engineering, 26(8):1838 1852, 2013.
[8] Zhen Han, Peng Chen, Yunpu Ma, and Volker Tresp. Explainable subgraph reasoning for forecasting on temporal knowledge graphs. In Proceedings of the International Conference on Learning Representatiinss pages 1-24, 2020.
[9] Woojeong Jin, Meng Qu, Xisen Jin, and Xiang Ren. Recurrent event network: Autoregressive structure inference over temporal knowledge graphs. In Proceedings of the Conference on Empirical Methody in Natural Language Processing, pages 6669-6683. 2020.
[10] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6998, 2014.
[11] Harold Kuhn. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2):83-97, 1955.
[12] Julien Leblay and Melisachew Wudage Chekol. Deriving validity time in knowledge graph. In Proceedings of the Web Conference, pages 1771-1776, 2018.
[13] Kalev Leetaru and Philip A Schrodt. GDELT: Global data events, location, and tone, 2012. In Proceedings of ISA Annual Convention, volume 2, pages 1-49, 2013.
[14] Yujia Li. Shiliang Sun, and Jing Zhao. TiRGN: Time-guided recurrent graph network with local-global historical patterns for temporal knowledge graph reasoning. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 2152-2158, 2022.
[15] Zixuan Li, Xiaolong Jin, Saiping Guan, Wei Li, Jiafeng Guo, Yuanzhuo Wang, and Xueqi Cheng. Search from history and reason for future: Two-stage reasoning on temporal knowledge graphs. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 4732-4743, 2021.

---

[16] Zixuan Li, Xiaolong Jin, Wei Li, Saiping Guan, Jiafeng Guo, Huawei Shen, Yuanzhuo Wang, and Xueqi Cheng. Temporal knowledge graph reasoning based on evolutional representation learning. In P Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 408 4088-17,40 2021.
[17] Kangzheng Liu, Feng Zhao, Hongxu Chen, Yicong Li, Guandong Xu, and Hai Jin. Da-Net: Distributed attention network for temporal knowledge graph reasoning. In Proceedings the ACM International Conference on Information & Knowledge Management, pages 1289-1298,
2022.
[18] Yushan Liu, Yunpu Ma, Marcel Hildebrandt, Mitchell Joblin, and Volker Tresp. TLogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 4120-4127, 2022.
[19] Zhenghao Liu, Chenyan Xiong, Maosong Sun, and Zhiyuan Liu. Entity-duet neural ranking: Understanding the role of knowledge graph semantics in neural information retrieval. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 2395-2405, 2018.
[20] Lijia Ma, Yutao Zhang, Jianqiang Li, Qiuzhen Lin, Qing Bao, Shanfeng Wang. and Maoguo Gong. Community-awart dynamic network embedding by using deep autoencoder. Information Sciences, 519:22-42. 2020.
[21] Xiaoke Ma and Di Dong. Evolutionary nonnegative matrix factorization algorithms for commu- nity detection in dynamic networks. IEEE Transactions on Knowledge and Data Engineering, 20455:00451100 1058, 2017.
[22] Farzaneh Mahdisoltani, Joanna Biega, and Fabian M Suchanek. Yago3: A knowledge base from multilingual wikipedias. In Proceedings of Conference on Innovative Data Systems Research, 2013.
[23] Yue Pu, Wenbin Yao, and Xiaoyong Li. EM-IFCM: Fuzzy c-means clustering algorithm based on edge modification for imbalanced data. Information Sciences, 659:120029, 2024.
[24] Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In Proceedings of the European Semantic Web Conference, pages 593-607, 2018.
[25] Chao Shang, Yun Tang, Jing Huang, Jinbo Bi, Xiaodong He, and Bowen Zhou. End-to-end structure-awart convolutional networks for knowledge base completion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3060-3067, 2019.
[26] Zongjiang Shang and Ling Chen. MSHyper: Multi-scale hypergraph transformer for long- long-range time series forecasting. arXiv preprini arXiv:2401.098 2922611110,,,,,. 2024.
[27] Haohai Sun, Jialun Zhong, Yunpu Ma, Zhen Han, and Kun He. Timelraveler: Reinforcement learning for temporal knowledge graph forecasting. arXiv preprint 10X.v:2119.041000 2021.
[28) Xing Tang and Ling Chen. GTRL: An entity group-aware temporal knowledge graph represen- tation learning method. IEEE Transactions Knowledge and Data Engineering, pages 1-16, 2023.
[29] Xing Tang, Ling Chen, Hongyu Shi, and Dandan Lyu. DHyper: A recurrent dual hypergraph neural network for event prediction in temporal knowledge graphs. ACM Transactions on Information Systems, pages 1-25, 2024.
[30] Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. Know-Evolve: Deep temporal reasoning for dynamic knowledge graphs. In Proceedings of International Conference on Machine Learning, pages 3462-3471, 2017.
[31] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE, Journal of Machine Learning Research, 0(11):2579-2605 2021.
12

---

[32] Xiang Wang, Xiangnan He, Yixin Cao. Meng Liu, and Tat-Seng Chua. KGAT: Knowledge graph attention network for recommendation. In Proceedings of the ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pages 950-958, 2019.
[33] Binqing Wu, Weiqi Chen, Wengwei Wang, Bingqing Peng. Liang Sun, and Ling Chen. Weath- erGNN: Exploiting meteo- and ppatial-dependenciss for local numerical weather prediction bias corsecoorection. In Proceedings of the International Joint Conference Artificial Intelligence, page 2433-2441, 2024.
[34] Jiapeng Wu, Meng Cao, Jackie Chi Kit Cheung, and William L Hamilton. TeMP: Temporal message passing for temporal knowledge graph completion. arXiv preprini urXiv:2010.03526, 2020.
[35] Yi Xu, Junjie Ou, Hui Xu, and Luoyi Fu. Temporal knowledge graph reasoning with histor- ical contrastive learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 4765-4773, 2023.
[36] Min Yang, Xiaoliang Chen, Baiyang Chen, Peng Lu, and Yajun Du. DNETC: Dynamic network embedding preserving both triadic closure evolution and community structures. Knowledge and Information Systems,
2023.
[37] Yuhang Yao and Carlee Joe-Wong, Interpretable clustering dynamic graphs with recurrent graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages (6608-4614 2021.
[38] Jingyi You, Chenlong Hu, Hidetaka Kamigaito, Kotaro Funakoshi, and Manabu Okumura. Robust dynamic clustering for temporal networks. In Proceedings of the ACM International Conference on Information & Knowledge Management, pages 2424-2433, 2021.
[39] Jiasheng Zhang, Shuang Liang, Yongpan Sheng, and Jie Shao. Temporal knowledge graph rep- resentation learning with local and global evolutions. Knowledge- Based Systems, 251: 2022.
13

---

Table 6: The statistics of datasets
Dataset
#Entity
#Relation
#Training set
#Validation set
#Test set
#Time interval
ICEWS14
7128
230
74,845
8,514
7,371
24 hours
ICEWS14C
205
171
35,665
7,369
7,068
24 hours
ICEWS18
23,033
256
373,018
45,995
49,545
24 hours
ICEWS18C
164
34,497
4,412
4,661
hours
GDELT
7,691
240
1,734,399
238,765
305,241
15 mins
WIKI
12.354
24
539,286
67,538
63,110
1 year
YAGO
10,623
10
161,540
19,523
20,026
year
Table 7: The final choices of hyper-parameter values
Syper-parameter
ICEWS14
ICEWS14C
ICEWS18
ICEWS18C
GDELT
WIKI
YAGO
N
14
6
18
16
18
16
NDECRL luyer
2
Misstarial window
9
7
2
1
0.2
0.2
0.2
0.2
0.2
0.2
0.2
A
Appendix
A.1
Complexity analysis
The time complexity of the relation- aware GCN is O(N,N,D²²) where N, and N, are the numbers of entities and relations, respectively. 1 the dimension of representations. The time complexity of the evolutionary clustering module is (D(N,N,DD) N + D), where N. is the number of clusters. The time complexity
the cluster graph message passing module is O(N D2). The time complexity of the time residual gate is O(D²). For the attentive temporal encoder, the time complexity is O D), where T is the length of the history. For the event prediction module, the time complexity O()). The total complexity of DECRL is O2 + Nº + N D2 D).).
A.2 Statistics of datasets
All datasets are split into training (80%), validation (10%), and test (10%) sets following 16 The statistics of these datasets are summarized in T
A.3 Experimental settings
DECRL is implemented in Python using PyTorch and trained one NVIDIA RTX 3080 GPU with 10GB memory. We leverage the Neural Network Intelligence (NNI) toolki to automatically identify the optimal hyper-parameter values. The search space for the number of clusters No, the number of DECRL layers NDECRL layer the length of historical windows Naistorical window and the value of range from to 20 with the step of to 5 with the step of 1, 14 with the step of and 0.1 to 0.5 with the step of 0. respectively. The final hyper-paramette values are presented in Table 7 For NNI configurations, the maximum number of trials is the Tree-structured Parzen Estimator
layer
30. and the optimization algorithm used
We utilize the Adam [10 optimizer with initial learning of 0.01. The batch size is set to 16. The representation dimension is set to 200. The hidden sizes for the time residual gate and the attentive temporal encoder are both independent runs. The evaluation metrics used in this paper include Mean Reciprocal Rank (MRR) and Hits 1/3/10, which represent the proportion of correct predictions ranked within the top and 10 positions, respectively, all expressed as percentages. Higher Hits @ k and MRR scores indicate better performance.
200. The results reported are the averages across five
Puaps://gihubbcom//icccsotttttte
14

---

A.4
Description of baselines
Shallow encoder-based approaches:
TTransE
12
extends the TransE by incorporating timestamps as corresponding representa-
tions.
HyTE
5 models timestamps corresponding hyperplanes.
DNN-based approaches:
RE-NET 9 leverages GCNs to model the influence of neighbor entities and uses RNNs to capture temporal dependencies among events.
Glean 6 employs CompGCN to model the influence of neighbor entities and utilizes GRUs to capture temporal dependencies among representations.
TeMP 34 utilizes relation-aware GCN to model the influence of neighbor entities and employs frequency-based gating GRU to capture temporal dependencies among inactive events.
uses relation-aware GCN model the influence of neighbor entities and employs an autoregressive GRU to capture temporal dependencies among events.
RE-GCN [66
DACHA introduces dual graph convolution network to obtain entity representations and uses self-attentive encoder to model temporal dependencies among relations.
TiRGN 14] is the SOTA approach, using a multi-relational GCN to capture graph structure information and double recurrent mechanism model temporal dependencies.
Derived structure-based approaches:
TITer [27] incorporates temporal agent-based reinforcement learning to search paths and obtain entity representations via the inductive mean.
EvoExplore
39
establishes dynamic communities to model the influence of neighbor
entities.
uses entity group modeling to capture the influence of distant and unreachable entities and employs GRUs to model temporal dependencies among representations.
GTRL 28
DHyper [29] is the SOTA approach, which utilizes hypergraph neural networks to model high-order correlations among entities and among relations.
A.5 Hyper-parameter sensitivity analysis
In this sub-section, we study the hyper- parameeer sensitivity of DECRL on ICEWSI8C, including the length of historical windows Whistorical window, the number of clusters Nç. the number of DECRL layers NDECRL layer and the value A. We show the performance changes by varying the syper-parameet values in Figure
We present the impact of Naistorical window in Figure 3a The results indicate that Niittorica window increases, the performance of DECRL gradually improves, peaking at the window length of 10. Beyond this point, performance declines rapidly, likely due to the inclusion of excessive irrelevant information in longer historical windows, which negatively impacts the performance of DECRI..
In Figure 3b
present the effect of Ng. The performance of DECRL remains relatively stable across most metrics as N. increases. Hits shows a slight initial increase until N, and then stabilizes. MRR, Hits @3, and Hits @ 10 remain consistent, indicating that the performance of DECRL not substantially affected by variations of No.
We show the impact of NDECRL layer and À in Figures 3e 3d respectively. In Figure 3c performance metrics reach their peak when NDECRI, layer 2. Beyond this point, the performance DECRL declines, likely due to increased model complexity, which raises the risk of over-fitting. Similarly, in Figure 33 the performance metrics peak at À = 0.2, which suggests the need set the appropriate trade-off between event prediction loss and temporal smoothness loss.
15

---

NBR
11::31
HBS38
50
HS530
Reauts Metrie
50-
(a) Hyper-paramee pperrpparmmttr N Vistarical window on ICEWSI8C
(b) Hyper-parament on ICEWS18C
Hitug3
1ibs010
Resurto Mestes
Niau ol Moemee
100
011
0.2
0.3
0.4
0.
(c) Elyper-paramete NDEERL byer ICEWS18C
(d) Hyper-parameter À on ICEWS18C
Figure 3: Results of hyper-parameters changes of DECRL on ICEWS18C.
Table 8: The entity prediction performance of DECRL and the compared approaches on GDELT. "OOM" and "TLE" indicate out of memory and a single epoch exceeded 24 hours. indicates that DECRL is statistically superior to the compared approaches according to pairwise t-test at a 95% significance level. The best results are bold and second best results are underlined
Approach
MRR
Hits@1
Hits@3
Hits@10 10
TTransE (WWW 2018)
10.04
6.31*
19.23
28.08*
HyTE (EMNLP 2018)
18.41
5.16*
21.75*
30.47*
RE-NET (EMNLP 2020)
20.78*
15.37*
23.16*
35,59*
Glean (KDD 2020)
21.14
19.29s
27.28
36.37
TeMP (EMNLP 2020)
36.23*
31.37*
39.47*
48.45*
RE-GCN (SIGIR 2021)
21.19*
19.53*
22.92*
35,71
DACHA (TKDD 2022)
31.05
28.11
40.12
48.
TIRGN (ICAI 2022)
23.64*
20.95
26.88
40.26*
TITer (EMNLP 2021)
TLE
TLE
TLE
TLE
EvoExplore (KBS 2022)
23.61*
17.44*
32.37
41.47*
GTRL (TKDE 2023)
39.23
33.37
42.45*
51.49*
DHyper (TOIS 2024)
0OM
OOM
OOM
OOM
DECRI.
37.24
28.60
46.69
66.47
Improvement
-5.07% %
-14.29 %
9.99%
29.09 %
16

---

Table 9: Top relations predicted by TiRGN, DHyper, and DECRL
Test sample
TiRGN
DHyper
DECRL
Engage diplomatic cooperation
Criticize or denounce
Accuse
(Russin,
7.
Make empathetic commmnnt
Sign formal agreement
Criticize denounce
Ukraine,
Apologize
Reduce or break diplomatic relations
Use conventional military force
2014/12/100
Appeal for ecomomic aid Host visit
Engage in negotiation
Appeal for economic and
Praise endorse
Impose emhargo, boyentt, sanctions
Make statement
Impose embargo boycott, sanctions
Disapprove
(the States,
United
Make visit
Sign formal agrecment
Sign formal agreement
Sign formal agreement
Imipose embargo, hoycott, sanctions
Russia,
2014/12/17)
Express intent to meet or negotiate Impose embargo, boycott, sanctions
Engage in negotiation
Express intent yield
conventional military force
Use unconventional violence
A.6 Entity prediction performance
We have conducted additional experiments to evaluate the performance of DECRL on the future entity prediction task, as shown in Table Although DECRL does not achieve the SOTA performance in terms of MRR and Hits @1 metrics, it achieves the best results in Hits @ffs@3 and Hits @ 10 metrics. It is important to note that DECRL not specifically designed for entity prediction tasks. Nevertheless, these results demonstrate the effectiveness and robustness of DECRI,, particularly in capturing broader range relevant entities.
A.7 Case study
preseesss the top 5 relations predicted by the SOTA DNN-based approach TiRGN, the SOTA derived structure-based approach DHyper, and our proposed approach DECRL for two test samples on ICEWS144 The test samples pertain to the conflict between Russia and Ukraine that began in February 2014. During this period, Russia deployed military forces to the Crimean region of Ukraine, which resulted in widespread condemnations and sanctions from several countries, including the United States and various European nations. Correct predictions are underlined in Table Compared to TiRGN and DHyper, DECRL predicts more correct relations and ranks the correct predictions higher. The results indicate that by modeling the temporal evolution the high-order correlations among entities, DECRL achieve more accurate prediction results.
A.8 Societal impacts
In this paper, Deep Evolutionary Clustering jointed temporal knowledge graph Representation Learning approach (DECRL) is proposed for event prediction in temporal knowledge graphs, which offers more accurate and credible results. We summarize the positive and possible negative societal impacts follows:
Positive societal impacts:
Optimizing social governance: Temporal knowledge graph event prediction can help goverments and relevant institutions foresee potential future events, enabling them to take preemptive measures and optimize social governance.
Supporting business decisions: Companies can use the prediction results for market analysis and business decisions, enhancing their competitiveness.
Enhancing public safety: : By predicting potential threats and dangerous events, relevant departments can allocate resources in advance to ensure public safety.
Advancing academic research: This research can promote progress in the fields of temporal knowledge graphs and event prediction, providing new directions and methods for academic studies.
Negative societal impacts:
Misuse risks: Accurate predictive technology might be exploited by malicious actors to forecast and manipulate events.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope (see Abstract and Introduction).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper discusses the limitations of the work (see Conclusions and limita- tions).
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiiatinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
18

---

Answer: [Yes]
Justification: The paper provides the full set of assumptions and a complete (and correct) proof (see Section 4).
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The paper fully discloses all the information needed to reproduce the main experimental results of the paper (see Appendix A.3).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
19

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We upload source code and datasets on Anonymous GitHub (see Appendix
A.3)
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The paper specifies all the training and test details (e.g., data splits, hyperpa- rameters, how they were chosen, type of optimizer, etc.) necessary to understand the results (see Appendices A.2 and A.3).
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper reports appropriate information about the statistical significance of the experiments (see Sub-section 5.2)
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
20

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides sufficient information the computer resources and time complexity analysis (see Appendices A. and A.3).
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:////eaaaaasscccc////i/ccccc inn
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses both potential positive socictal impacts and negative societal impacts of the work performed (see Appendices A.8)
Guidelines:
21

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The datasets used in this work are public and out of risk. Our approach poses no such risks, involving pretrained language models or image generators.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The creators or original owners of assets (c.g., code, data, models), used in the paper, are properly credited, and the license and terms of use are explicitly mentioned and properly respected.
Justification:
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
22

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thcode com comddatasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The instruction of dataset/code/moodl is provided on the project homepage at the anonymous Github. The URL of the anonymous repository please Abstract.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
23

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "XkmU1eqyNx", "reviewer": "Reviewer_RZGp", "summary": "The paper proposed a deep evolutionary clustering method for TKGE to capture the temporal evolution of high-order correlation in TKGs. A cluster-aware unsupervised alignment mechanism is introduced to ensure the precise one-to-one alignment of soft overlapping clusters\nacross timestamps. Extensive experiments on four real-world datasets demonstrate the remarkable improvement of the proposed method compared to other baselines.", "questions": "1. Will the proposed method still outperform other baselines by large margins on Wikidata and YAGO datasets? The type of time information in these two datasets is different from ICEWS datasets.", "limitations": "The proposed method mainly focuses on improving the accuracy of link prediction for TKGs, while other aspects such as efficiency and transparency are not discussed.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The experimental results are remarkable for the improvement of the ICEWS datasets.\n2. The paper is well-organized and clearly presented.", "weaknesses": "1. The paper proposed to capture the temporal evolution of high-order correlation, while no intuitional case study is provided.\n2. Though extensive experiments are conducted, two other main TKGE datasets are omitted, namely Wikidata and YAGO.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s meaningful comments and insightful questions.\n\n>Q1: Lack of intuitive case study demonstrating temporal evolution of high-order correlations\n\n**We would like to draw attention to the comparison between Figure 2d (Final DECRL) and Figure 2f (Final DECRL-w/o-fusion, which only models high-order correlations without capturing their temporal evolution) in the manuscript.** This comparison clearly illustrates that capturing the temporal evolution of high-order correlations leads to superior entity representations, as evidenced by the larger inter-cluster distances and tighter intra-cluster entity groupings. Furthermore, by comparing the first and third columns of Figure 2 in the manuscript, we can observe the progression of training. This comparison demonstrates that capturing the temporal evolution of high-order correlations gradually increases the separation between clusters while simultaneously tightening the grouping of entities within clusters. This observation reveals that the capability of DECRL to model the temporal evolution of high-order correlation significantly enhances its ability to capture more nuanced cluster representations.\n\n**In addition, we also incorporate the case study results from DHyper to further substantiate the effectiveness of our approach, as shown in Figure 2 of the rebuttal PDF (which is attached in the global rebuttal in the beginning).**\n\n\n>Q2: Omission of Wikidata and YAGO datasets in the experiments\n\nFirstly, there is a fundamental difference in timestamp types between ICEWS and the Wikidata and YAGO datasets. ICEWS uses single-point timestamps, while Wikidata and YAGO use time intervals for events. Our research focuses on modeling the temporal evolution of high-order correlations between entities. Datasets with single-point timestamps, e.g., ICEWS, show more frequent temporal changes and higher temporal complexity, making them more suitable for capturing the temporal evolution of high-order correlations. This is why we initially focused on the ICEWS dataset.\n\nMoreover, the SOTA models have already achieved very high performance on Wikidata and YAGO datasets, with MRR scores exceeding 99\\% for relation prediction. Given the limited room for improvement, we initially excluded these datasets.\n\nHowever, we acknowledge the importance of comprehensive evaluation. **In light of your valuable feedback, we have conducted additional experiments on the Wikidata, YAGO, and GDELT datasets, as shown in Tables 1 and 3 of the attached rebuttal PDF.** We are pleased to report that our approach has achieved the SOTA relation prediction performance across all these datasets, demonstrating the effectiveness and robustness of our approach.\n\n\n>Q3: Lack of discussion on efficiency and transparency aspects of the proposed method\n\n**We have indeed considered the efficiency of our approach and have calculated its time complexity, which is detailed in Appendix A.1 of our manuscript.** Our approach employs evolutionary clustering to capture the temporal evolution of high-order correlations, requiring fewer parameters and lower memory resources compared to approaches using learnable structures like hypergraphs. This design choice significantly contributes to the overall efficiency of our approach.\n\n**To further demonstrate the efficiency of our approach, we have conducted additional experiments comparing the training time of our approach with DHyper.** The results of these experiments are presented in Figure 1 of the attached rebuttal PDF, clearly illustrating the computational advantages of our approach.\n\nRegarding transparency, we acknowledge that this aspect is not thoroughly addressed. We appreciate the reviewer bringing this to our attention. **We will include a comprehensive discussion on the transparency of our approach in the limitations section of the camera-ready version.** This addition will highlight areas for future improvement.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "O5a4vO71je", "reviewer": "Reviewer_S3KV", "summary": "This paper studies the temporal knowledge graph representation learning task and proposes a temporal evolution-aware framework DECRL.\n\nBy assigning different entities to distinct clusters at each timestamp and modeling the evolution and shifts of these clusters, cluster-aware information is explicitly incorporated into both entity and relation embeddings. This enables better temporal intelligence for making precise predictions.\n\nExtensive experiments demonstrate promising results on several benchmarks.", "questions": "Please see weaknesses", "limitations": "NA", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* This paper is well-written and easy to follow.\n* Technical details are well presented and, to some extent, clearly explained.\n* The experiments are extensive, covering various benchmarks, comparing diverse baselines, and demonstrating effectiveness through both qualitative and quantitative analyses.", "weaknesses": "* The motivation for introducing a cluster graph at each timestamp to capture temporal shift information at the clustering scale remains unclear. The authors should further elaborate on this overall motivation. Additionally, the authors mentioned that “some researchers have leveraged derived structures, e.g., communities, entity groups, and hypergraphs, to model high-order correlations among …” (Lines 29-30). In my understanding, the modeling of communities and groups is similar to this paper’s clustering, and hypergraphs can be regarded as another approach for group modeling since a hyperedge connects different nodes. The differences in motivation and technique between DECRL and these methods should be briefly discussed.\n* The methodology section of DECRL incorporates many minor techniques without corresponding ablation studies to demonstrate their effectiveness or detailed explanations of the rationale. For example, the effectiveness of temporal attentive pooling (Section 4.5) lacks ablation study evidence. Besides, detailed operations for each variant in the ablation study should be provided.\n* For Figure 2 in the case study, additional textual explanations are needed, such as clarifying what the red dots represent. Additionally, the authors should consider including baseline methods in the figure, rather than only showing DECRL and its variants.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s meaningful comments and insightful questions.\n\n>Q1: Unclear motivation for introducing a cluster graph at each timestamp to capture temporal shift information at the clustering scale\n\nWe consider the complex dynamics of international alliances and conflicts, which exhibit high-order correlations that evolve over time. **Countries rarely interact in isolation**, for instance, the relationship between the USA and Russia affects not only these two countries but also influences their respective allies and trade partners. We use clustering to capture these complex high-order correlations.\n\n**These high-order correlations evolve smoothly over time.** The Cold War, for example, did not end abruptly but gradually thawed through a series of events. When constructing our cluster graph, we consider how past high-order correlations (previous clusters) influence current ones, modeling this smooth temporal evolution.\n\n**Furthermore, within a single timestamp, different clusters of entities can affect each other.** For example, tensions between NATO countries and Russia might influence relations between OPEC members and Western nations. We model these intra-timestamp influences through an implicit correlation encoder in our proposed approach.\n\nBy capturing these aspects, our approach can represent the nuanced, temporal evolving nature of high-order correlations more accurately.\n\n\n>Q2: Lack of discussion on the differences in motivation and technique between DECRL and other methods using derived structures\n\nWhile entity groups, hypergraphs, and clustering can all model high-order correlations, our approach offers unique advantages:\n\nFirstly, methods using entity groups and hypergraphs require learning entity assignment mappers at each timestamp, a process that is complex to update and maintain, resulting in significant computational overhead. **Our clustering-based approach, however, adapts more flexibly to dynamic data changes and is lightweight, facilitating easier integration with other techniques.**\n\nSecondly, we use a fuzzy clustering algorithm with a fuzzy smoothing hyperparameter that controls node membership distribution, preventing clusters with very few nodes. **This approach allows for effective cluster construction even for nodes with limited interactions.** Using entity graphs or hypergraphs to achieve a similar advantage would be much more computationally expensive and resource-intensive.\n\n**In addition, our experiments demonstrate the effectiveness of our approach compared to methods using entity groups and hypergraphs.** For example, the DECRL-w/o-fusion variant, which uses only clustering for representation learning, achieves MRR, Hits@1, Hits@3, and Hits@10 scores of 57.98, 41.90, 66.97, and 92.00, respectively. These results outperform the hypergraph-based method, i.e., DHyper, which scores 56.15, 43.76, 65.46, and 85.89 on the same metrics. This superior performance can be partly attributed to fuzzy clustering’s ability to prevent the formation of extremely small clusters.\n\n\n>Q3: Lack of ablation studies for minor techniques and insufficient explanation of variant operations in existing ablation studies\n\nThank you for raising this concern. We recognize the oversight in not including an ablation study for the temporal attentive pooling component. To address this, **we have conducted an additional experiment to demonstrate its effectiveness, as shown in Table 4 of the rebuttal PDF (which is attached in the global rebuttal in the beginning).** These results clearly illustrate the impact of temporal attentive pooling on our approach’s performance.\n\nIn addition, in light of your valuable feedback, we acknowledge that our initial description of the ablation study variants may have been insufficient. To address this, **we will prepare more detailed explanations for each variant in our camera-ready version.**\n\n\n>Q4: Need for additional explanations in Figure 2 and inclusion of baseline methods in the case study\n\n**We will enhance the explanations of Figure 2 in the manuscript**, clarifying that red dots represent individual entities in the temporal knowledge graph, with their groupings indicating entity clusters.\n\n**Furthermore, we would like to highlight key observations from the Figure 2.** The comparison between Figure 2d (Final DECRL) and Figure 2f (Final DECRL-w/o-fusion, which only models high-order correlations without capturing their temporal evolution) in the manuscript clearly illustrates that capturing the temporal evolution of high-order correlations leads to superior entity representations, as evidenced by the larger inter-cluster distances and tighter intra-cluster entity groupings. In addition, by comparing the first and third columns of Figure 2 in the manuscript, we can observe the progression of training. This comparison demonstrates that capturing the temporal evolution of high-order correlations gradually increases the separation between clusters while simultaneously tightening the grouping of entities within clusters. This observation reveals that the capability of DECRL to model the temporal evolution of high-order correlation significantly enhances its ability to capture more nuanced cluster representations.\n\nWe also acknowledge the importance of comparing our approach with baselines in the case study. **We have conducted additional visualizations for DHyper, the second-best baseline model, in Figure 2 of the attached rebuttal PDF.**\n\nIn light of your valuable feedback, we will refine all the content above and incorporate it into the camera-ready version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9jsa2VzjHl", "reviewer": "Reviewer_C7a5", "summary": "The paper addresses Temporal Knowledge Graph (TKG) representation learning, which aims to embed temporally evolving entities and relations into a continuous low-dimensional vector space. Existing methods struggle to capture the temporal evolution of high-order correlations in TKGs. The authors propose a novel approach called Deep Evolutionary Clustering jointed temporal knowledge graph Representation Learning (DECRL). DECRL is the first to integrate deep evolutionary clustering with TKG representation learning to capture the temporal evolution of high-order correlations.", "questions": "1 When clustering nodes using fuzzy clustering, have the authors considered incorporating domain knowledge for node classification? Additionally, have they compared their approach with other node classification methods?\n2 Some nodes may not have clear relationships with other nodes. Will clustering them together affect the behavior prediction of these nodes?", "limitations": "Please refer to the weaknesses.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1 The author clearly describes the motivation for the paper and the methods used. \n2 The experimental results demonstrate that DECRL achieves state-of-the-art (SOTA) performance.", "weaknesses": "1 Most event prediction models are often tested on ICEWS05-15 and GDELT datasets. The author does not give experimental results on these two datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s meaningful comments and insightful questions.\n\n>Q1: Absence of experimental results on ICEWS05-15 and GDELT datasets\n\nICEWS05-15 dataset shares the same source as ICEWS and has a similar scale to ICEWS18, which we included in our initial experiments, so we do not test our approach on ICEWS05-15 dataset. In addition, we initially excluded GDELT dataset due to its known issues with false positives [1] and a high proportion of abstract conceptual entities (e.g., POLICE and GOVERNMENT) [2], as we cannot predict a government’s activities without knowing which country it belongs to.\n\nHowever, we acknowledge that including a wider range of datasets would provide a more comprehensive evaluation of our approach’s performance and robustness. **In light of your valuable feedback, we have conducted additional experiments on the GDELT dataset, as well as on the WIKI and YAGO datasets.** The results of these experiments are presented in Tables 1 and 3 of the attached rebuttal PDF (which is attached in the global rebuttal in the beginning). We are pleased to report that our approach has achieved the SOTA relation prediction performance across all these datasets, demonstrating the effectiveness and robustness of our approach.\n\n[1] Ward, M. D., Beger, A., Cutler, J., Dickenson, M., Dorff, C., \\& Radford, B. Comparing GDELT and ICEWS event data. *Analysis*, 21(1): 267-297, 2013.\n\n[2] Li, Z., Jin, X., Li, W., Guan, S., Guo, J., Shen, H., et al. Temporal knowledge graph reasoning based on evolutional representation learning. In *Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval*, pages 408-417, 2021.\n\n\n>Q2: Consideration of domain knowledge in node classification and comparison with other node classification methods\n\nWe did not incorporate domain knowledge into our fuzzy clustering process for two primary reasons: First, the publicly available datasets we used do not provide corresponding domain knowledge information. Second, since additional domain knowledge are typically not employed in other temporal knowledge graph research, doing so could lead to unfair comparisons with existing methods.\n\n**In temporal knowledge graphs, there are no explicit ground truth entity categories.** Given this lack of predefined classes, we chose an unsupervised clustering algorithm to model high-order correlations among entities, allowing us to discover latent structures without relying on pre-existing labels.\n\nHowever, we acknowledge the importance of comparing our approach with alternative clustering techniques. To address this, **we have conducted additional experiments using different clustering algorithms as variants of our approach.** The results of these experiments are shown in Table 4 of the attached rebuttal PDF, providing a more comprehensive evaluation of our approach’s effectiveness compared to other potential clustering strategies.\n\n\n>Q3: Potential impact of clustering nodes with unclear relationships on behavior prediction\n\nFirstly, in our temporal knowledge graph datasets, nodes cannot exist in complete isolation as the data is structured around events, ensuring each node participates in at least one event and thus has a relationship with at least one other node.\n\nWe acknowledge that some nodes may have fewer interactions than others. Our approach addresses this variation effectively through a fuzzy clustering algorithm, which allows nodes to belong to multiple clusters with varying degrees of membership. **The fuzzy smoothing hyperparameter controls node membership distribution across clusters, preventing scenarios where clusters contain very few nodes.** This method enables effective cluster construction even for less frequently interacting nodes, allowing them to have partial memberships in multiple clusters and reflecting their potentially ambiguous relationships.\n\nTo further address this concern, in our camera-ready version, we will group nodes based on their interaction frequency in the training set and analyze relation prediction performance across these different node groups.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j9aiwiZttz", "reviewer": "Reviewer_fmj8", "summary": "This paper proposed an interesting model to integrate high-order clusters into the TKG representation learning. A cluster-aware unsupervised alignment mechanism is introduced to ensure the alignment of soft overlapping clusters across timestamps. An implicit correlation encoder is also proposed to capture latent correlations between clusters. Experimental results show the effectiveness of the proposed model.", "questions": "See weakness", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Integrating high-order structure in TKG representation learning is interesting.\n2. The proposed model seems technically reasonable.", "weaknesses": "1. Entity graph and cluster graph are two important concepts for understanding the proposed method. However, they are never mentioned in the introduction and their definitions in Section 3 are very brief, making the intuition behind the proposed method unclear to me.\n2. The authors use a relatively simple task future relation prediction to evaluate the quality of the learned TKG representations. However, the future entity prediction (i.e., [s, r, ?, t]) is more challenging due to the large size of the entity set and the evolution of entity semantics. Intuitively, modeling high-order correlation among entities can also be beneficial for entity prediction. It is better to also evaluate the performance of the learned representations on this task.\n3. The authors cluster entities only based on their representations. However, in knowledge graphs, entities are connected via various relations and different relations may indicate different correlations. For example, the relation \"leave from\" means an athlete will not interact with a club, but \"transfer to\" means they will have more interactions. How can the proposed method handle such correlations brought by relation semantics?\n4. Only two benchmark datasets are used for evaluation, making the experimental results unconvincing. ICEWS14C and ICEWS18C datasets are actually subsets of ICEWS14 and ICEWS18, and they are both derived from the same resource (i.e., ICEWS). More TKGs especially from different resources such as GDELT, YAGO, and Wikidata should also be used for evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s meaningful comments and insightful questions.\n\n>Q1: Lack of clarity in introducing and defining entity graph and cluster graph concepts\n\nThank you for raising this concern. We acknowledge that these crucial concepts were not sufficiently introduced or defined, potentially obscuring the intuition behind our approach. **Therefore, in the camera-ready version, we will incorporate a concise yet informative discussion of entity graphs and cluster graphs in the introduction**, providing readers with an early understanding of these key concepts and their role in our approach. **In Section 3 (Preliminaries), we will expand our definitions of entity graphs and cluster graphs, including more detailed explanations of their structures, properties, and significance in our approach.**\n\n\n>Q2: Lack of evaluation on future entity prediction task\n\nIn response to this valuable feedback, **we have conducted additional experiments to evaluate our model’s performance on the future entity prediction task, as shown in Table 2 of the rebuttal PDF (which is attached in the global rebuttal in the beginning).** Although our approach do not achieve the SOTA performance in terms of MRR and Hits@1, it achieves the best results for Hits@3 and Hits@10. These results demonstrate the effectiveness and robustness of our approach, particularly in capturing a broader range of relevant entities.\n\n\n>Q3: Handling of different relation semantics in entity clustering\n\nFirstly, it is important to note that TKG datasets do not provide explicit semantic descriptions of relations like “leave from” or “transfer to”. The training process typically uses only entity and relation IDs. However, we do model different relation types using a Relation-Aware Graph Convolutional Network, capturing distinct characteristics of various relation types even without explicit semantic descriptions.\n\n**Moreover, our DECRL approach is designed to capture the temporal evolution of high-order correlations, which indirectly addresses the issue of different relation semantics.** For example, if entities consistently interact over a continuous period, they have a higher probability of being clustered together at each timestamp. By capturing the temporal evolution of high-order correlations, our approach reinforces the closeness of their relationship over time. Conversely, if entities do not consistently interact over time, they have a lower probability of being clustered together. The temporal evolution component of our approach allows for the gradual distancing of these entities in the representation space. Therefore, DECRL can effectively handle scenarios where different relations may indicate varying levels of future interaction, without relying on explicit semantic information.\n\n**To illustrate this capability, we would like to draw attention to the comparison between Figure 2d (Final DECRL) and Figure 2f (Final DECRL-w/o-fusion, which only models high-order correlations without capturing their temporal evolution) in the manuscript.** This comparison clearly illustrates that capturing the temporal evolution of high-order correlations leads to superior entity representations, as evidenced by the larger inter-cluster distances and tighter intra-cluster entity groupings. Furthermore, by comparing the first and third columns of Figure 2 in the manuscript, we can observe the progression of training. This comparison demonstrates that capturing the temporal evolution of high-order correlations gradually increases the separation between clusters while simultaneously tightening the grouping of entities within clusters.\n\n\n>Q4: Omission of Wikidata, YAGO and GDELT datasets in the experiments.\n\nFirstly, there is a fundamental difference in timestamp types between ICEWS and the Wikidata and YAGO datasets. ICEWS uses single-point timestamps, while Wikidata and YAGO use time intervals for events. Our research focuses on modeling the temporal evolution of high-order correlations between entities. Datasets with single-point timestamps, e.g., ICEWS, show more frequent temporal changes and higher temporal complexity, making them more suitable for capturing the temporal evolution of high-order correlations. This is why we initially focused on the ICEWS dataset. Moreover, the SOTA models have already achieved very high performance on Wikidata and YAGO datasets, with MRR scores exceeding 99\\% for relation prediction. Given the limited room for improvement, we initially excluded these datasets from our preliminary manuscript.\n\nIn addition, we initially excluded GDELT dataset due to its known issues with false positives [1] and a high proportion of abstract conceptual entities (e.g., POLICE and GOVERNMENT) [2], since we cannot predict a government’s activities without knowing which country it belongs to.\n\nHowever, we acknowledge that including a wider range of datasets would provide a more comprehensive evaluation of our approach’s performance and robustness. **In light of your valuable feedback, we have conducted additional experiments on the GDELT dataset, as well as on the WIKI and YAGO datasets.** The results of these experiments are presented in Tables 1 and 3 of the attached rebuttal PDF (which is attached in the global rebuttal in the beginning). We are pleased to report that our approach has achieved the SOTA relation prediction performance across all these datasets, demonstrating the effectiveness and robustness of our approach.\n\n[1] Ward, M. D., Beger, A., Cutler, J., Dickenson, M., Dorff, C., \\& Radford, B. Comparing GDELT and ICEWS event data. *Analysis*, 21(1): 267-297, 2013.\n\n[2] Li, Z., Jin, X., Li, W., Guan, S., Guo, J., Shen, H. et al. Temporal knowledge graph reasoning based on evolutional representation learning. In *Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval*, pages 408-417, 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Summary of Revision:

We sincerely thank all the reviewers for their insightful reviews and valuable comments, which are instructive for us to improve our paper further.

The reviewers generally held positive opinions of our paper, in that the proposed approach is **“technically reasonable”, “well presented”, “clearly explained”**, and we **“clearly describes the motivation for the paper and the methods used”**; this paper is **“well-organized”, “clearly presented”, “well-written”**, and **“easy to follow”**; we **“demonstrate effectiveness through both qualitative and quantitative analyses”**, the experiment results are **“remarkable”**; and the proposed approach **“achieves state-of-the-art (SOTA) performance”**.

The reviewers also raised insightful and constructive concerns. We made every effort to address all the concerns by clarifying DECRL’s distinctions from related methods and the ability to address diverse relation semantics. We supplement with new experiments on Wikidata, YAGO, and GDELT datasets, along with additional ablation and case studies.

>Q1: DECRL’s distinctions from related methods

While entity groups, hypergraphs, and clustering can all model high-order correlations, our approach offers unique advantages:

Firstly, methods using entity groups and hypergraphs require learning entity assignment mappers at each timestamp, a process that is complex to update and maintain, resulting in significant computational overhead. **Our clustering-based approach, however, adapts more flexibly to dynamic data changes and is lightweight, facilitating easier integration with other techniques.**

Secondly, we use a fuzzy clustering algorithm with a fuzzy smoothing hyperparameter that controls node membership distribution, preventing clusters with very few nodes. **This approach allows for effective cluster construction even for nodes with limited interactions.** Using entity graphs or hypergraphs to achieve a similar advantage would be much more computationally expensive and resource-intensive.

**In addition, our experiments demonstrate the effectiveness of our approach compared to methods using entity groups and hypergraphs.** For example, the DECRL-w/o-fusion variant, which uses only clustering for representation learning, achieves MRR, Hits@1, Hits@3, and Hits@10 scores of 57.98, 41.90, 66.97, and 92.00, respectively. These results outperform the hypergraph-based method, i.e., DHyper, which scores 56.15, 43.76, 65.46, and 85.89 on the same metrics. This superior performance can be partly attributed to fuzzy clustering’s ability to prevent the formation of extremely small clusters.


>Q2: The ability to address diverse relation semantics

Firstly, it is important to note that TKG datasets do not provide explicit semantic descriptions of relations like “leave from” or “transfer to”. The training process typically uses only entity and relation IDs. However, we do model different relation types using a Relation-Aware Graph Convolutional Network, capturing distinct characteristics of various relation types even without explicit semantic descriptions.

**Moreover, DECRL is designed to capture the temporal evolution of high-order correlations, which indirectly addresses the issue of diverse relation semantics.** For example, if entities consistently interact over a continuous period, they have a higher probability of being clustered together at each timestamp. By capturing the temporal evolution of high-order correlations, our approach reinforces the closeness of their relationship over time. Conversely, if entities do not consistently interact over time, they have a lower probability of being clustered together. The temporal evolution component allows for the gradual distancing of these entities in the representation space. Therefore, DECRL can effectively handle scenarios where different relations may indicate varying levels of future interaction, without relying on explicit semantic information.

**To illustrate this capability, we would like to draw attention to the comparison between Figure 2d (Final DECRL) and Figure 2f (Final DECRL-w/o-fusion, which only models high-order correlations without capturing their temporal evolution).** This comparison clearly illustrates that capturing the temporal evolution of high-order correlations leads to superior entity representations, as evidenced by the larger inter-cluster distances and tighter intra-cluster entity groupings. Furthermore, by comparing the first and third columns of Figure 2 in the manuscript, we can observe the progression of training. This comparison demonstrates that capturing the temporal evolution of high-order correlations gradually increases the separation between clusters while simultaneously tightening the grouping of entities within clusters.


>Q3: Performance on Wikidata, YAGO, and GDELT datasets.

**We have conducted additional experiments on WIKI, YAGO, and GDELT datasets.** The results of these experiments are presented in Tables 1 and 3 of the attached rebuttal PDF. We are pleased to report that our approach has achieved the SOTA relation prediction performance across all these datasets, demonstrating the effectiveness and robustness of our approach.


**New experimental results (see the rebuttal PDF)**:

1. Performance on different datasets: Tables 1 and 3 in the rebuttal PDF show the relation prediction performance of DECRL on WIKI, YAGO, and GDELT.

2. Performance of entity prediction task: Table 2 in the rebuttal PDF shows the entity prediction performance of DECRL on GDELT.

3. The contributions of attentive temporal encoder and the fuzzy c-means clustering method: Table 4 in the rebuttal PDF shows the performance comparison of DECRL and its variants on ICEWS14.

4. Model efficiency: Figure 1 in the rebuttal PDF illustrates the training time comparison with DHyper on ICEWS14 (in seconds).

5. Case study: Figure 2 in the rebuttal PDF illustrates the entity representations of DHyper on ICEWS14C.

Meta-review: This work proposes a Deep Evolutionary Clustering jointed temporal knowledge graph Representation Learning approach (DECRL) to learn representations of temporally evolving entities and relations. The main ideas include a cluster-aware alignment mechanism for one-to-one alignment of soft overlapping clusters across timestamps and a correlation encoder for capturing latent correlations between any pair of clusters. While reviewers acknowledged the empirical performance of the proposed method and commented that the idea seems reasonable, many critical points were addressed during the rebuttal, which should be added to the final version of the paper.

1. The authors should include all additional experimental results shown in the rebuttal PDF, including the results on Wikidata, YAGO, and GDELT datasets and the entity prediction performance. 
2. The authors should clarify the definitions of 'entity graph' and 'cluster graph' and add more detailed explanations to Figure 2.
3. Regarding the work's limitations, the authors wrote a single sentence: "This study overlooks the continuous temporal evolution of diverse high-order correlations, a limitation that future work will address.", which may not be informative. In the final version, the authors should elaborate on their work's limitations in much more detail.
Decision: Accept (poster)