TAMI: Taming Heterogeneity in Temporal Interactions for Temporal Graph Link Prediction
Authors: Zhongyi Yu
Jianqiu Wu
Zhenghao Wu
Shuhan Zhong
Weifeng Su
Chul-Ho Lee
Weipeng Zhuo
Venue: NeurIPS 2025 poster ()
Keywords: Temporal Graph Neural Network
Interaction Heterogeneity
Skewed Distribution
Time Encoding Function
Link History Aggregation
Primary area: general_machine_learning
TL;DR: This is the first work to identify the heterogeneity in temporal interactions of CTTGs and investigate its impact on the performance of temporal link prediction. We propose TAMI, which effectively handles the heterogeneity in temporal interactions.

Abstract: Temporal graph link prediction aims to predict future interactions between nodes in a graph based on their historical interactions, which are encoded in node embeddings. We observe that heterogeneity naturally appears in temporal interactions, e.g., a few node pairs can make most interaction events, and interaction events happen at varying intervals. This leads to the problems of ineffective temporal information encoding and forgetting of past interactions for a pair of nodes that interact intermittently for their link prediction. Existing methods, however, do not consider such heterogeneity in their learning process, and thus their learned temporal node embeddings are less effective, especially when predicting the links for infrequently interacting node pairs. To cope with the heterogeneity, we propose a novel framework called TAMI, which contains two effective components, namely log time encoding function (LTE) and link history aggregation (LHA). LTE better encodes the temporal information through transforming interaction intervals into more balanced ones, and LHA prevents the historical interactions for each target node pair from being forgotten. State-of-the-art temporal graph neural networks can be seamlessly and readily integrated into TAMI to improve their effectiveness. Experiment results on 13 classic datasets and three newest temporal graph benchmark (TGB) datasets show that TAMI consistently improves the link prediction performance of the underlying models in both transductive and inductive settings. Our code is available at https://github.com/Alleinx/TAMI_temporal_graph.

TAMI: Taming Heterogeneity in Temporal Interactions for Temporal Graph Link Prediction
Zhongyi Yu', Jianqiu Wu', Zhenghao Wu?, Shuhan Zhong Weifeng Su Chul-Ho Lee", Weipeng Zhuo 1,5x
Beijing Normal-Hong Kong Baptist University, University College Dublin 3 The Hong Kong University of Science and Technology, "Texas State University 6 Guangdong Provincial / Zhuhai Key Laboratory of IRADS, China (zhongyiyu, jiangiiuu, jianqiuwu, wfsu, wwipengzhuu) Sunbe.dnbubuue.. en, zhenghao. su@ucdconnect. ie, szhongaj jose.. ust. ust.hk, chulho. lee@txstate.edu
Abstract
Temporal graph link prediction aims predict future interactions between nodes in a graph based on their historical interactions, which are encoded in node embed- dings. We observe that heterogeneity naturally appears in temporal interactions, e.g., few node pairs can make most interaction events, and interaction events happen varying intervals. This leads to the problems of ineffective temporal information encoding and forgetting of past interactions for a pair of nodes that interact intermittently for their link prediction. Existing methods, however, do not consider such heterogeneity in their learning process, and thus their learned temporal node embeddings are less effective, especially when predicting the links for infrequently interacting node pairs. To cope with the heterogeneity, we propose novel framework called TAMI, which contains two effective components, namely log time encoding function (LTE) and link history aggregation (LHA). LTTE better encodes the temporal information through transforming interaction intervals into more balanced ones, and LHA prevents the historical interactions for each target node pair from being forgotten. State-of-the-as temporal graph neural networks can be seamlessly and readily integrated into TAMI improve their effectiveness. Ex- periment results on 13 classic datasets and three newest temporal graph benchmark (TGB) datasets show that TAMI consistently improves the link prediction perfor- mance of the underlying models in both transductive and inductive settings. Our code is available wtthtps:/// http::///iitt htgggttuubb.. com..ccm///1/ wwtp:///////////eeeeeae caam///leeinm/MMLL_Lmorrraa____ temporal_graph
Introduction
Temporal link prediction is a fundamental task that forecasts future interactions between two nodes on continuous-time time temporal graphs (CTTGs). This important variety of real-world scenarios with dynamic graph topologies changing over time, such as social networks 24 MI user-item interaction
systems B
38 40
traffic networks [35 32 37 and physical systems 20 The temporal
link prediction task involves two major steps. The first step is to compute the temporal embedding of each node by aggregating information from its own historical interactions with neighboring nodes. For each target node pair for link prediction, the second step is to take their embeddings into a link predictor (typically an MLP) to estimate the probability of having link between them.
Heterogeneity arises in temporal interactions. For example, the number of interactions for each node pair varies substantially. The time between two consecutive interaction events can also be
"Coresponning author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

quite different. To see this, in Figure we plot the distribution of interaction intervals between any pair of nodes on the UCI dataset, showing that follows a power-law distribution. In other words, most interactions are frequent ones, while the infrequent ones (very large interaction intervals) still exist with a non-negligible probability. We also compute and report the Fisher's skewness in Figure [ indicating that the distribution is positively skewed or right-skewed. Here the skewness T of a random variable X defined as = E [(X /)") where μ and o denote the mean and standard deviation of X, respectively.
The heterogeneity temporal interactions poses two primary challenges to existing methods for temporal link prediction [34 7 [39] First, they use sinusoidal functions as their time encoding functions to encode the temporal difference between the target time and each interaction time t4- We observe that the frequency parameters of the time encoding functions are harder to learn when the distribution of the temporal difference is highly skewed, which the common case as seen from the distribution of interaction intervals. Second, they learn the embedding of each node based only on its recent interaction events. While the embedding encodes timely information, it can predominantly influenced by few neighbors having frequent interactions. As result, the temporal link prediction can be inaccurate for a target node pair having infrequent interactions over time. For example, is natural to predict that a couple would cat hamburgers if it is one of their recent favorites (frequent interactions). However, it makes more sense to predict that turkey will be on the table (infrequent interactions) if Thanksgiving is coming.
To address the above challenges, we propose TAMI, novel framework that tames the heterogeneity in temporal interactions to improve the performance of temporal link prediction. TAMI mainly contains two novel components, namely log time encoding function (LTE) and link history aggregation (LHA).
Dataset: UCI
10a
Probability (log scale)
10-1
10-
10-2
10-2
We propose to use logarithmic transformation LTE to rescale the temporal difference before it is taken into the time encoding function, so that the
10-3
10-3
10
10
frequency parameters of the time encoding func-
Skewness: 5.197
tion can be easier (or quicker) to learn. In addition, develop LHHA to preserve the information of the most recent & interactions for each target node pair for link prediction, regardless of their interaction frequency. In other words, the historical interac- tions of each node pair explicitly used for their link prediction, no matter when they happened.
101
10a
105
107
Interaction Intervals (log scale). Unit: second
The interaction intervals between any pair of nodes on the UCI dataset follow a power- law distribution and have a high (positive) skew- ness value, meaning that the intervals are highly right-skewed.
Figure 1:
Our key contributions can be summarized fol- lows:
First study on the heterogeneity in temporal interactions. To the best of our knowledge, this is the first work to identify the presence of heterogeneity in temporal interactions of CTTGs and investigate its impact on the performance of temporal link prediction.
a novel framework to cope with the heterogeneity in temporal interactions. We propose TAMI, which effectively handles the heterogeneity temporal interactions. TAMI contains two novel modules, namely LTE and LHA. LTE rescales the temporal differences using a logarithmic transformation while LHA is specifically designed to preserve historical interactions for each target node pair for link prediction. Existing temporal graph neural networks can be seamlessly and readily integrated into TAMI.
TAMI:
Extensive evaluation on open datasets. We validate the effectiveness of TAMI on 13 classic temporal graph datasets covering different fields with comprehensive temporal scales as well as three newest ones from temporal graph benchmark (TGB). Results show that TAMI consistently and substantially improves the link prediction accuracy and training efficiency of the underlying graph neural networks, with up to 87.05% improvement link prediction accuracy and 76. reduction in total training time.

---

Temporal Graph
Temporal mode embeddding:: feue, by
Popular Temporal
Link History Aggregation (LHHA)
Log Time Encoding
Neighbor Aggregator (og. DyGFormer)
Link Predictor
Function (L.TR)
(t)
hwv
gCr
Time encoding vector
Interaction history between nodes U, V
Puv
Frequent interaction
Link prediet time
Figure 2: The TAMI framework.
2
Related Work
Time Encoding Functions in Link Prediction. Most prior methods [4 39
29
a common approach for time encoding, which is based on either periodic time encoding functions introduced in (33] their simplified ones. To predict the existence of link between two nodes at the target time, the temporal difference between the target time and the time of each historical interaction needs to be calculated. The time differences are then mapped time encoding vectors using a periodic function such as sinusoidal function. However, this time encoding pipeline overlooks the effect of the skewness in the time differences. We observe that it takes longer than needed train a model, and it can also degrade the performance of link prediction, especially when predicting the links for infrequently interacting node pairs. In this work, we propose LTE effectively alleviate the skewness in the time differences via simple yet effective logarithmic transformation, leading to better training efficiency and improved link prediction accuracy.
Temporal Neighborhood Aggregation in Link Prediction. In learning temporal node embeddings using information aggregation, there typically (TGNNs), which are random walk-based TGNNS and temporal neighbor-based TGNNs. Random walk-based TGNNs [3 [0 [4] first generate multiple causal, anonymous walks for each node and then aggregate information from these walks to obtain the final node embeddings. Since frequent interactions predominantly appear as edges in the graph, it may be less likely to sample infrequent interactions in each walk. In addition, temporal neighbor- based (or graph convolution- compute temporal node embeddings by aggregating node or edge features from the temporal neighbors of each target node. to store the states of historical interactions. DyGFormer 3D computes node embeddings by aggregating an extensive number of single-hop temporal neighbors using an attention mechanism. GraphMixer proposes to use MI.-Mixer (26) and neighbor mean-poot wean--ooling to compute node embeddings. However, these methods mainly rely on the recent interactions of node to compute its embedding, which are again dominated by frequent interactions with limited number of its neighbors. Thus, they can be less effective in predicting the links for infrequently interacting node pairs. contrast, we propose LHA to effectively address this problem by explicitly capturing the historical interactions between each target node pair for link prediction in the modeling process.
types of temporal graph neural networks
based) TGNNs (27 1 22
44
3
17
34 29
39 25|
15
For example, recurrent networks or temporal point processes 27 are leveraged in [
3 TAMI Design
3.1 Problem Definition
A CTTG over a time interval [0, 2] is characterized by Gr = (Vr,Er). which is a sequence of chronologically ordered interaction events between nodes up to time T (inclusive), where Vr is the set of nodes and Er is the set of temporal edges up to T.. Here, a temporal edge e E Er between two nodes E Vr and U e V/ represents interaction event between and at time t << T. Note that multiple edges can exist between two nodes in the temporal graph.
Given two target nodes and u the target time T, and all historical interaction events in the graph up to T (exclusive) that are characterized by Ger, our task to predict how likely nodes u and v will interact with each other at time 7, as illustrated in Figure 2 In other words, it is to estimate the
probability Pur of having link between u and v at time r. To tackle this problem, there are two
common major steps involved in most TGNNs 34
39
2 19 First, they compute the temporal
3

---

Probability (log scale)
Average Precision
1.00
0.98
10-2
10"!
10
0.96
10
10c
107
0.94
Ours: LTE
Baseline: Original TE
Skewness:
2.385
0.92
101
10
10
107
50.25
(0.25,1.88) (1.88,13.91) >13,91
Temporal Differences: (log scale)
Interaction Interval. Unit: days
(a) The distribution of temporal difference At on the UCI dataset is highly right-skewed.
(b) LTE effectively mitigates the skewness and im- proves the model (GraphMixer ( performance.
Figure 3: (a) Distribution of temporal difference At and (b) model performance on the UCI dataset.
embeddings of nodes u and v by aggregating information from their recent neighbors who have recently had interactions with them. Second, the learned embeddings then fed into link predictor, e.g., an MLP with a sigmoid activation function, to estimate the probability Pue-
3.2 LTE: Log Time Encoding Function
In most TGNNs temporal edge in (<r) is associated with time encoding vector ztt), which is given by
39
each historical interaction at time T (or its corresponding
(1) = cos A x w);
with At -- and learnable frequency parameters - (t-1)/1))) Jو where dr is the dimension of the time encoding vector, and the values of and
initialized = B vdr. This time encoding function first maps At to a monotonically decreasing vector AL x w such that the values within (0, At] and then projects them to [-1, using the cosine function. For the sake of clarity, we refer to this type of time encoding as original TE.
We observe that a given node can interact with various other nodes at different frequencies. Thus, their interaction intervals can vary significantly. As shown in Figure I the interaction intervals between nodes highly right-skewed on the UCI dataset. Such right skewness also appears in the distribution of temporal differences At, as can be seen from Figure 3a TGNNs trained on these skewed inputs using original TE learn better the interactions that frequently appear but struggle with the ones that seldom occur. As result, they are ineffective when making link prediction for a pair of nodes that interact rarely whose interaction intervals long, shown in Figure 3bb] where we group testing node pairs for link prediction according to their average interaction intervals and report the average prediction accuracy for each group of node pairs.
To address this challenge, we propose LTE, simple yet effective time encoding function, which rescales the value of At via a logarithmic transformation so that it follows more balanced distribution. With LTE, a large variance At no longer leads to large discrepancy, making it easier to learn the frequency parameters The time encoding function in LTE is formally defined as
z (t) cos( At; W), with At, In(1 At).
(2)
To see how the logarithmic transformation mitigates the skewness in the distribution, we consider At follow Pareto distribution, which is a pawer-law distribution. We have the following:
Proposition Suppose At follows Pareto distribution with the shape parameter a 3, whose skewness is always greater than 2. LTE reduces the skewness to 2.
The proof is provided in Appendix A
In practice, At may not strictly follow a Pareto distribution. Nonetheless. LTE can still effectively mitigate skewness, thereby improving the performance the underlying model. As shown in Figure 3b LTE improves the model performance by reducing the skewness of A/ on the UCI dataset from 2.385 to 1.144 Please refer to Table TT Appendix B.5 for the values of skewness with and without LTE on different datasets.

---

Once the time encoding vector is obtained. the next step in TAMI is to compute the temporal node embeddings. For this purpose, any existing TGNN In temporal neighbor-based methods [3
31 can be adopted. the temporal embedding h,, E ERd of node is generally computed based on the m most recent interactions of with its temporal neighbors. Specifically, N,, be the set of the m recent neighbors of u, æ, be the initial embedding of node j. and o be the initial embedding of temporal edge Euj. and (t,)) ) the interaction event with at time fz. Then, the embedding h. is obtained as follows:
27
33
34
13
22
be the time encoding vector for
hn = AGGREGATE (((my; Buyi z: (1(())))
where AGGREGATE()) is an aggregation function and [:] is the concatenation operation. Similarly, random-wall based methods [31 obtain the embedding hn by aggregating the information from nodes that appear in random walks starting from Both classes of methods can be readily integrated into TAMI, as shall be demonstrated in Section 4.5
3.3 LHA: Link History Aggregation
As mentioned above, it is common practice that the temporal embedding of a node u is based only on its most recent m interactions or the recent ones that appear in random walks from u. However, this can be problematic for predicting a link between two nodes u and especially when they do not appear in their mutual nearest neighbors. In that case, their historical interactions are forgotten in updating their temporal embeddings, thereby degrading the link prediction accuracy. See Figure 4 for an illustration. We empirically observe that this is indeed the case, as shown in Figure [ There is non-negligible (possibly significant) portion of node pairs that could have forgotten their mutual interaction history for temporal link prediction, and their link prediction accuracy is the worst.
To resolve this problem, we propose a novel light-weight module called link history aggre- gation (LHA). Its core idea is to preserve the most recent interactions for a target node pair and leverage this historical information, along with their temporal node embeddings, to pre- dict a link between the target node pair. Let e et ewes e denote the most recent k in- teractions between nodes u and v that happen at times l1, 12,..., ,,4 before time т, respec- tively, where ty < tx-1 < The i-th historical interaction ei is associated with d,- dimennsiona historical edge embed- ding rt; E Rdd,, which encodes the informa- tion of the interaction. We use MT)(T) (ris, ri...... rit to indicate the historical
Forgotten Internction (H,15 4)
U
Peeeaaneeeee predict at t
Frequent Interaction
U4
U6
Ug
U7
< т.
O Nade
TTemparal Link
4
Moot nocel interactione sen: Link to pred et at
Figure 4: When predicting the future link between u and u at 41, their historical interaction at ls is no
longer retained in their latest node embeddings.
edge embeddings of the most recent k interactions before time T. Then, the edge embedding vector rhe for link prediction at time - is defined as a weighted sum of the current node embeddings and the previous edge embedding as:
rhe y x Cue + +1 10--- 7) x r
where Cuv = MLP((h,:; ; h..l) E Rdd encodes the current states nodes u and v, with h. and he
being the temporal node embeddings of nodes u and respectively, and rá: is the most recent edge embedding in (T) Here the hyperparameter 7 E [0, 1] controls the 'forgetting' rate of historical interactions. For example, if = 1, the entire interaction history is discarded in computing r To bootstrap the link prediction between u and i which do not have interaction history, i.e. Ma (T) = 0. we set t = O.
To predict a link between nodes u and v at time 7 as the last step of TAMI, we first aggregate all the historical link embeddings in Mas (T) into single vector hwv. which summarizes the most recent k interaction histories between u and v. This can be written as
hup AGGREGATE (rih ris Ter..., aV) E Bd
where AGGREGATE()) denotes an aggregation function. While various options can be considered (such as sum, mean, attention), we employ the most-recent aggregator. It utilizes the edge
5

---

Dataset: EN
Dataset: LA
Dataset:
Dataset:
Percent
80
60
40
Exclusive Kolated
Mutual
Exclusive
Isolated
Mutual
Exclusive
Icolated
Mutual
Exclusive Kolated
Mutual
Precision
0.95
0.90
GrophMikor
GraghhMoor
0.90
GraphMoor
+HHA
0.76
GrophMxxer
Exclusive
Mutual
Exclusive
Isolated
Mutual
Exclusive
Isolated
Mutual
Exclusive
bolated
Figure 5: The performance of GraphMixer and its improvement with LHA. "Exclusive" means that neither of two nodes appears in the other's m recent interactions. "Isolated" indicates that only one node appears in the other's 's m recent interactions. "Mutual" means that both nodes appear.
embedding of the most recent historical interaction to update huv. i.e. hue r Finally, the link probability Pue is computed using an MLP with the sigmoid activation function, i.e.,
Pue MLP (h he: huvl)-
(6)
Whenever interaction between nodes and occurs, their corresponding history set Ma (-) needs to be updated. Suppose u and v are connected at time T. A new edge embedding vector Tie computed using Equation (4), is added to (T) for the next link prediction. If the original size of (T [ is equal to k; we first remove the oldest historical edge embedding from Mun(T) and then insert r In other words, we keep the most recent l interactions only.
Remarks. As shown in Figure 3 and shall be demonstrated the subsequent section, LHA improves the performance of the underlying TGNN, with up 25.33% improvement in link prediction accuracy. LHA also has high efficiency in terms of GPU memory usage. Note that huv is the only additional embedding used in computing the link probability Pue. and obtain Auv r Also, the historical edge embedding r't, is updated based on its previous one, as in Equation 4 Thus, letting N be the number of target node pairs for prediction in the dataset, the total space complexity of LHA is O(Nd,), where d, is the dimension of each historical edge embedding. Since mini-batch computation target node pairs in GPU memory. Instead, they can be stored in CPU's system memory and loaded dynamically as required. Therefore, the additional GPU memory overhead introduced by LHA is (bd,,), where denotes the mini-batch size, and < N. Furthermore, with a slight increase in complexity, LHA also speeds up the convergence of the underlying model in training, with up to 76.7% % reduction in total training time, as shall be demonstrated shortly.
the most-recent aggregator to
be employed, it is unnecessary to load the edge embeddings for all
4 Experiments
4.1 Experimental Settings
We conduct experiments on 13 classic open datasets [1] Can. Parl. (CP), Contact (CO), Enron (EN), Flights (FL), Lastfm (LA), Mooc (MO), Reddit (RE), Social Evo (SE), Uci (UC). UN Trade (UT), UN Vote (UV), US Legis (US), and Wikipedia (WK). These datasets cover various domains and their details are provided in Section B.I We integrate two state-of-the-a TGNNs, namely GraphMixer 7 and DyGFormer 339. into our TAMI framework. More specifically, we use their temporal neighbor aggregation function We compare the performance with their vanilla counterparts, as well as seven other state-of-the the art TGNNs for CTTGs, including JODIE [3]. DyRep (27]. TGAT [34] TGN [22], CAWN [31],
the Temporal Neighbor Aggregator module of TAMI.
Edgebank (21). and TCL [29]. Descriptions of the basclines provided in Section B.2
We evaluate the link prediction performance of TGNNs under two settings: (1) the transductive setting where future links are predicted between nodes observed during training, and (2) the inductive setting where predictions are made for nodes unseen during training. Following [2] 39, we chronologically split each dataset into 70%/15%//15 615%% for aainiing/vaiddation/essingg and adopt the average precision (AP) score as the evaluation metric. We present the implementation details baseline configurations in Section B.4 Unless otherwise specified, we below present the major results for the transductive setting only and report additional results for the transductive and inductive settings in Section C and Section D respectively. For the generation of negative links, we follow
Section B.3 and

---

Table 1: AP for transductive link prediction under three different negative sampling strategies (NSS). Imp. (%) denotes the percentage of improvement. The first and the second best performers are marked in bold and underlined, respectively. Standard deviations five runs are reported in Table 21
NSS
Methods
CP
CO
EN
FL
LA
MO
RE
SE
UC
UT
UV
US
WK
JODIE
69.26
95.31
84.77
95.60
70.85
80.23
98. 31
89.89
89.43
64.94
63.91
75.05
96.50
DyRep
66.54
95.98
82.38
95.29
71.92
81.97
98.
88.87
65.14
63.21
62.81
75.34
94.86
TGAT
70.73
96.28
71.12
94,03
73.42
85.84
98.52
93.16
79.63
61.47
52.21
68.52
96.94
TGN
70.88
96.89
86.53
97.95
77.07
89.15
98.63
93.57
92.34 34
65.03
65.72
75.99
98 45
CAWN
69.82
90.26
89.56
98.51
86.99
80.15
99.11
84.96
95.18
65.39
52.84
70.58
98.76
EdgeBank
64.55
92.58
83,53
89.35
79.29
57.97
94.86
74.95
76.20
60.41
58.49
58.39
90.37
TCL
92.44
79.70
91.23
67.27
82.38
97.53
93.13
89.57
62.21
51,90
96.47
rnd
GraphMixer
75.90
91.94
82.26
90.98
75.56
82.83
97.33
93.34
93.38
62.61
52.20
71.55
97.23
DyGFormer
97.91
98.31
92.46
98.92
93.01
87.66
99.22
94.66
95.66
65.07
55.88
70.44
99.02
with TAMI
GraphMixer 78.38 95.26
90.97
96.75
88.13
83,53
98.84
93.41
96,20
62.98
57.74
71.57
98.89
Imp. (%)
3.27% 3.61%
10.59%
6.34%
16.64%
0.85%
1,56%
0.07%
3.02%
0.59%
10.61%
0.03%
1.71%
DyGFormer 98.67
98,70
92,66
98.94
94.03
88.49
99.29
94.74
96.72
66.39
56,02
71.40
99.25
imp. (96)
0.78% 0.40%
0.22%
0.02%
1.10%
0.95%
0.07%
0.08%
1.11%
2.03%
0.25%
1.36%
0.23%
GraphMixer 74.34
93.29
77.98
71.47
72.47
77.77
78
94.93
84.11
$7.05
51,20
81.65
90.90
DyGFormer 97.00
97.57
75.63
66.59
81.57
85,85
81.57
97.38
82.17
64.41
60.84
85,30
82.23
hist with TAMI
GraphMixer 78.81
93.30
81.68
73.01
80.23
83.61
82.56
96.80
87.69
69.74
70,90
84.56
90.97
Imp. (%)
6.01%
0.01%
4.74%
2.15%
10.71%
7.51%
5.25%
1.97%
4.26%
22. 24%
38.48 48%4
3.56%
0.08%
DyGFormer
98.96
97.72
81.02
67.77
83.40
86.26
85.18
97.56
85.89
65.16
81.72
86.10
82.38
Imp. (%)
2.02% 0.15%
7.13%
1.77%
2.24%
0.48%
4.43%
0.18%
4.53%
1.16%
34.32%
0.94%
0.18%
GraphMixer 69.48
90.87
75.01
74.87
68.12
74.26
85.26
94.72
80.10
60.15
51.60
79.63
88.59
DyGFormer 95.44
94.75
77.41
70.92
73.97
81.24
91.11
97.68
72.25
55.79
51.91
81.25
78.29
ind with TAMI
GraphMixer 70.94 96.12
88.95
93.64
91.06
79.82
96.19
96.09
84.12
87.73
79.53
83.31
93.89
Imp. (%)
2.10% 5.78%
18.58%
25.07%
33.68%
7.49%
12.82%
1.45%
5.02%
45.85%
54.13%
4.62%
5.98%
DyGFormer 97.25 98.47
86.23
75.55
74.03
92.39
94.37
97.76
80.13
68.01
78.
81,31
78.96
Imp. (%-
1.90% 3.93%
11.39%
6.53%
0.08%
13.72%
3.58%
0.08%
10.91%
21.90%
50.63%
0.07%
0.86%
the negative sampling strategies 21,. where random negative sampling is used for training and random, historical, and inductive negative samplings are applied during evaluation, with random negative sampling as the default choice.
4.2 Main Results
Performance under different negative sampling strategies. As shown in Table 1 TAMI substan- tially improves the performance of the integrated models under all three negative sampling strategies. In particular, for the random negative sampling strategy (Ind), the performance of both GraphMixer and DyGFormer improves on all 13 datasets, with improvement up to 16.64%. This is because LTE balances the skewness in temporal differences during the time encoding process, especially for nodes that interact rarely or whose interaction intervals are long, while LHA prevents historical interactions between the target node pair from being forgotten in predicting their future links, improving the link prediction accuracy. In addition, we show in Section [hhat TAMI remains effective in the inductive setting, consistently improving the performance of the integrated models. These results validate the effectiveness and versatility of the proposed TAMI framework.
We also present the results of TAMI under the historical (hist) and the inductive (ind) negative sampling strategies in Table [ As shown in Ta- [ TAMI consistently improves TGNNs under both negative sampling strategies, achieving im- provements of up to 38.48% and 54. 53.13% for the historical and inductive negative sampling strate- gies, respectively. This is because LHA maintains the interaction histories of node pairs as the test stage progresses, allowing the underlying models to better capture historical interaction patterns under historical sampling and inductive sampling, more accurately predicting future connections. Please refer to Table 12 for full results.
Table 2: Test mean reciprocal rank (MRR) scores on TGB datasets. TGB leaderboard is publicly available
Datasets tgbl-wiki 0.798 (rank 1)
DyGFormer
w/ TAMI Imp (%)
0.815 (rank 1) 2.13%
igbl-review 0.224 (rank 6) 0.419 (rank 2) 87.05%
tgbl-coin 0.752 (rank 2) 0.794 (rank 1) 5.59%
7

---

Table 3:
AP for transductive link prediction.
Methods
CP
CO
EN
FI.
LA
MO
RF
SF.
UC
UT
UV
US
WK
GraphMixer
75.90
91.94
82.26
90.98
75.56
82.83
97.33
93.34
93.38
62.61
$2.20
71.55
97.23
w/ LTE
78.07
92.22
82.76
90.99
75.21
83.09
97.35
92.64
94.98
62.65
52.21
70.88
97.28
Imp. (%)
2.86%
0.30%
0.61%
0.01%
-0.46%
0.31%
0.02%
-0.75%
1.71%
0.06%
0.02%
-0.94%
0.05%
w/ LHA
75.93
95.15
89,88
96.72
88.15
83.36
98.81
93.71
94,90
62.83
57.57
71.61
98.85
Tmp. (%)
0.04%
3.49%
9.26%
6.31%
16.66%
0.64%
1.52%
0.40%
1.63%
0.35%
10.30%
0.08%
1.67%
w/ TAMI
78.38
95.26
90.97
96.75
88,13
83.53
98.84
93.41
96.20
62.98
57.74
71.57
98.89
Tmp. (%)
3.27%
3.61%
10.59%
6.34%
16.64%
0.85%
1.55%
0.07%
3.02%
0.59%
10.61%
0.03%
1.71%
DyGFormer
97.91
98.31
92.46
98.92
93.01
87.66
99.22
94.66
95.66
65.07
88.88
70.44
99.02
w/ LTE
98.73
98.36
92.61
98.93
93.92
88.59
99.27
94.74
96.68
67.24
56.37
70.98
99.23
Tmp. (%)
0.84%
0.05%
0.16%
0.02%
0.98%
1.06%
0.06%
0.08% %
1.06%
3.34%
0.88%
0.77%
0.21%
w/ LIIA
97.57
98.65
92.59
98.92
93.44
87.55
99.23
94.72
96.00
64.53
55.99
71.23
99.07
Imp. (96)
-0.35%
0,34%
0.14%
0.00%
0.47%
-0.13%
0.02%
0.07% %
0.36%
-0.83%
0.20%
1.12%
0.05%
w/ TAMI
98.67
98.70 70
92.66
98. 94
94.03
88 449
99.29
94.74
96.72
66.39
56.02
71,40
99.25
1mp. (%)
0.78%
0.40%
0.22%
0.03%
1.10%
0.95%
0.07%
0.08%
1.11%
2.03%
0.25%
1.36%
0.23%
Performance on three datasets from temporal graph benchmark (TGB). We further evaluate TAMI on the igbl-wiki, igbl-review, and tgbl-coin datasets from TGB [9] As shown in Table 2 incorporating DyGFormer into the proposed TAMI framework significantly improves its performance, making it the top-performing model on the rgbl-wiki and tgbl-coin datasets, and the second-best model on the rgbl-review dataset. This is because LTE is robust to different levels of skewness in datasets (see Section C.3) and LHA improves prediction accuracy whenever the interaction history between the target node pair are informative in predicting their future link. Detailed dataset statistics and experimental settings are provided in Section
E
4.3 Ablation Study
We conduct experiments to examine the effectiveness of the proposed LTE and LHA modules. Table presents the test performance of TAMI and its two variants: w/ LTE, where we replace the original TE in TGNNs with the proposed LTE and keep the rest unchanged; w/ LHA, where we integrate the LHA module into TGNNs and keep the rest unchanged. As shown from the results, both LTE and LHA improve the underlying TGNN performance when integrated individually. The performance further boosts when they are combined together. This indicates that our designed LTE and LHA are highly effective and versatile across datasets with various domains and temporal scales. We demonstrate in Section D.2 that L.TE and I.HA are also effective in the inductive setting, consistently enhancing the performance of integrated TGNNs.
4.4 Robustness to the Increase of Negative Links
In this experiment, we evaluate the robust- ness of TAMI against increasing num- ber of negative links per positive link. A negative link refers to pair of nodes that are not currently connected and are used as negative samples in the link prediction process. Ideally, a link prediction model should assign a higher connection probabil- ity to positive links and a probability close to zero to negative links. The more nega- tive links, the more difficult the task is. In the default setting, the number of negative links is set to 1 per positive link. We run the experiments on four representative datasets (EN, LA, UC, and UV), and report the re- sults in Table 4 "NEG=50" denotes that each positive link is evaluated against 50 negative links when computing its connec- tion probability.
Table 4: AP of methods under various numbers of nega- tive links during testing. NEG=50 indicates that each positive link is evaluated against 50 negative links in the AP computation.
Method
LA
UV
Method
UV
NBG 1
GraphMiser
82.36
75.56
91.38
52.20
Dy.FFomeer
92.46
93.01
95.66
55.88
w/ w/TAMI
90.97
88.13
96.20
57.74
w/ TAMI
92.66
94.03
96.72
56.02
Imp
16.61%
10.61%
Imp (5)
0.22%
0.258
GraphMiser
53.43
47.06
82.05
18.17
DyCFommer 7581
77.77
68.77
20.86
w/ TAMI
70.36
65.05
80
21.62
w/ TAMI
76.37
80.58
90.76
20.97
Imp
38
14.
7.73%
18.99%
Imp
0.74%
3.61%
0.5558
NBG 25
GraphMissr 22.77
23.77
63.34
4.36
DyGiaamer 45.15
31.58
78.96
5:13
w/ TAMI
36.57
34.97
73.36
5.32
w/ TAMI
46.39
56.61
81.74
5.28
Imp
60661%
47.12%
15.30%
22.04%
Imp
9.74%
2.92%
NBG 50
GraphMiser 14.09
16.80
54.14
2.25
DyGiaamer $1.06
35.64
75.47
2.65
w/ LAME
23.43
21.66
63.74
2.74
w/ TAMI
32.43
44.76
76.59
2.72
Imp
46.766
17.74%
21.78%
Imp
4.41%
25.599
4.25%
2.6458
8

---

Valicaton AP
0.80
0:45
W
0.60
Toal
Valicuiion AP
0.05
WTAII
0.2
TAM
0.00
SyCramer
D92
TOAT
0yShome
0.80
0.70
0.34
GrastMon
20
50
100
20
Treining Epochs
Thaining Spocts
(a) The EN datasct
(b) The UC dataset
Figure 6: Validation AP vs. training epochs on the (a) EN and (b) UC datasets. TAMI enables
TGNNs to achieve higher validation average precision with fewer training epochs.
As shown in Table 4 integrating GraphMixer and DyGFormer into TAMI consistently enhances their performance across different numbers of negative links per positive link. Furthermore, the improvement ratio steadily increases as the number of negative links grows. For example, on the EN dataset, the improvement ratio for GraphMixer increases from 10.59% to 66.29% as the number of negative links per positive link increases from 1 to 50. This is because leveraging the interaction histories stored in LHA between target node pairs helps to avoid predicting negative links as positive ones. These results suggest that the proposed TAMI framework effectively improves TGNN performance in scenarios where the ratio of negative links to positive links is high, reflecting conditions that are more practical for sparse CTTGs.
4.5
Adaptivity to Different TGNN Architectures
We below show the adaptivity of TAMI to different TGNN architectures. Specifically, integrate random walk-based TGNN,
Table 5: AP Improvement (Imp.%) of various
TGNNs when integrated into TAMI. The format is of AP (+)p.%). Please refer to Table 13 for full results.
ie. CAWN (TI, and three temporal neighbor-based TGNNs, i.e., TGAT (34] TGN (22). and JODIE [13], into TAMI. Since JODIE does not utilize the time en- coding function, we report its performance with LHA integrated. As shown in Table
Method TAMD]
UC
CAWN
(-3288)) 91.02 (44.6488) 96.69) (+1.8198)
57.49
IGAT
91.37 (1.37(225.255)) 91.00 (1243856)) 96.36 (+21 5356)
60.0 (+12988)
TGN
92.34 (36))%)) 92.83 (422.5995)
95, 36 (43.80%)
67.50 (+-.22%)
JODI
(-6.88%) 87.95 (425.3396)
92.44 (+3.68%)
65.57 (43.2256)
TAMI consistently improves their performance, indicating the effectiveness of TAMI for different TGNN architectures. Additionally, we show in Table [3 that applying LTE and LHA individually
also boosts the performance of the underlying TGNNs.
4.6 Improved Training Efficiency
We observe that TAMI can speed up the training of underlying TGNNs. To show this, we plot the validation AP of TGNNs dur- ing training in Figure 6 On the EN and UC datasets, TGNNs within the TAMI framework achieve higher validation AP scores with much fewer train- ing epochs, compared to their vanilla counterparts. In other words, the underlying models converge faster than their vanilla versions. These findings sug- gest that TAMI can speed up the convergence of TGNNs while yielding improved validation AP scores. This because LTE facil- itates the learning of frequency parameters in the time encoding function by balancing input tem-
Dataset: UC
Dataset: EN
0.90
0.94
Validation Ap
0.88
(#
0.92
DyGFormer (# HI 32)
DyGForee (# HI 2)
0.86
w/ TAMI (# HI 32)
0.90
DyGFormer HI 32)
0.84
DyGFurmer HI 256)
20
40
20
Training Epochs (#HI legend: Number of historical Interactions)
GPU Memory
Training Time Per Epoch (Second)
Metric
Usage (MiB)
Method
UC
EN
UC
EN
DyGFormer (32, 256)
2429
3303
37
DyGFormer (2, 32) w/ TAMI
1359
1545
22
Imp. (%)
44.05%
53.22%
40.54%
29.63%
Figure DyGFormer integrated with TAMI can compute tempo- ral node embeddings using significantly fewer historical interac- tions while achieving better performance. poral differences, while LHA enables the underlying model to more effectively capture the patterns of historical interactions between target node pairs.
9

---

In addition, we demonstrate that DyGFormer, when integrated into our TAMI framework, can compute temporal node embeddings using significantly fewer historical interactions while achieving equal or even better performance. As shown in Figure 7 when integrated into TAMI, DyGiFormer on the UC and EN datasets requires only 2 and 32 historical interactions for attention calculation, respectively, outperforming the vanilla counterpart that uses 16x and 8x more historical interactions. The reduced number historical interactions results in lower GPU memory usage (up to 53.22%) Moreover, on both datasets, DyGFormer under the TAMI framework converges much faster than the vanilla version, with fewer training epochs and up to 40.54% reduction in training time per epoch.
Conclusion
In this paper, we observed the presence of heterogeneity in temporal interactions of CTTGs and proposed a novel framework TAMI to address the challenges therein. TAMI has two main modules, namely LTE and LHA. LTE balances the skewness in the time differences via simple yet effective logarithmic transformation, and LHA prevents the historical interactions for each target node pair from being forgotten. Existing temporal graph neural networks can be seamlessly and readily integrated into TAMI. Extensive experiments on 13 classic open datasets and three TGB datasets show that TAMI substantially improves the link prediction accuracy of the underlying models as well as their training efficiency, in both transductive and inductive settings.
Acknowledgments
We thank the anonymous reviewers for their constructive feedback. This work was supported in part by the Guangdong Provincial Key Laboratory of IRADS 2022B1212010000 the Guangdong Higher Education Upgrading Plan (2021-2025), and the Guangdong and Hong Kong Universities "1+1+1" Joint Research Collaboration Scheme (No. 2025A0505000004 and No. 2025A0505000012), This work was also supported in part by the National Science Foundation under Grant No. IIS-2209921, and the International Energy Joint R&D Program of the Korea Institute of Energy Technology Evaluation and Planning (KETEP), granted financial resource from the Ministry of Trade, Industry & Energy, Republic of Korea (No. 20228530050030).
References
[1] Unai Alvarez-Rodriguss Federico Battiston, Guilherme Ferraz de Arruda, Yamir Moreno, Matjaž Perc, and Vito Latora. Evolutionary dynamics of higher-order interactions in social networks. Nature Human Behaviour, 2021.
[2] Stan Brown. Mcasures of shape: Skewness and kurtosis, 2011.
[3] Yizhou Chen, Anxiang Zeng, Qingtao Yu. Kerui Zhang, Cao Yuanpeng, Kangle Wu, Guangda Huzhang, Han Yu, and Zhiming Zhou. Recurrent temporal revision graph networks. Advances in Neural Information Processing Systems, 2024.
[4] Ke Cheng, Peng Linzhi, Junchen Ye, Leilei Sun, and Bowen Du. Co-neighbor encoding schema: A light-cost structure encoding method for dynamic link prediction. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024.
[5] Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the
properties of neural machine translation: Encoder- decoder approaches. In Proceedings of SSST- 8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation. Association for Computational Linguistics, 2014.
[6] Hsing-Huan Chung, Shravan S Chaudhari, Xing Han, Yoav Wald, Suchi Saria, and Joydcep Ghosh. Between linear and sinusoidal: Rethinking the time encoder in dynamic graph learning. Transactions on Machine Learning Research, 2025.
[7] Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi. Do we really need complicated model architectures for temporal networks? In The Eleventh International Conference Learning Representations, 2023.
10

---

[8] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in Neural Information Processing Systems. 2017.
[9] Shenyang Huang, Farimah Poursafaei, Jacob Danovitch, Manthias Fey, Weihua Hu, Emanuele Rossi, Jure Leskovec, Michael M. Bronstein, Guillaume Rabusseau, and Reihaneh Rabbany. Temporal graph benchmark for machine learning on temporal graphs. In Thirty-eeventh Confer- ence Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
[10] Ming Jin, Yuan-Fang Li, and Shirui Pan. Neural temporal walks: Motif-aware representation learning on continuous-time dynamic graphs. Advances in Neural Information Processing Systems, 2022.
[11] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, 2015.
[12] Stephen Kokoska and Daniel Zwillinger. CRC standard probability and statistics tables and formulae. Crc Press, 2000.
[13] Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019.
[14] Xiaodong Lu, Leilei Sun, Tongyu Zhu, and Weifeng Lv. Improving temporal link prediction via temporal walk matrix projection. Advances in Neural Information Processing Systems, 2024.
[15] Linhao Luo, Gholamreza Haffari, and Shirui Pan. Graph sequential neural ode process for link prediction on dynamic and sparse graphs. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, 2023.
[16] Yuhong Luo and Pan Li. Neighborhood awaea scalable temporal network representation learning. Learning Graphs Conference, 2022.
[17] Yao Ma, Ziyi Guo, Zhaocun Ren, Jiliang Tang, and Dawei Yin. Streaming graph neural networks. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020.
[18] James W Pennebaker. Linguistic inquiry and word count: Liwe 2001, 2001.
[19] Katarina Petrović, Shenyang Huang, Farimah Poursafaei, and Petar Veličković. Temporal graph rewiring with expander graphs. arXiv preprint, 2024.
[20] Tobias Pfaff, Meire Fortunato, Alvaro based simulation with graph networks. In International Conference on Learning Representations, 2020.
and Peter Battaglia. Learning mesh-
[21] Farimah Poursafaci, Shenyang Huang, Kellin Pelrine, and Reihanch Rabbany. Towards better evaluation for dynamic link prediction. Advances in Neural Information Processing Systems, 2022.
[22] Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs. In ICML 2020 Workshop on Graph Representation Learning, 2020.
[23] Alvaro Sanchez-Gonvalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning simulate complex physics with graph networks. In International Conference on Machine Learning, 2020.
[24] Weiping Song, Zhiping Xiao, Yifan Wang, Laurent Charlin, Ming Zhang, and Jian Tang. Session-based social recommendation via dynamic graph attention networks. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, 2019.
[25] Yuxing Tian, Yiyan Qi, and Fan Guo. Freedyg: Frequency enhanced continuous-time dy- namic graph model for link prediction. In The Twelfth International Conference on Learning Representations, 2024.
11

---

[26] Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner. Jessica Yung. Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mip-mixer: An all-mip architecture for vision. Advances in Neural Information Processing Systems, 2021.
[27] Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. Dyrep: Learning representations over dynamic graphs. In International Conference on Learning Representations, 2019.
[28] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph Attention Networks. International Conference on Learning Representations, 2018.
[29] Lu Wang, Xiaofu Chang, Shuang Li, Yunfci Chu, Hui Li, Wei Zhang, Xiaofeng Hc, Le Song, Jingren Zhou, and Hongxia Yang. TCL: Transformer-baser Dynamic Graph Modelling via Contrastive Learning. arXiv preprint, 2021.
[30] Xuhong Wang, Ding Lyu, Mengjian Li, Yang Xia. Qi Yang. Xinwen Wang, Xinguang Wang, Ping Cui, Yupu Yang. Bowen Sun, et al. Apan: Asynchronous propagation attention network for real-time temporal graph embedding. In Proceedings of the 2021 International Conference on Management of Data, 2021.
[31] Yanbang Wang, Yen-Yu Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks. In International Conference on Learning Representations, 2021.
[32] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. In International Joint Conference on Artificial Intelligence, 2019.
[33] Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Self-attention with functional time representation learning. In Advances in Neural Information Processing Systems, 2019.
[34] Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Inductive repre- sentation learning on temporal graphs. In International Conference on Learning Representations, 2020.
[35] Bing Yu. Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, 2018.
[36] Le Yu. An empirical evaluation of temporal graph benchmark. arXiv preprint, 2023.
[37] Le Yu, Bowen Du, Xiao Hu, Leilei Sun, Liangzhe Han, and Weifeng Lv. Deep spatio-temporal graph convolutional network for traffic accident prediction. Neurocomputing, 2021.
[38] Le Yu, Zihang Liu, Leilei Sun, Bowen Du, Chuanren Liu, and Weifeng Lv. Continuous-tima user preference modelling for temporal sets prediction. IEEE Transactions on Knowledge and Data Engineering, 2023.
[39] Le Yu. Leilei Sun, Bowen Du and Weifeng Lv. Towards better dynamic graph learning: New architecture and unified library. Advances Neural Information Processing Systems, 2023.
[40] Le Yu, Guanghui Wu, Leilei Sun, Bowen Du, and Weifeng Lv. Element-guidee temporal graph representation learning for temporal sets prediction. Proceedings of the ACM Web Conference 2022, 2022.
[41] Mengqi Zhang, Shu Wu, Xueli Yu, Qiang Liu, and Liang Wang. Dynamic graph neural networks for sequential recommendation, IEEE Transactions on Knowledge and Data Engineering, 2022.
[42] Xiaohui Zhang, Yanbo Wang, Xiyuan Wang, and Muhan Zhang. Efficient neural common neighbor for temporal graph link prediction. arXiv preprint, 2024.
12

---

[43] Zeyang Zhang, Xin Wang, Ziwei Zhang, Haoyang Li, Zhou Qin, and Wenwu Zhu. Dynamic graph neural networks under spatio-temporal distribution shift. In Advances in Neural Informa- lion Processing Systems, 2022.
[44] Hongkuan Zhou, Da Zheng, Xiang Song, George Karypis, and Viktor Prasanna. Dis[TGL: Distributed Memory-Bare Based Temporal Graph Neural Network Training. In Proceedings of the International Conference for High Performance Computing, Networking. Storage and Analysis, 2023.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: This is the first study to identify the presence of heterogeneity in temporal interactions of CTTGs and investigate its impact on link prediction performance. We propose novel framework TAMI to handle the heterogeneity in temporal interactions, and existing graph neural networks can be seamlessly integrated into our TAMI framework. The claims in the paper supported by related work discussion in Section 2 experiments in Section [ and mathematical derivations in Section A Please also refer to the additional experiments in the Appendix.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
It is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please refer to Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specificattin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
14

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We provide the mathematical steps and justification of Proposition in Section
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed description of experimental setup in Section B.3 Section B.4] and Section We also include our code in the supplementary material.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility.
15

---

In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Dataset sources are provided in Section B.1 and Section Our code is provided in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cc
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
7 //nips. //nips.cc/publlt wwtpc/ppppppg.ud....so.pppppppee httee/gguudde/ooeSSubbbbiiie
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We describe dataset splits in Section 4. tion and detailed implementations in Section B.3
hyperparameter settings in Sec-
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report the mean and standard deviation of results all our experiments, over 5 repeated runs. We state these details explicitly the result tables. Due to the limited space, standard deviations of the results in the main text tables are provided in Section
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Computational resources are described in Section B.3 We provided a detailed
space complexity discussion the end of Section 3.3
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http:////wwrppp.cppui/cee///////
Answer: [Yes]
Justification: Our proposed framework is widely applicable to science and engineering applications and does not have any conflict with the NeurIPS code Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
17

---

The proposed method can be applied to wide variety of science and engi- neering applications. There no clear path negative societal impact from our work. Guidelines:
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out, For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not include data models with a high risk for misuse. Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
All benchmarks used in the paper are properly cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
18

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: Code has been submitted with the paper and includes clear documentation for running the experiments.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
19

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM for what should or should not be described.
20

---

Technical Appendix and Supplementary Material
In the Appendix, provide additional supplementary material to the main paper. The structure is as follows:
Section
outlines the proof Proposition
2. Section
outlines the experimental settings in detail.
3. Section
presents additional results for link prediction in the transductive setting.
4. Section
shows the results the inductive setting.
5. Section
outlines the experimental settings on TGB datasets.
6. Section
presents the standard deviations of the results in the main text tables.
A Proof of Proposition
Suppose At follows Pareto distribution with the shape parameter a > 3, whose skewness is always greater than 2. Note that if the shape parameter is 0 S 2, the variance is infinite, the skewness is undefined. Also, the shape parameter is < a S 3, its third moment is infinite. Thus, we focus on the case of a Then, for any value of a of A to 2.
show that LTE always reduces the skewness
Proof. Let T be a random variable that represents the temporal difference At and follows Pareto distribution. Then, 7 + 1 also follows a (shifted) Pareto distribution. Letting X = T + 1, we can write the probability density function (PDF) of X
OC I0+1>
fx(x)
I
Umin:
where >> 3 is the shape parameter, and Tmin > 1 is the scale parameter.
First, we show that the skewness of X always greater than 2. We write the skewness of X as function of a, i.c.,
r r= 2(1 a) a 3 11
2
is then straightforward see that g(a:) is monotonically decreasing, with g(a) + To show that g(ar) is monotonically decreasing, we can show that In g(a) exists and In g(a) is monotonically decreasing. We first take the natural logarithm
oo.
both sides of 7 and differentiate
In g(a) with respect to a. We can then show that g'(a)
g(a) < 0. In addition, we can easily see from Therefore, the skewness I of X is always greater than 2.
that g(a) + 2 as
Next, for any value of a >> 3, we show that LTE always reduces the skewness I to Recall that LTE transforms X via logarithmic function Y = In( (X). By using the change-of- varriablss formula for probability distributions, we can obtain the PDF Y as follows:
dx
(v = (x(x)
or -0y
dy
Letting = In(mmin) > O, we have
(v) ae a)
424
which is a shifted exponential distribution with the rate parameter 3. Since the skewness of (shifted) exponential distribution regardless the value of the rate parameter a, LTE always reduces the skewness to 2.
B Experimental Settings
B.1 Descriptions of Datasets
We use 13 datasets collected by (11 in experiments.
21

---

1. Can. Parl. (CP) is dynamic political network that captures the interactions among Canadian Members of Parliament (MPs) from 2006 to 2019. Each node represents an MP from an electoral district, and an edge is established when two MPs cast a "yes" vote on the same bill. The weight of each edge reflects the annual frequency with which one MP votes "yes" alongside another.
2. Contact (CO) describes how the physical proximity evolves among about 700 university students over a month. Each student has a unique and edges between students denote that they within close proximity to each other. Each edge is assigned a weight that reflects the physical proximity between students.
3. Enron (EN) is an email correspondence dataset that records the emails exchanged among employees of the ENRON energy company over three years.
4. Flights (FL) is dynamic flight network illustrating the development of air traffic during the COVID-19 pandemic. Nodes represent airports and the tracked flights are denoted as edges. The edge weights reflect the number of flights between two airports in a day.
5. LastEM (LA) records the interaction between users and songs. Users and songs are nodes and edges between them represent a eser-listens-to-sone relation. The dataset contains no attributes.
6. Mooc (MO) is a dataset that captures students' interactions with online course materi- als. Each edge represents student accessing content unit and is associated with 4-dimensional feature vector.
7. Reddit (RE) comprises user posts submitted to subreddits over one month. Users and subreddits are nodes, while timestamped posting requests form the edges. Edge features LIWC-feature vectors
18 of edit texts.
8. Social Evo. (SE) is a mobile phone proximity network that tracks the daily interactions of an undergraduate dormitory over eight months. Each edge is associated with 2-dimensional feature vector.
9. UCI (UC) is an unattributed online communication network among university students. Nodes are university students and edges are messages posted by students.
UN Trade (UT) is food and agriculture trading graph between 181 nations for more than 30 years. The edge weights indicate the total sum of normalized agriculture import export values between two countries.
11. UN Vote (UV) captures roll-call voting behavior in the United Nations General Assembly. Each time two nations vote a "yes" the same item, the edge weight between them is incremented by one.
12. US Legis. (US) is a senate co-sponsorship graph that captures the social dynamics among US legislators. The edge weights specify the number of times congresspersons have co-sponsored a bill given Congress.
13. Wikipedia (WK) records the edits on Wikipedia pages over month. Editors and Wiki pages are modeled as nodes, and posting requests are timestamped edges. Edge features are 17 172-dimensional LIWC feature vectors [8]
We present the dataset statistics in Table 6 where "#N&E Feat" refers to the dimensions of node and raw edge features. Table 7 summarizes the skewness of all 13 datasets, where the skewness is measured for the interaction intervals between pairs of nodes in each dataset.
B.2 Descriptions of Baselines
We select nine baseline link prediction methods, covering wide range of underlying TGNN architec- tures: random walk-based TGNN (CAWN), and temporal neighbor- based TGNNs (TGN, EdgeBank, JODIE, DyRep, TGAT, TCL, GraphMixer, DyGFormer).
1. TGN maintains an evolving memory for each node in temporal graph, updating its memory when the node participates an interaction. The stored historical states of the node are subsequently used by an embedding module to compute its future representation.
22

---

Table 6: Statistics of Datasets. The '.' symbol denotes that the dataset does not contain the corre-
sponding feature.
Datasels
Domains
#Nodes
oFdges
Unique Edges
#N&T Feat
Duration
Unique Steps
Time Granularity
Politics
734
74,478
51,331
years
years
CO
Proximity
694
2,426,280
79,531
&
I month
8,065
minutes
Social
184
125,235
3,125
&
years
22.632
Unix timestarpps
FL
Transport
13,169
1,927.145
395,072
months
122
days
LA
Interaction
1,980
1,293,107
154,993
&
month
1,283,614
Unix timestamps
178,443
mnonts
Unix timestamps
RE
Social
10,984
672.447
78,516
& 172
I month
669,065
Unix timestamps
Proximity
2,099,519
4,486
months 565,933
Unix timestamps
UC
Social
1,899
59,835
20,296
&
196 days
58,911
Unix timestamps
UT
Econamics
255
507.497
36,182
-&1
years
32
years
UV
Palities
201
1,035,742
31,516
years
72
years congresses
US
Politics
225
60,396
26,423
-&
12 congresses
12
9,227
157,474
18,257
172
month
152,757
Unix timestarps
Table 7: Skewness of datasets measured by interaction intervals.
Datasets
CP
CO
EN
FL.
T.A.
MO
RF
SF
UC
17
UV
US
WK
Skewness
1.71
10.96
4.24
2.37
3.15
4.33
2.46
18.63
5.2
6.35
6,36
9.17
4.48
2. EdgeBank EdgeBank is transductive link prediction method with no trainable parameters. It stores observed interactions between nodes in a memory unit, which is updated using various strategies. A future interaction is predicted as positive if it is retained in memory, and negative otherwise (2I). Depending on the memory update strategies, EdgeBank has four variants: EdgeBank uses unlimited memory and retains all observed edges; EdgeBank and EdgeBank retain only recent edges within a fixed-size time window. The window size for EdgeBanktw-ts is set to the duration of the test split, while EdgeBank,w-re adjusts the window size based on the time intervals between repeated edges; EdgeBank,, retains only edges that appear more than specified threshold number of times. We evaluate all four variants and report the best-performing one.
3. JODIE is designed for temporal bipartite networks involving user-item interactions. It maintains the states of both user and item nodes and utilizes two coupled recurrent neural networks to update these node states. Additionally, projection operation is introduced to learn the future representation trajectory for each user and item [13.
4. DyRep introduces a recurrent architecture to update node states at each interaction, com- plemented by a temporal-ateer attentttteee aggregation module that captures the evolving structural information in temporal graphs. [27].
5. CAWN first extracts multiple causal anonymous walks for each node, enabling the explo- ration of the causality in network dynamics. Then, it employs recurrent neural networks to encode each walk and aggregates them to obtain the final node representation [31].
6. TGAT It computes the representation of node by aggregating its temporal neighbors using the graph attention mechanism (28). with a time encoding function to capture temporal patterns 34
7. TCL first performs a breadth-first search on the temporal subgraph to identify the temporal neighbors of the target node. Subsequently, a graph transformer is used to encode neigh- bor embeddings and graph topologies, enabling the computation of node representations. Furthermore, a cross-attention mechanism is employed to capture the interdependencies
between the two interacting nodes (29]
8. GraphMixer utilizes the MLP-Mixer 26 to encode both temporal information and the historical interactions of the target node. Additionally, node encoder employed to aggregate the node features of temporal neighbors 77. In their experiments, they show that fixed time encoding function outperforms its trainable counterpart.
9. DyGFormer a Transformer-baset architecture that computes node representations by aggregating features from each node's 's temporal neighbors. It introduces neighbor co- occurrence encoding scheme capture the correlations between nodes within an interaction and patching technique to help the model capture long-tern dependencies [39].
23

---

B.3 Implementation Details
We train all TGNNs (excluding EdgeBank, which has no trainable parameters) using the Adam optimizer [. with binary cross-entropy loss as the objective function. TGNNs are trained for 100 epochs with early stopping, where the patience score set to 20. We use learning rate 0.0001 and batch size of 200 for all methods and datasets. The model with the best validation performance selected for testing. Each method is run five times using random seeds ranging from the average performance reported to minimize any deviations. For the hyperparameter in our LHA module, we search for the best ? range from 0.0001 to during the training and validation phases and then use the y with the best validation performance in the test phase. Specifically, we set = 0.0001 for the MO and SE datasets, 7 = 0.1 for the UV dataset, and 7 = 0.9 for the remaining ten datasets. The dimension of historical edge embedding is set equal the dimension of temporal node embedding, i..., d, d, where detailed configurations can be found in Section
with
All our experiments are conducted CUDA 12.1. We train and test the proposed TAMI framework using single NVIDIA A100 80G GPU.
GPU server rumning Ubuntu 22.04, with PyTorch 2. 2.0 and
B.4
Configurations of Baselines
conducted an extensive hyperparameter search across all 13 datasets. For consistency, we adopt the optimal hyperparameter settings reported in [9 for all baseline methods. We first outline the configurations that remain consistent across all datasets, followed by the specific hyperparameter settings for each dataset.
39
The consistent configurations are as follows:
JODIE
1. Dimension of node memory: 172
2. Dimension of output representation: 172
3. Memory updater: vanilla recurrent neural network
DyRep
1. Dimension of time encoding: 100
2. Dimension of node memory: 172
3. Dimension of output representation: 172
Number of graph attention heads:
Number of graph convolution layers:
6. Memory updater: vanilla recurrent neural network
TGAT
1. Dimension of time encoding: 100
Dimension of output representation: 172
3. Number of graph attention heads:
4. Number of graph convolution layers: 2
TGN
1. Dimension of time encoding: 100
2. Dimension of node memory: 172
Dimension of output representation: 172
Number of graph attention heads:
Number of graph convolution layers:
6. Memory updater: gated recurrent unit
CAWN
1. Dimension of time encoding: 100
2. Dimension of position encoding: 172
3. Dimension of output representation: 172

---

4. Number of attention heads for encoding walks: 8
5. Length of each walk (including the target node): Time scaling factor le-6
TCL
1. Dimension of time encoding: 100
Dimension of depth encoding: 172
Dimension of output representation: 172
Number of attention heads:
5. Number of Transformer layers: 2
GraphMixer
1. Dimension of time encoding: 100
2. Dimension of output representation: 172
3. Number of MLP-Mixer layers: 2
Time gap T: 2000
DyGFormer
1. Dimension of time encoding: 100
Dimension of neighbor co-occurrence encoding de: 50
Dimension of aligned encoding d: 50
4. Dimension of output representation: 172
5. Number of attention heads: 2
Number of Transformer layers:
The hyperparameter settings for each method across different datasets are shown in Table 8 Table 9 and Table [0
Table 8: Dropout rates of different methods.
Datasets
Dy Rep
CAWN
GraphMixer
DyGFormer
CP
0.0
0.2
0.9
0.2
U.2
0.1
CO
0.1
0.1
0.0
EN
0.1
0.0
0.2
0.D
D.1
0.1
0.5
0.0
TAA
0.3
D.O
0.1
0.3
D.I
0.1
0.0
0.1
MO
0.2
D.1
0.1
RE
0.1
0.1
0.1
D.1
0,1
0,5
0.2
SE
0.1
D.1
0.1
D.1
0.D
0.3
0.1
UC
UT
0.4
0.1
0.1
0.2
D.1
0.0
0.1
0.0
US
0.2
0.0
0.1
0.1
0.1
0.3
0.4
0.0
WK
0.1
0.1
0.1
0.1
B.5 Skewness of Temporal Difference
Table I1 presents the skewness temporal difference for both the baseline (Original TE) and the proposed (LTE) time encoding functions. The skewness score is computed using Fisher's moment coefficient of skewness [12 2] A positive skewness score indicates that the temporal differences are right-sk skewed, negative score indicates they left-skewed, and
of 0 suggests balanced distribution. Compared to the baseline encoding function, the proposed LTE method effectively reduces the skewness of temporal difference.
C Comprehensive Results for Transductive Temporal Link Prediction
This section presents additional results for transductive link prediction. Section C. presents the comprehensive results of Table [ Section C.2 shows that applying LTE and LHA. individually also improves the link prediction accuracy of the integrated model. Section C.3 discusses how the
25

---

Table 9: The sample size of temporal neighbors, the number of causal anonymous walks, and the length input sequences & the patch size of different methods. The number parentheses indicates the number of historical interactions used compute each temporal node embedding during the test stage. For example, on the CP dataset, setting the input sequence length to 2048 denotes that, on average, 99. 99% of historical interactions are used compute single temporal node embedding.
Datasets
JODIE
TGAT
TGN
CAWN
TCL
GraphMixer
DyGFormer
CP
10
128
20
20
2048 (99.98%) &
CO
64
20
20
(0.4918) &
EN
32
20
20
256 (31.69%) &
FL
10
64
20
20
256 (34,93%) &
LA
128
(41 (41.23%) 16
MO
10
20
10
64
20
20
256 (57.59%) &
RE
32
64 (40.48%)
SE
64
20
(0.05%) &
UC
64
20
20
32 (34,13%)
UT
64
256 (7.05%)
UV
10
10
64
20
20
128 (66%) &
WK
10
20
10
32
20
30
32 (39.379) &
Table 10: Strategies for sampling temporal neighbors during testing and the best-performing variants of EdgeBank.
Dutasets
DyRep
TGAT
TGN
TCL
GruphMixer
EdgeBBank Variant
uniform
uniform
uniform
uniform
uniform
FdgeBankenr
CO
recent
recent
recent
recent
recent
EdgeBankew-rre
EN
recent
rocmt
recett
recent
recent
EdgeBanke -t
FL
EdgeeBunko
LA
recent
recent
recent
recent
recent
EddeeBank
MO
EdgeBanke
RE
recent
uniform
recent
uniform
recent
EdgeeBankse
SE
recent
recent
recent
recent
recent
EdgeBankes
UC
recent
EdgeBankas
UT
recet
uniform
revent
uniform
uniform
EdgeBanke
UV
uniform
uniform
uniform
EdgeBBank eoreee
US
recent
recent
recent
uniform
recent
EdgeBank;
WK
EdgeBank.e
Table 11: Skewness of temporal difference different time encoding functions on 13 datasets. The proposed LTE time encoding function effectively reduces the skewness of temporal difference.
Datasets
CP
CO
US
WK
Original TE
1.73
28.81
6.35
7.45
32.76
4.14
3.47
140.42
2.38
0.34
0.62
9.22
2.64
LTE
0.69
1.96
-0.60
2.37
-0.61
0.30
-0.31
0.07
-1.44
-0.96
-0.95
9.22
-0.8865
effectiveness of LTE is influenced by the skewness of temporal differences in dataset and the configurations of underlying TGNNs. Section|C..]exxy C.4eexplores the impact of the hyperparameter ? in our LHA module. Section C.5 stuusss the robustness of TAMI under different aggregation strategies and values of & in our LHA module. Section /.66///ghlihhhh that TGNNs suffer from the loss of interaction histories, which reduces their link prediction performance. Our LHA module helps mitigate this loss and improves TGNNs' performance. Section C.7 evaluates the effectiveness of LHA under conditions of limited memory.
C.1 Comprehensive Results of Table
Table pressenss the comprehensive results of Table Results show that TAMI consistently improves the link prediction accuracy of underlying TGNNs and remains effective under different negative sampling strategies.
C.2 Adaptivity of LTE and LHA to Various Types of TGNNs
Table 13 demonstrates the comprehensive results of Table in Section 4.5 also presents the link
prediction performance of TGNNs integrated with the proposed LTE and LHA, alongside their vanilla
26

---

Table 12: AP for transductive link prediction. Negative edges generated using the random (rnd), historical (hist), and inductive (ind) negative sampling strategies proposed in [21] NSS stands for negative sampling strategies. Standard deviations summarized in Table 21
NSS
Methods
CP
CO
EN
FL
LA
MO
RE
SE
UC
UT
UV
US
WK
JODIE
69.26
95.31
84.77
95.60
70.85
80.23
98.
89.89
89.43
64.94
63.91
75.05
96.50
DyRep
66.54
95.98
82.38
95.29
71.92
81.97
98.22
88.87
65.14
63.21
62.81
75.34
94.86
TGAT
70.73
96.28
71.12
94.03
73.42
85.84
98.52
93.16
79.63
61.47
52.21
68.52
96.94
TGN
70.88
96.89
86.53
97.95
77.07
89.15
98
93.57
92.34
65.03
68.72
75,99
98.45
CAWN
86.99
95.18
52.84
70.58
98.76
EdgeBank
64.55
92.58
83,53
89.35
79.29
57.97
94.86
74.95
76.20
60.41
58.49
58,39
90.37
68.67
79.70
67.27
89.57
51.90
69.59
96.47
md
GraphMixer
75.90
91.94
82.26
90 00.98
75.56
82. 83
97.33
93,34
93.35 23.38
62.61
52.20
71.55
97.23
DFormer
97.91
98.31
92.46
98.92
93.01
87.66
99.22
94.66
95.66
65.07
55.88
70.44
99.02
with TAMI
GraphMixer
78.38
95.26
96.75
88.13
83.53
98.84
62.98
57.74
71:57
98.89
Imp. (%)
3,27%
3.61%
10.59%
6.34%%
16.64%
0.85%
1.56
0.07%
3.02%
0.591
10.61%
0.03%
1.71%
Foomerr
94.03
96.72
56.02
71.40
99.25
Imp. (%)
0.78%
0.40%
0,22%
0.02%
1.10%
0.95%
0.07%
0.08%
1.11%
2.03%
0.25 96
1.36%
0.23%
JODIE
51.79
95.31
69.85 85
66.48
74.35
78:94
80.03
87,44 97.44
75.24
61.39
70.02
51.71
83.01
DyRep
63.31
96.39
71.19
67.61
74.92
75.60
79.83
93.29
55.10
59.19
69. 30
86.88
79.93
TGAT
71.59
52.96
62.14
TGN
68.42
93.05
73.91
66,70
76.87
87.06
81.22
94,45
80.43
58.44
69.37
74.00
86.86
CAWN
66.53
84.16
64.73
64.72
69.86
74.05
80.82
85.53
65.30
55.71
51.26
68.82
71.21
EdgeBank
63.84
88.81
76.53
70.53
73.03
60.71
73.59
80.57
65.50
81.32
84.89
63,20
73.35
hist
TCL.
65.93
93.86
70.66
70.68
59.30
77.06
77. 14
94,74
80.25
55.90
52.
80.53
89.05
GraphMixer
74.34
72.47
77.77
78
57.05
51.20
81.65
90.90
Dy OFormer
97.00
97.57
75.63
66.59
81.57
85.85
81.57
97.38
82.17
64.41
60.84
85.30
82.23
with TAMI
GraphMixer
78.81
93.30
81.68
73.01
80.23
83.61
82.56
96.80
87.69
69.74
70.90
84.56
90.97
Imp. (%)
6.01%)
0.01%
4.74%
2.15%
10.71%
7.51%
5.25%
1.97%
4.26/%
22.24%
38.4818
3.56%
0.08%
DyGroreer
98.96
97.72
81.02
67.77
83.40
86.26
85.18
97.56
85.89
65.16
81.72
86.10
82.38
Imp. (%)
7.13%
2.24%
4.53%
34.32%
0.18%
JODIE
48.42
93.43
68.96
69.07
62.67
65.23
86.96
89.82
65.99
60.42
67.79
50,27
75.65
DyRep
58.61
94.18
67.79
70.57
64.41
61.66
86.30
93.28
54.79
60.19
67.53
83.44
70.21
TGAT
68.82
94.35
63.94
75.48
71.13
75.95
89.59
94.84
68.67
60.61
52.89
61.91
87.00
67.63
CAWN
67.75
89.31
75.15
69.18
67.48
73.51
91.67
88.32
64.61
62.54
52.19
65.81
74.06
EdgeBank
62.16
73.89
81.08
75.49
83.69
57.43
72.97
66.30
80.63
ind
TCL
65.85
91.35
71.29
74.62
58.21
74.65
87.45
94,90
76:01
61.06
50.62
78.15
86.76
GraphMixer
69.48
90.87
75.01
74.87
68.12
74.26
85.26
94.72
80.10
60.15
51.60
79.63
88.59
GFomme
95.44
94.75
77.41
70.92
73.97
81.24
91.11
97.68
72.25
55.79
51.91
81,25
78.29
with TAMI
GraphMixer
70.94
96.12
88.95
93.64
91.06
79.82
96.19
96.09
84.12
87.73
79.53
83.31
93.89
Imp. (%)
33.68%
7.49%
54.13%
5.98%
DyGFormer
97.25
98.47
86.23
75.55
74.03
92.39
94.37
97.76
80.13
68.01
78.19
81.31
78.96
Imp. (%)
1.90%
3.93%
11.39%
6.53%
0.08%
13.72%
3.58%
0.08%
10.91%
21.90%
50.63%
0.07%
0.86%
counterparts. The results suggest that TAMI is adaptable to different types of TGNNs, consistently improving the performance of underlying models. Moreover, applying LTE and LHA individually also improves the link prediction accuracy of the integrated model.
C.3 Application Scenarios of LTE
The effectiveness of the proposed LTE is influenced by the distribution of temporal distances within the dataset. Based on the skewness of the temporal differences in the original TE, we classify the datasets into two types: (1) skewed, where small temporal differences dominate, while a considerable number of large temporal differences exist; and (2) balanced, where the temporal differences roughly follow normal distribution. As shown in Table 3 and Table IT for balanced datasets (e.g.. UV and UT), the improvements observed in GraphMixer are marginal. This is because LTE specifically designed to address the skewness of temporal differences, which is rarely presented in balanced datasets. For datasets with skewed temporal differences (e.g., CP, EN, and UC), LTE significantly improves GraphMixer's performance compared to the vanilla version with the original TE.
27

---

Table 13: AP for transductive setting. The proposed LTE and LHA be integrated into various types of TGNNs. When applied individually, both LTE and LHA improve the link prediction accuracy of the integrated models. No LTE and TAMI results are reported for JODIE because it does not utilize the time encoding function.
Method
EN
UV
87.09
1.04
75.73
+:
91.87 +
TGAT
72.89
1: 1,12
73.37
1 0.07
79.29
1 0.06
53.14
1 0.56
CAWN
88.39
+ 0.07
86.99
+ 0.01
94,97
+ 0.08
52.88
+ 0.08
JODIE
84.79
4,80
70.17
89.15
63.52
LTE
TGN
90.23 + 0.47
85.03 +
1.45
94, = 1.12
64.90
+ 0.32
3.61%
12.27%
2.51%
-1.20%
TGAT
84.75
83.99
0.01
94.85 = 0.50
54.97
+ 3.96
Imp. (%)
16.27%
13.79%
19.62%
3,45%
CAWN
90.70
0.39
88.76 0.03
52.92
E 0.77
Imp. (9)
2.61%
2.03%
1.75%
0.08%
LJHA
90.20 +:
Imp. (%)
3.57%
19.00%
1.61%
3.46%
TGAT
88.06 + 0.12
87.97 + 0.23
89.74 = 0.11
57.58 + 0.27
(%)
20.81%
19.89%
13.17%
8.37%
CAWN
90.13 + 0.01
89.29
= 0.04
95. 16 = 0.03
56.50 +: 0.54
Imp. (%)
1.96%
2.64%
0.21%
6.85%
JODIE
90.62 + 0.01
87.95 + 0.01
92.44 + 0.02
65.57 + 0.34
Imp. (%)
6.88%
25.33%
3.68%
3.22%
TAMI
92.34 +
+
+: 3.22%
1mp. (%)
6.03%
22.58%
3,80%
TGAT
91.37 = 0.04
91.60
0.13
96.36 = 0.18
60.03 + 0.34
(%)
25.35%
24.85%
21.53%
12.98%
CAWN
91.23
= 0.06
91.02 = 0.00 96.69 = 0.13
57.49 + 0.18
Imp. (%)
3.21%
4.64%
8.72%
C.4 Analysis of the Hyperparameter y in LHA
In this experiment, we study the impact of the hyperparameter 7 in our LHA module on link prediction performance. We train, validate, and test models with 7 = (0.1, 0.3, 0.5, 0.7,0.9. 1) and report the test AP in Table [1]] As shown, integrating GraphMixer into TAMI consistently improves its performance over its vanilla counterpart across all values y. addition, 7 affects link prediction accuracy differently across different datasets. For the UV dataset, the performance of GraphMixer with LHA remains similar across different values of y. When comes to the EN, LA, and UC datasets, higher values generally yield better performance. This is because larger values place more weight on recent historical interactions, while reducing the influence of older ones. These results suggest that prioritizing recent interactions is more ad advantagcous for these datasets.
Table 14: AP of GraphMixer with varying values of the hyperparameter y when integrated with our LHA module. The first and the second best performances are marked in bold and underlined respectively.
EN
LA
UC
UV
GraphMixer
82.26
75.56
93.38
52.20
0.1
90.41
85.97
95.80
58.02
57.99
= 0.5
90.97
87.94
96.22
$7.95
Y
$7.87
0.9
90.77
88.19
96.20
57.74
90.60
88.09
96.16
$7.55
28

---

C,5 Analysis of the Choice of Aggregation Strategies and the Value of l in LHA
To predict the future link between two nodes, LHA retrieves all stored k historical edge embeddings for the target node pair and summarizes them into a single vector using an aggregation function. In this experiment,
study the effectiveness of TAMI under different values of k and aggregation strategies. By default, we set h = 1 and use the most-recent aggregator. Note that our LHA module uses an exponentially weighted moving average to compute the dedicated historical edge embeddings for a given target pair of nodes, as shown in Equation Thus, even if A = 1, LHA considers all the interaction history between the pair of nodes.
In Table 15| we show the performance of TAMI with different values and aggregation functions. The mean aggregator computes the average of the k historical edge embeddings, whereas the max aggregator follows the implementation in GraphSAGE [8]. where each historical edge embedding is passed through a fully connected neural network, followed by an element-wiss max-pooling operation across all transformed vectors. Overall, the performance of different variants of TAMI is better than the vanilla version (without TAMI), indicating the effectiveness of our design. It is also important to note that larger value does not necessarily lead to better performance. This is expected as different datasets may have different levels of dependency on the interaction history for temporal link prediction. In addition, TAMI is not sensitive to the choice of aggregation functions.
Table 15: Test AP of DyGFormer on the EN and CP datasets with different values and aggregation strategies.
Method
EN
CP
DyGFormer (Vanilla)
92.46
97.91
with TAMI most-recent
92.66 98.67
mean (k 1)
92.66 98.67
mean (k- 2)
92.60 98.76
mean (k 3)
92.56 98.77
max (k 1)
92.66 98.67
max (k: = 2)
92.84 98.74
(k:
92.60 98.74
C.6 Preventing the Forgetting of Interaction Histories by LHA
In Figure 5] we demonstrate that the loss of interaction histories occurs in GraphMixer 7 and degrades its link prediction performance. Here we extend our analysis JODIE [3). JODIE adopts two RNNs to maintain an evolving temporal memory for each node and uses stored historical node states to compute temporal node embeddings. When predicting the link between two nodes, if neither node appears in the other node's 20 most recent interactions, we consider their interaction histories as no longer retained in their temporal node embeddings. Figure g presents the distribution of the appearance index along with the link prediction accuracy. The appearance index represents the earliest position at which at least one node appears in the interaction sequence of the other. For instance, an index value of 1 means that at least one node is the most recent encounter of the other, while a larger index indicates that fewer historical interactions between the nodes are retained. The term "> 20" denotes the case where neither node appears in the most recent 20 interactions of the other and their interaction histories are forgotten.
As shown in Figure g the "> 20" term has the highest percentage across all examined datasets. This suggests that the loss of historical interactions also occurs in memory- based TGNNs. On the EN, LA, and UC datasets, the AP of the vanilla JODIE model decreases as the appearance index increases, indicating that retaining fewer historical interactions leads to worse link prediction performance. In comparison, when the proposed LHA is incorporated, the AP of JODIE consistently improves across all datasets, demonstrating the effectiveness of our LHA module in mitigating the loss of historical interactions.
29

---

60
40
Percent
40
60
20
10
20
21
Average Precision
1.00
0.98
0.90
JODIE
0.95
W/ LHA.
0.96
0.94
0.90
0.55
0.92
JODIE
0.85
0.90
0.80
0.82
Appearance index over the 20 most recent interactions.
Figure 8: AP of JODIE to the number of interaction histories retained. An increase in the appearance index indicates that fewer interaction histories are retained for predicting future links.
C.7 Cold Start of LHA in Inference
LHA maintains the historical connections between nodes. It may happen that in the training set, two nodes never interact. In this experiment, we evaluate how LHA performs that case. Specifically, we introduce variant called "w/ LHA (no history before test)", where the interaction histories accumulated during the training and validation phases are not loaded when the test stage begins. In other words, interaction histories must be constructed from scratch, the test progresses. This variant is compared against the default L.HA setting, "w/ LHA," where the history accumulated during training and validation is loaded before testing. Figure iillssraate the test AP of various methods. The x-axis shows the test phase timeline, with timestamps normalized to [0, 1] for better visualization.
First, integrating LHA into GraphMixer and JODIE consistently boosts their performance throughout the testing phase, surpassing their vanilla counterparts. Second, even when LHA history is not initialized at the beginning of the test phase (i.e., in the "w/ LHA (no history before test)" variant), the augmented TGNNs still outperform their vanilla versions on the EN, LA, and UC datasets. This indicates that the LHA module provides significant performance improvements, even with limited history in the early stage of inference. As test progresses and more interaction histories are stored in the LHA, the performance of the "w/ LHA (no history before test)" variant gradually converges to that of the "w/ LHA" variant. This is expected, as both variants share increasingly similar histories, leading to similar performance over time.
GraphMixer
JODIE
Dataset history test)
EN
Dataset:
Datacet:
2.62
3.60
0.98
5.70
2..0
5.75
0.00
0.96
0.73
0.92
2.30
Average Predzion
3.03
1.00
3.08
5.85
0.90
2.82
2.70
0.50
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1:00
0.00
0.25
0.50
0.25
1.00
0.30
0.25
0.50
5.75
1.00
Elapsed since the started (normalized 0-1)
Figure 9: LHA consistently improves the performance TGNNs throughout the entire testing phase. The "w/ LHA" variant denotes that interaction histories accumulated during the training and validation phases before test)" variant starts the test phase without any interaction histories stored in the LHA memory module. Test node pairs are grouped into chronological bins, and the average link prediction accuracy for each bin is reported.
loaded the beginning of the phase. In contrast, the "w/ LHA (no history
30

---

Table 16: AP for inductive link prediction. Imp. (%) denotes the percentage of improvement. Improved results are colored in blue. The first and the second best performances are marked in bold and underlined respectively. For dataset names, please kindly refer to Section B.1
Methods
CP
CO
EN
FL
LA
MO
RE
JODIE
$3.92 + 0.94
94.34
1.45
80.67
211
94.74
0.37
78.31
+ 3,82
79.63
1.92
96.50 + 0.13
DyRop
54.02 0.76
92.18
041
14.55
=455=399
92.88
0.73
83.02
148
81.07
= 0.44
96.00
0.11
TGAT
55.18
95.87
67.33
-
88.73
78.48
+ 0.07
85,50
0.19
97.09
- 0.04
TGN
54.10 0.93
93.2
0.99
78,74
93.03
0.00
80.05
4.00
89.04
1.17
97.50
0.07
CAWN
5.69
TCL
54.30 1: 0.66
91.11
0.12
76.14
0.79
8341
73.53
1 1.66
80,60
0.22
94.09
1 0.07
GraphMixer
0.09
0.21
+
0.25
81.35
95.23
DyGFormer
88.14 0.19
98.06
0.01
89.86
0.28
97.79 + 0.01
94.20
0.17
27.24
0.35
98.83 + 0.03
with
GraphMixer Imp. (%)
0.92
94.49
0.21
0.45
92.64
0.06
90.71
0.64
81.55
0.10
97.93 + 2.84%
1.93%
4.28%
12.63%
11.62%
10.55%
0.25%
DyGioomme 0.41
0.00
89.87
0.06
97.85
95.08
88.08
0.23
+:
Imp. (%)
0.94%
0.24%
0.01% %
0.06%
0.93%
0.96%
0.08%
Methods
SE
UC
UT
UV
US
WK
JODI
91.96 + 0.48
79.27
1.97
99.65
0.77
56.86
+
0,06
54.93
2.29
94.82 = 0.20
DyRep
90.04 1. 0.47
57.48
1.87
57.02
0.69
53.62
2.22
57.28 1: 0.71
92.43
0.37
91.41
TGN
90.77 0.771.0.880 0.86
86.66
1.83
58.31
3.15
58.63
0.37
97.83
0.04
79:94
0.87
53.17
98.24
TCL
91.55 0.09
87.36
2.03
62.21
0.12
51.00
52.59
0.97
96.22
0.17
GraphMixer
91.71 0.13
91.39
0.05
62.26
0.13
50.58
0.45
50.72
0.64
96.47
0.13
DyGFomee
93.16 0.08
94.40
0:1
63.86
0.35
56.16
0.07
54.93
0.53
98.54
0.07
GraphMixer
93.03 0.46
93.44
0.29
62.40
1,50
56.63
0.36
51.44 +: 0.75
98.17 = 0.01
Imp. 1.44%
2.24%
0.22%
11.96%
1.42%
1.76%
DyGFormer 0.01
0.09
0.24
56.23
0.17
55.08
Imp. (%) 0.01%
1.35%
0.61%
0.12%
0.27%
0.35%
D Comprehensive Results for Inductive Temporal Link Prediction
This section presents comprehensive results for inductive link prediction. Section D.T demonstrates that LTE and LHA improve the link prediction performance of integrated models in the inductive setting. Section D.2 presents the results an ablation study in the inductive setting, showing that both LTE and LHA remain effective. In Section D.3 we show that LTE and LHA are robust to increasing number of negative links per positive link the inductive setting. Finally, Section D.4 shows that the proposed LTE and LHA can be integrated into various types of TGNN while enhancing their performance the inductive setting.
D.1 Main Results for Inductive Link Prediction
Table Summmmarzees the test performance of methods in the inductive setting. The results demonstrate that TAMI improves GraphMixer and DyGFormer on all the 13 datasets. This highlights the effectiveness and versatility of TAMI, suggesting TAMI can enhance link prediction performance in inductive settings.
D.2 Ablation Study for Inductive Link Prediction
In this experiment, we study the individual contributions of LTE and LHA to the performance improvements of the integrated TGNNs. Table 17 presents the test performance of TAMI and its two variants: w/ LTE, where we replace the original TE in TGNNs with the proposed LTE and keep the rest unchanged; w/ LHA, where integrate the LHA module into TGNNs and keep the rest unchanged.
First, replacing the original time encoding function with the proposed LTE improves the performance of GraphMixer on 11 datasets and the performance of DyGFormer on datasets. This suggests that LTE can enhance the link prediction performance of integrated models in the inductive setting.
31

---

Second, incorporating the LHA module improves the performance of TGNNs. In the inductive setting, even though no test edges are observed during training (i.e., no historical interactions are stored in the LHA memory at the beginning of the test phase), the LHA module still improves the performance of GraphMixer and DyGFormer on 12 datasets. This because L.HA can update its memory as the test stage progresses and improve the accuracy of subsequent link predictions. These results suggest that the LHA unit remains effective in the inductive setting.
D.3 Robustness to the Increase of Negative Links (Inductive)
In Section 4.4 we demonstrate that TAMI is robust to an increasing ratio of negative links to positive links under the transductive setting. In this experiment, we evaluate the robustmess of TAMI in the inductive setting. Table T8 presents the test performance of TGNNs under the inductive setting. The term "NEG=50" denotes that the connection probability of each positive link is evaluated against 50 negative links. The results demonstrate that the performance of underlying TGNNs consistently improves across various numbers of negative links per positive link on all examined datasets. These findings suggest that the proposed TAMI framework remains effective against the increasing ratio of negative links in the inductive setting.
D.4 Adaptivity to Different TGNN Architectures (Inductive)
In Section 4.5 and Section we demonstrate that the proposed TAMI, LTE, and LHA can enhance the link prediction performance of underlying TGNNs the transductive setting. In this experiment, we explore whether the proposed methods can also improve the performance of TGNNs in the inductive setting. 19 the performance of the vanilla TGNNs along with those enhanced with the proposed modules. No results for w/ LTE and w/ TAMI are reported for JODIE, it does not incorporate the time encoding function. The results demonstrate that LTE, LHA, and TAMI consistently improve the inductive AP of TGNNs. This suggests TAMI and the two novel modules LTE and LHA remain effective in the inductive setting and can improve the performance of various TGNNs.
E Experimental Settings the TGB Datasets
We further evaluate TAMI on three large-scale datasets from the Temporal Graph Benchmark (TGB) (9. including t2gl-wiki-v2, -gbl-review-v2, and tghl-coin-v2. Dataset statistics available https//gb..ompcccaaaalb..oo////s com//ocsslinkprop//l We apply TAMI to the recent state-of-the-ar method DyGFormer and compare its performance against the original version. For fair comparison, adopt the best hyperparameter settings of DyGFormer on the TGB benchmark, as detailed in [36]. In our LHA module, the hyperparameter y fixed 0.9 across the training, validation, and testing phases. The dimension of historical edge embeddings is equal to the dimension of temporal node embeddings. The detailed results are presented in Table 2 We also report the original skewness of temporal differences in the TGB datasets and the skewness after applying our LTE in Table 20 below. Results show that our LTE effectively reduces the skewness in the distribution the temporal differences.
F Standard Deviations in Main Text
Table 22 and Table 23 present the standard deviations of the results shown in Table [ Table 21
Table 3 and Table in the main text, respectively.
32

---

Table 17: Ablation study. AP for inductive link prediction. For dataset names, please kindly refer to Section B.1
Methods
LA
MO
GraphMixer
57.46 :: 0.11
90.61
0.09
75.94 + 0.21
83.00
+ 0.04
82.06 + 0.25
81.35 + 0.01
95.23 + 0.03
LTE
1.14 58.25
0.27 90.83
76.15 + 0.26
83.05 + 0.18
81.22
+ 0.90
81.73
+ 0.18
95.31 =
Imp. (%6)
1.37%
0.25%
0.27%
0,07%
-1.02%
0.47%
0.08%
w/ LIAA
57.30 1: 0.30
95.03
1.06
85,16 1. 0.15
92.60 1. 0.06
90.92
0.24
81.50 1 0.06
97.94 1. 0.01 2.85%
Tmp. (%)
-0.29%
4,88%
12.13%
11.57%
10.80%
0.19%
w/ TAMI 1mp. (%)
58.57 = 0.92
94,49 + 0,21
85.54 + 0.45
92.64 # 0.06
90.71 + 0.64
81.55 0.10
97.93 = 0.01 2.84%
1.93%
4.28%
12.63%
11.62%
10.55%
0.25%
DyGForeer
88.14 1: 0.19
98.06
0.01
89.86 1. 0.28
97.79 1 0.01
94.20
0.17
87.24
0.35
98.83 1. 0.03
w/ L.T.F
88.72 0.36
98.10
0.01
89.97 + 0.51
97.82 + 0.02
0.04 94.94
88.30 + 0.28
98.93 + 0.03
Imp. (%)
0.06%
0.04%
0.12%
0.03%
0.79%
1.21%
0.10%
w/ LHA Imp. (%)
89.41 0.21
98.28 + 0,03
90.36 + 0.49 0.56%
97.91 + 0.01 0.12%
94.24 + 0.08
87.45 + 0.01
98.87 + 0.13 0.04%
1.44%
0.22%
0.4%%
0.24%
w/ TAMI
88.97 + 0.41
98.30 + 0.00
89.87 + 0.06
97.85 + 0.02
95.08 + 0.13
88.08 + 0.23
98.91 + 0.00
Imp. (%)
0.94%
0.24%
0.01%
0.06%
0.93%
0.96%
Methods
SF
UC
UT
UV
US
WK
GraphMixer
91.73 = 0.13
91.39 = 0.05
62.26
0.13
50,58
0.45
50.72 + 0.64
96.47 + 0.13
w/ LTE
91.81 + 0.11
92.07 + 0.14
61.92
0.37
50.76 = 0.34
50.87 + 0.21
96.58 + 0.13
Imp. (S)
0.09%
0.75%
-0.54%
0.37%
0.30%
0.11%
L.HA Tmp. (%6
91.91 +:
+:
63.05
0.83
55.37
0.22
50.99 +
0.01
0.20%
1.59%
1,28%
9.47%
0.53%
1,70%
w/ TAMI Imp. (%)
93.03 1: 0.46
93.44 1: 0.29
62.40
1.50
56.63 =: 0.36
51.44 1 0.75
98.17 0.01
1.44%
2.24%
0.22%
11.96%
1.42%
1.76%
DyGioome
93.16 + 0.08
94.40 + 0.21
63.86
0.35
56.16 = 0.07
54.93 + 0.53
18.54-0 +: 0.07
w/ LTE
93.21 + 0.10
95.62 +
64.94
0.08
56.27
0.16
54.06 + 1.03
86 0.01
Imp. (56)
0.05%
1.29%
1.69%
0.20%
-1.58%
0.32%
LHA
94,49 + 0.01
94.62 0.01
63.65
0.18
56,20 = 0.47
55.85 85 + 0.20
98 0.06
Imp. (%6
1.42%
0.23%
-0.33%
0.07%
1.67%
0.09%
DyGFamer
93.17 + 0.01
95.67 + 0.09
64,25
0.24
56.23 = 0.17
55.08 +: 0.21
98.88 0.07
Imp. (%)
0.01%
1.35%
0.61%
0.12%
0.27%
0.35%
Table 18: Inductive link prediction performance. AP of methods under various numbers of negative links during testing. NEG=50 indicates that each positive link is evaluated against 50 negative links in the AP computation. For dataset names, please kindly refer to Section B.I
Methods
EN
LA
UC
UV
Methods
EN
UC
UV
=
GrighMiser
75.94 4: 0.21
82.06 d: 0.25
91.39 à 0,05
50.58 0.45
DyCFommer Imp. (95)
89.36 +: 0.28 94,20 d: 0.17
94.40 4: 0.21
56.16 0.07
TAMI
85.54 + 0.45
90.71
+ 0.64
93.44
0.29
56633
0.06
95.08
0.13
95.67 +
0.09
56.23
Imp. (96)
120056
10.35%
2.24%
11.90%
0.1216
OraphMixer
1: 0.35
58.37
0.69
77.20
0,05
17.38
DyCFomeer TAMI Imp. (%)
66.38 1: 0.33
1: 0.52
83.86
0:45
21.26
TAMI
57.22
0.37
72.63 +
81.75 = 0.74
20.89 0.21
67.31 =
84.37 = 0.50
88.02 + 0.28
21.
Imp. (96
36.91%
24.45% 3.90%
20.20%
0.64%
3.01% 2.51% 1.13%
NEG 25
GraptMiser
15.16 +: 0.33
33.93
+ 1:25
55.31 4: 0.21
4.14 4: 0.22
DyGmmmeer
34,06 + 0.24
58.96
de 1.15
71.39 4: L.73
5.31 + 0.13
w/ TAMI
24.19 4 0.09
45.18 + 4.36 60:48 + 1.58
5.09 + 0.08
w/ TAMI Imp.
35.85 + 0.04
63.78 = 2.10
75.52 + 0.51
5.43 + 0.11
Imp.
59.58%
33.18%
9.33%
22.9%%
3.42%
8.18%
5.00%
2.2618
NEG
OraphMiser TAMI
887-11 26.85 1: 1.30
45.00 4: 0.51
2.15 4 0.15
DyCFvmmer W/TAMI 23.72 1 0.06
22.53 1 0.04
46.74 : 1.36
62.61
4: 3.26
2.76 1: 0.08
14.50
1 4.48
2.18
2.64 1 0.06
1.61
67.75
0.70
2.79 1 0.05
Imp. (9)
63.425
35.45%
10.0%
22.54%
Imp (5)
5.28%
14.39%
8.11%
1.09%
33

---

Table 19: AP for inductive link prediction. The proposed LTE and LHA can be integrated into various types of TGNN. No LTE and TAMI results are reported for JODIE because it does not utilize the time encoding function.
Method
EN
LA
UC
UV
TGN
78.74 + 1.53
80.05 +
86.66
1.83
54.13 =
2.37
TGAT
67.63
+ 0,96
78.48 + 0.07
79.11
0.28
51.98
0.14
CAWN
84.97
0.21
89.48 1 0.02
92.45
0.06
49.29
0.87
JODIE
80.67
2.11
78.31
+ 3.82
79.27
1.97
56.86
0.06
w/
TGN
84.02 + 1.63
89.33 + 0.87
91.36 = 1.41
57.92 3.41
Imp. (%)
5.66%
TGAT
77.70 + 0.34
88.27 + 0.05
92.61
0.29
50.03 0.31
Imp. (%)
14.89%
12.4%%
17.07%
-3.75%
CAWN
86.88
0.25
Tmp. (%)
2.25%
1.86%
2.37%
-1.51%
LHA
TGN
85.97
92.35 +
3.58
Imp- (%)
9.18%
15.37%
3.34%
5.79%
83.89
0.01
90.26 +
1.02
Imp. (5)
24.04
15.01%
11.4%
4.69%
CAWN
86.24 +: 0.15
91.03 +: 0.06
92.74 0.06
55.62 1.66
Imp. (56)
1.49%
1.73%
0.31%
12.84%
JODIE
85.48 + 0.24
91.71 # 0.56
83.56
0.08
$7.64 1.41
Imp. (%)
5.97%
17.11%
5.42%
1.37%
w/ TAMI
TON
85.54 +: 0.45
90.71 + 0.64
93.44 = 0.29
$7.63 0.36
Imp. (96)
8.64%
13.32%
7.82%
6.47%
TGAT
84.53
+ 0.13
93.32 + 0.20
94.25 0. 10 19.15%
54,29 :: 0.30
Imp. (56)
24.98%
18.91%
4,44%
CAWN
87.45 + 0.23
92.61 +: 0.04
94.69 = 0.11
64.33 +: 1.37
Imp. (%)
2.42%
Table 20: Skewness of temporal differences before and after applying our LTE on the three TGB datasets tested.
Datasets
tghl-wiki
tgbl-review
tgbl-coin
Originally
4.181
3.297
4.562
with LTE
-0.307
-0.524
0.308
34

---

Table 21: The standard deviations of five runs for results in Table and Table I2
NS8
Methods
CP
C0
EN
FL
LA
MO
RE
SB
UC
UT
UV
US
WK
JODIE
0.31
1.33
0.30
1.73
2.13
2.44
0.14
0.55
1.09
0.31
0.81
1.52
0.14
DyRop
2.76
0.15
3.36
0.72
2.21
0.49
0.04
0.30
2.30
0.33
0.80
0.39
0.06
TGAT
0.06
3.97
1.37
0.06
CAWN
2.34
0.28
0.09
0.01
0.06
0.25
0.01
0.09
0.06
0.12
0.10
0.48
0.03
EdgeBank
0.00
0.00
0.00
0.00
0.30
0.00
0.00
0.00
0.00
0.00
0.00
0100
0:00
TCT.
GanpliMixer
0.02
0.01
0.16
0.13
DyGieormer
1.89
0.08
0.50
0.01
1.53
0.17
0.01
0.11
1.15
0.02
0.02
0.34
0.11
w/ 'AAI
GrngliMixer
0.06
0,41
0,49
DyGiFormer
0.12
0.01
0.21
0.00
0.08
0.14
0.00
0.015
0.21
0.04
0.03
01.28
0.01
JODIE
0.63
2.13
2.70
2.59
3.81
1.25
0.36
6.78
5.80
1,83
0.81
5.76
0.66
DyRep
1.23
0.20
2,76
0.99
2,45
1.12
0.31
10445
3.14
1.07
1.12
2.25
0.56
TGAT
1,05
0.34
0.20
0.44
5.51
CAWN
2.77
0.49
0.36
0.97
0.43
0.95
0.45
0.38
0.49
0.38
0.04
8:23
1.67
EdgeBank
0.00
0.00
0.00
0.00
0.00
0.00
0.00
1.000
0.00
0.00
0.00
0.00
0.00
hist
TCL
3.00
0.21
0.39
2.31
0.16
2.74
1.17
2.35
3.95
0.39
GrapliMiter
0,49
1.22
DyGierrmrr
0.31
0.00
0.25
0.49
0.48
0.66
0.67
0.14
0.82
1.40
1.58
3.38
2,56
w/ ''AAII
GrapliMixes
0:03
2,15
DyGieormer
0.01
0.02
0.01
2.93
1.84
0.89
0.30
0.04
0.32
1,46
2.12
2.50
0.24
JODIE
0.66
1.78
0.98
4.02
4.69
2.19
0.16
4.11
1.40
1.48
1.48
5.13
0.79
DyRep
0.86
0.10
1,53
1.82
2,70
0.95
0.26
0.48
1.76
1.24
1.24
1.16
1.58
TGAT
1.36
0122
0.44
1.24
CAWN
1.00
0.27
0.58
1.52
0.77
0.24
0.27
0.48
0.67
0.67
8.52
2.62
EdgeBaak
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
TCL
1.75
0.21
0.32
0.18
0.89
0.54
0.29
0.36
1.74
1.74
3:34
0.72
GraphiMixer
DyGierrmer
0.57
0.28
0.89
1.78
0.50
0.69
0.40
0.10
1.71
1,02
1.02
3.62
5.38
w/
GraptiMixer
DyGieormer
0.09
0.06
1.39
2.82
1,03
1.38
0.21
0.21
0.35
3.27
1.48
2.50
0.08
Table 22: The standard deviations of five runs for results in Table
Methods
CP
CO
EN
FL
MO
UC
UV
US
WK
GraphMiser
0.46
0.09
0.06
0.01
0.02
0.04
0.01
0.16
0.05
0.13
0.09
0.23
0.04
LTE
W/LHA
0.60
0.04
0.03
0.19
0.06
0.18
0.17
0.00
w/ TAMI
0.07
0.06
0.06
0.01
0.41
0.04
0.04
0.11
0.21
0.29
0.08
0.26
0.00
DyGFemmer
0.26
0.00
0.04
0.01
0.19
0.24
0.01
0.08
0.30
0.94
0.32
0.91
0.05
W/L.E.
0.07
0.01
0.06
0.00
0.00
0.11
0.01
0.01
0.22
0.01
0:45
0.69
0.02
W/LHA
TAMI
0.12
0.01
0.21
0.00
0.14
0.00
0.03
0.21
0.04
0.03
0.28
0.01
Table 23: The standard deviations of five runs for results Table
Method
EN
UC
UV
Method
EN
UC
UV
NEG
GraghMiser
0.06
0.02
0.05
0.09
DyCFormer
0.04
0.19
0.30
0.32
TAMI
NEG
GraghMixer
0.39
0.23
0.09
0.16
DyGRommer
0.07
0.71
0.69
0.36
GraghMiser
0.20
0.53
0.27
0.08
DyCFermer
0.0
1.46
1.18
0.19
1.19
0.52
GrnghMiieer
0.16 0.90
DyGRomer TAMI
0.06
TAMI
1,73
0.59
0.53
0.11
35


Reviews:
[{"review_id": "1s0udiIOnC", "reviewer": "Reviewer_btob", "summary": "This paper addresses limitations in existing time encoding methods for dynamic graphs by analyzing the statistical distribution of temporal interactions, showing that they follow a Pareto distribution. The authors argue that commonly used sinusoidal time encoding functions are biased towards frequent interactions and fail to capture long-range, infrequent ones effectively. Furthermore, they identify a limitation in the practice of using only the most recent $m$ interactions for node embeddings, which tends to neglect important historical context between node pairs. To address these issues, the paper introduces two key contributions: (1) Log Time Encoding (LTE), which applies a logarithmic transformation to rescale time differences, making temporal patterns easier to learn; and (2) Link History Aggregation (LHA), which preserves a fixed number of most recent interactions per target node pair to ensure rare but relevant past links are retained during link prediction. These design choices are theoretically motivated and empirically evaluated to demonstrate improved performance.", "questions": "1. [1] has found simple linear time encoding functions to be more effective than the sinusoidal function used in your work. Can the linear time encoding function benefit from your module?\n2. LHA module maintains historical interactions for a target node pair that may be missed when performing temporal node message passing. Why does this approach beat RNN-like methods (e.g., TGN) that can also track very long distance interactions? How does this method compare to DyG2Vec [2] which also uses that past $k$ interactions for a target node pair to build temporal embeddings?\n3. Does the choice of aggregation function for LHA affect performance?\n\n[1] Chung et al. Between Linear and Sinusoidal: Rethinking the Time Encoder in Dynamic Graph Learning. TMLR\n\n[2] Alomrani et al. DyG2Vec: Efficient Representation Learning for Dynamic Graphs. TMLR", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Extensive experimental setup across several dynamic graph models (e.g., DyGFormer, TGN, etc), datasets, and settings (inductive, transductive).\n2. The proposed LTE module can be easily integrated into any dynamic graph encoder. Experiments show that it almost never hurts performance.\n3. Improvement is more pronounces when increasing the number of negative edges.\n\nWeaknesses:\n1. No experiments on other dynamic graph tasks (e.g., dynamic node classification).\n2. Less improvement in the inductive setting and for balanced datasets where the temporal differences roughly follow a normal distribution.\n3. The LTE and LHA modules have minor improvements for some models (e.g., DyGFormer). No provided intuition as to why this occurs.", "was_revised": true, "final_justification": "The authors have conducted all the ablation and analysis experiments that I asked for.", "rebuttal": "We thank the reviewer for recognizing our design choices as **theoretically motivated** and our experiments as **extensive**. Below we respond to the concerns raised.\n\n- **W1. No experiments on other dynamic graph tasks (e.g., dynamic node classification)**\n\nThanks for the comment. We would like to clarify that our TAMI framework is specifically designed for the temporal link prediction task, which is the main downstream task in TGNNs. We observe that temporal interactions are heterogeneous, e.g., a few node pairs can make most interaction events, and interaction events happen at varying intervals. This can significantly degrade the link prediction performance of existing TGNNs, especially when predicting the links for infrequently interacting node pairs. Thus, we develop the TAMI framework to mitigate the heterogeneity issue in temporal interactions and in turn improve the link prediction accuracy. While it is beyond the scope of this work to extend TAMI to other dynamic graph tasks, such as dynamic node classification, we believe it is a promising direction for our future work.\n\n- **W2. Less improvement in the inductive setting and for balanced datasets where the temporal differences roughly follow a normal distribution.**\n\nThanks for the thoughtful comment. For the inductive setting, it is similar to the case discussed in the section Cold Start of LHA in Inference in Appendix C.6. Specifically, the target node pairs in testing may not be observed during training. Hence, LHA's memory may need to be constructed from scratch during the test phase. At the beginning of the test phase, LHA contains no historical information for some node pairs, limiting its effectiveness and leading to smaller performance gains. However, as the test phase progresses and more interaction events are accumulated, LHA becomes more and more effective, resulting in higher performance gains.\n\nIn addition, as shown in Table 11 in Appendix B.5, most of the datasets exhibit skewness (or heterogeneity in temporal interactions). TAMI is designed to tackle the problems associated with this skewness. Hence, when integrated into TAMI, the performance of underlying TGNNs improves substantially for most cases. For few balanced datasets, the performance gain from TAMI may not be as significant as the one with skewed datasets, but TAMI still enables underlying TGNNs to achieve better performance than their vanilla counterparts. This is somewhat expected because the skewness is mild in the balanced datasets. Nonetheless, many real-world datasets exhibit high skewness.\n\n- **W3. The LTE and LHA modules have minor improvements for some models (e.g., DyGFormer). No provided intuition as to why this occurs.**\n\nThanks for the comment. DyGFormer is the top performer in all the baselines. It achieves state-of-the-art performance by leveraging the attention mechanism across almost all (up to 99%) historical interactions of a node to learn its node embedding. However, this comes at the cost of high GPU memory consumption and slow convergence speed. As demonstrated in Section 4.6, when integrating DyGFormer into our TAMI framework, we are not only able to reduce its GPU memory usage (by 53.22%) and improve its convergence speed (by 40.54%) by leveraging much fewer (16$\\times$ fewer) historical interactions of a node, but still improve the performance compared with its vanilla version. This is thanks to LTE, our effective time encoding function to mitigate the skewness of the temporal differences, and LHA for explicit encoding of interaction history of the target node pairs in temporal link prediction.\n\nWe also present the ablation study for the historical (hist) and the inductive (ind) negative sampling strategies in the following table. The results confirm the effectiveness of both modules.\n\n**Table 1: Test AP for transductive link prediction.**\n| NSS  | Methods    | CP    | EN     | LA     | UC     |\n|------|------------|-------|--------|--------|--------|\n| hist | DyGFormer  | 97.00 | 75.63  | 81.57  | 82.17  |\n|      | + LTE      | 98.94 | 76.67  | 81.76  | 83.46  |\n|      | + LHA      | 97.63 | 80.99  | 81.96  | 84.35  |\n|      | **+ TAMI** | **98.96** | **81.02** | **83.40** | **85.89** |\n| ind  | DyGFormer  | 95.44 | 77.41  | 73.97  | 72.25  |\n|      | + LTE      | 96.46 | 77.54  | 73.89  | 77.40  |\n|      | + LHA      | 97.16 | 86.12  | 75.02  | 76.86  |\n|      | **+ TAMI** | **97.25** | **86.23** | **74.03** | **80.13** |\n\n- **Q1. [1] has found simple linear time encoding functions to be more effective than the sinusoidal function used in your work. Can the linear time encoding function benefit from your module?**\n\nThanks for the comment. Yes, the linear time encoding function proposed in [1] can also benefit from our LTE module. Although the linear time encoding function differs from the sinusoidal one, it still takes the raw temporal difference $\\Delta t$ as input. We observe that these temporal differences are often right skewed, and TGNNs trained on the skewed input struggle with the link prediction for target node pairs that interact intermittently. By applying the logarithmic transformation to $\\Delta t$ using LTE, the skewness of $\\Delta t$ can be reduced, resulting in a more balanced input distribution. We have also conducted extra experiments using the linear time encoding function from [1] with LTE. Results in Table 2 below show that with LTE, the performance consistently improves.\n\n**Table 2: Test AP of TGNNs using the linear time encoding function from [1] and the enhanced version with our LTE module. Negative links are generated using the random negative sampling strategy.**\n|Method \\ Datasets|uci (UC) |enron (EN)|\n|-|-|-|\n|TGAT (Linear)| 95.41 | 82.31 |\n|**TGAT (Linear + LTE)**| **95.90** | **84.07** |\n|DyGFormer (Linear)| 96.00 | 93.29 |\n|**DyGFormer (Linear + LTE)**| **97.14** | **93.59** |\n\n[1] Chung et al. Between Linear and Sinusoidal: Rethinking the Time Encoder in Dynamic Graph Learning. TMLR\n\n- **Q2. LHA module maintains historical interactions for a target node pair that may be missed when performing temporal node message passing. Why does this approach beat RNN-like methods (e.g., TGN) that can also track very long distance interactions?**\n\nThanks for the comment. In real-world temporal graphs, each node often interacts with a large number of other nodes, resulting in a long interaction history, or a long sequence, per node. For instance, in the UV dataset, each node interacts with over 5,000 nodes on average, forming a per-node sequence of average length longer than 5,000. However, RNNs are usually not able to track such a long sequence. Furthermore, RNN-based TGNNs still learn each node's temporal embedding based on its recent interactions. Hence, infrequent interactions are still likely to be forgotten. This forgetting problem is also empirically observed in RNN-based TGNNs, as detailed in Appendix C.5. In contrast, our LHA module explicitly encodes the historical interactions between each target node pair using dedicated historical edge embeddings and leverages these edge embeddings together with the node embeddings for the link prediction of each target node pair, thereby mitigating the forgetting problem and improving the link prediction performance.\n\n- **Q2 (cont'd). How does LHA compare to DyG2Vec [2] which also uses that past interactions for a target node pair to build temporal embeddings?**\n\nThanks for the comment. We respectfully disagree that DyG2Vec utilizes the past interactions between a target node pair to build temporal embeddings. Instead, DyG2Vec, similar to other TGNNs such as GraphMixer and DyGFormer, learns temporal node embeddings based on each node's individual interaction history. Suppose we are to predict the link between node $u$ and node $v$. In DyG2Vec, it considers the $m$-hop neighbors (i.e., past interactions in $m$ time steps) of $u$ to learn its node embedding, together with the edge embeddings from these $m$-hop neighbors. Similarly for node $v$. However, the resulting node embeddings may not capture the interaction between $u$ and $v$, as $v$ may not be in the $m$-hop neighborhood of $u$ (and vice versa). In contrast, our LHA directly encodes the past interactions between $u$ and $v$ using dedicated historical edge embeddings, and leverages this edge embedding explicitly for the link prediction between them.\n\n[2] Alomrani et al. DyG2Vec: Efficient Representation Learning for Dynamic Graphs. TMLR\n\n- **Q3. Does the choice of aggregation function for LHA affect performance?**\n\nThanks for the comment. We have conducted extra experiments using different aggregation functions with varying values of $k$, where $k$ is the number of historical edge embeddings used in Equation (5) of our manuscript. The results presented in Table 3 below show that LHA is robust to the choice of aggregation functions.\n\n**Table 3: Test AP of DyGFormer on the EN and CP datasets with different $k$ values and aggregation strategies.**\n| Method | EN | CP |\n|-|-|-|\n| DyGFormer (Vanilla) | 92.46 | 97.91 |\n| w/ TAMI             |       |       |\n| Most-recent         | 92.66 | 98.67 |\n| Mean ($k=1$)        | 92.66 | 98.67 |\n| Mean ($k=2$)        | 92.60 | 98.76 |\n| **Mean ($k=3$)**        | 92.56 | **98.77** |\n| Max ($k=1$)         | 92.66 | 98.67 |\n| **Max ($k=2$)**         | **92.84** | 98.74 |\n| Max ($k=3$)         | 92.60 | 98.74 |\n\n- **Possible typo in figure 4: According to your notation $t5$ should be the oldest timestep while $t1$ is the latest timestep but the figure suggests otherwise**\n\nThanks for pointing this out. We have updated accordingly."}, {"review_id": "9dJWJkAjJ1", "reviewer": "Reviewer_yD2r", "summary": "This manuscript focuses on the problem of link prediction over temporal graphs. The authors first identify the observation that node interactions in temporal graphs are highly skewed in terms of frequency and interval, which negatively impacts the performance of existing temporal link predication methods. To address this issue, a light TAMI framework is proposed, consisting of two components LTE and LHA. LTE alleviates the skewness in node interactions, while LHA addresses the forgetting of historical interactions. Extensive experiments demonstrate that TAMI significantly enhances the performance of TGNNs on link prediction, while also accelerating convergence and reducing GPU memory usage.", "questions": "I hope the authors could provide a clear response to the weaknesses mentioned above, especially point (O1) and (O2).", "limitations": "none", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "This manuscript has the following strengths.\n\n(S1) The paper is well organized, with clear motivation for the LTE and LHA, effectively supported by Figure 3 and Figure 5, respectively.\n\n(S2) The TAMI framework is light-weight and can be easily implemented and incorporated with various backbone TGNNs.\n\n(S3) TAMI demonstrates effectiveness for both transductive and inductive link prediction.\n\nNevertheless, there are also several weaknesses in this manuscript that should be further improved.\n\n(O1) In Section 3.3, the authors state that LHA uses most-recent aggregation. This effectively renders the parameter k redundant, as only a single historical interaction is considered regardless the value of k. Intuitively, using a larger k should enhance the performance of TAMI by incorporating more historical context. Therefore, additional experiments should be conducted with varying values of k, as well as alternative aggregation functions such as sum, which would make practical use of multiple historical interactions.\n\n(O2) Some experimental results require more detailed interpretation. As shown in Table 11, datasets CO, LA, and SE exhibit higher skewness. However, Table 1 shows that the performance improvement obtained from TAMI on these datasets are relatively modest. Notably, the SE dataset, despite having a skewness over 140, shows less than 2% performance improvement, which is significantly lower than the improvements observed on other datasets. This finding appears to contradict the motivation that TAMI should be especially effective on highly skewed datasets. The authors should provide a deeper explanation for this discrepancy. Besides, the skewness of TGB generated datasets should also be reported.\n\n(O3) Similarly, in Figure 5, the performance improvement achieved by LHA on exclusive nodes for datasets EN and UC is smaller than improvement achieved on isolated and mutual nodes. Intuitively, one would expect LHA to yield the most significant improvement on exclusive nodes. The authors should provide additional insights or analysis to clarify why LHA appears less effective on exclusive nodes in EN and UC compared to the other two datasets.", "was_revised": true, "final_justification": "Thank you for the author's submission. Appreciate the authors’ efforts and contributions. The final rating reflects a comprehensive evaluation of the paper’s strengths and limitations.", "rebuttal": "We thank the reviewer for recognizing our work with **clear motivation** that is **effectively supported**, our framework as **light-weight** and **easily implemented**. Below we respond to the concerns raised.\n\n- **(O1) In Section 3.3, the authors state that LHA uses most-recent aggregation. This effectively renders the parameter k redundant, as only a single historical interaction is considered regardless the value of k. Intuitively, using a larger k should enhance the performance of TAMI by incorporating more historical context. Therefore, additional experiments should be conducted with varying values of k, as well as alternative aggregation functions such as sum, which would make practical use of multiple historical interactions.**\n\nThanks for the comment. We respectfully disagree that only a single historical interaction is considered. We use an exponentially weighted moving average as in Equation (4) of the manuscript to compute the dedicated edge embedding for a given target pair of nodes. Thus, even if $k=1$, LHA considers all the interaction history between the pair of nodes. Varying $k$ values only controls how much freshness of information for the pair of nodes we maintain with a higher weight in the exponentially weighted moving average.\n\nIn Table 1 below, we show the results of additional experiments to evaluate the performance of TAMI with different $k$ values and aggregation functions. Overall, the performance of different variants of TAMI is better than the vanilla version (without TAMI), indicating the effectiveness of our design. It is also important to note that larger $k$ value does not necessarily lead to better performance. This is expected as different datasets may have different levels of dependency on the interaction history for temporal link prediction. In addition, TAMI is not sensitive to the choice of aggregation functions.\n\n**Table 1: Test AP of DyGFormer on the EN and CP datasets with different $k$ values and aggregation strategies.**\n| Method | EN | CP |\n|-|-|-|\n| DyGFormer (Vanilla) | 92.46 | 97.91 |\n| w/ TAMI             |       |       |\n| Most-recent         | 92.66 | 98.67 |\n| Mean ($k=1$)        | 92.66 | 98.67 |\n| Mean ($k=2$)        | 92.60 | 98.76 |\n| **Mean ($k=3$)**        | 92.56 | **98.77** |\n| Max ($k=1$)         | 92.66 | 98.67 |\n| **Max ($k=2$)**         | **92.84** | 98.74 |\n| Max ($k=3$)         | 92.60 | 98.74 |\n\n- **(O2) Some experimental results require more detailed interpretation. As shown in Table 11, datasets CO, LA, and SE exhibit higher skewness. However, Table 1 shows that the performance improvement obtained from TAMI on these datasets are relatively modest. Notably, the SE dataset, despite having a skewness over 140, shows less than 2\\% performance improvement, which is significantly lower than the improvements observed on other datasets. This finding appears to contradict the motivation that TAMI should be especially effective on highly skewed datasets. The authors should provide a deeper explanation for this discrepancy.**\n\nThank you very much for this insightful comment. We respectfully argue that the performance gains achieved by TAMI on the **CO** and **LA** datasets are satisfactory. On the **LA** dataset, GraphMixer with TAMI improves by up to 16.64%, while DyGFormer with TAMI improves by up to 2.24% among all the three negative sampling strategies. Likewise, on the **CO** dataset, GraphMixer with TAMI improves by up to 5.77%, while DyGFormer with TAMI improves by up to 3.93%. These results all demonstrate the effectiveness of TAMI.\n\nFor **SE (SocialEvo)**, we have looked into the dataset carefully and found that the high skewness is caused by a two-mode distribution. As shown in Table 2 below, one mode takes almost all the temporal differences while the other mode consists of only few ones (0.002%). There are no values between the two modes. This is due to the nature of the dataset. It is a mobile phone proximity network that captures daily interactions among students living in a dormitory over eight months. Temporal edges are formed between two students when they interact with each other via mobile phones. Hence, most interactions occur daily, but there are a few interactions that take place after more than two months, probably summer breaks, leading to ultra large temporal differences compared with the others. Even in this extreme case, TAMI can still bring an improvement by balancing the distribution, again showing the effectiveness of our design.\n\nWe hope this clarifies and thanks again for the reviewer's meticulous review.\n\n**Table 2: The distribution of temporal differences on the SE dataset exhibits a bimodal pattern, with 99.998\\% of the temporal differences being less than 19.67 days.**\n| Temporal differences |&nbsp; [0.0, 19.67) days |&nbsp; [19.67, 75.23) days |&nbsp; $\\ge$ 75.23 days |\n|-|-|-|-|\n| Ratio  | &nbsp;  99.998\\% | &nbsp; 0 | &nbsp; 0.002\\%  |\n\n- **(O2) (cont'd). Besides, the skewness of TGB generated datasets should also be reported.**\n\nThanks for the suggestion. We report the original skewness of temporal differences in the TGB datasets and the skewness after applying our LTE in Table 3 below. Results show that our LTE effectively reduces the skewness in the distribution of the temporal differences.\n\n**Table 3: Skewness of temporal differences before and after applying our LTE on the three TGB datasets tested**\n| Datasets    | tgbl-wiki | tgbl-review | tgbl-coin |\n|-------------|-----------|-------------|-----------|\n| Originally  | 4.181     | 3.297       | 4.562     |\n| with LTE    | -0.307    | -0.524      | 0.308     |\n\n- **(O3) Similarly, in Figure 5, the performance improvement achieved by LHA on exclusive nodes for datasets EN and UC is smaller than improvement achieved on isolated and mutual nodes. Intuitively, one would expect LHA to yield the most significant improvement on exclusive nodes. The authors should provide additional insights or analysis to clarify why LHA appears less effective on exclusive nodes in EN and UC compared to the other two datasets.**\n\nThanks for the comment. The performance improvement achieved by LHA on exclusive nodes is smaller than that on isolated and mutual nodes in the **EN** and **UC** datasets. This is because most exclusive target node pairs have no interaction prior to prediction, resulting in empty LHA memory at prediction time, as shown in Table 4 below. For example, 89.67% of the exclusive target node pairs in the **UC** dataset have not interacted with each other during training. This is possible as the edges in the **UC** dataset are formed by messages sent between students on some social media. Many students may only interact once if they are not familiar with each other, which is used for their link prediction in testing. Without historical information stored, LHA cannot contribute to these predictions, leading to limited or no performance gains. However, all isolated and mutual node pairs have interacted at least once during training, allowing LHA to store and utilize historical interactions between target node pairs and thus improve performance.\n\nThe case for exclusive node pairs from the **UC** dataset can also be treated as a special case of the cold start for LHA, which is studied in Appendix C.6 in the manuscript. As shown in Figure 9 in the manuscript, if we have more interaction events between target node pairs, the performance gain will be larger.\n\n**Table 4: Proportion of node pairs that *do not* have any interaction in training set**\n| Dataset | EN     | UC     | LA     | UV  |\n|-|-|-|-|-|\n| Exclusive node pairs | 41.55% | 89.67% | 37.31% | 0% |\n| Isolated node pairs  | 0%    | 0%    | 0%    | 0% |\n| Mutual node pairs    | 0%    | 0%    | 0%    | 0% |"}, {"review_id": "OebrgzVnw3", "reviewer": "Reviewer_PJpx", "summary": "The paper addresses the challenge of temporal graph link prediction by identifying heterogeneity in interaction patterns, such as skewed interaction frequencies and irregular time intervals, which hinder effective temporal information encoding and cause forgetting of past interactions. It proposes TAMI, a novel framework with two components: Log Time Encoding, which balances skewed time intervals using logarithmic transformation, and Link History Aggregation, which prevents forgetting historical interactions for node pairs.TAMI integrates seamlessly with existing temporal graph neural networks, enhancing their performance. Extensive experiments on 13 classic datasets and 3 TGB datasets demonstrate that TAMI significantly improves link prediction accuracy and training efficiency.", "questions": "In Section 4.6, the authors state that “UC and EN datasets require only 2 and 32 historical interactions for attention calculation,” indicating that the choice of the number of historical interactions (k) varies across datasets. Is this variation related to dataset characteristics such as sparsity, scale, or interaction depth?", "limitations": "The authors’ contributions focus on improving temporal encoding and information aggregation, offering a novel perspective and effectively enhancing existing models. However, the approach lacks methodological innovation, resulting in somewhat limited overall contributions.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\nThe paper is well-organized, providing a clear and comprehensive overview of Temporal Graph Link Prediction. It features precise formula definitions, well-formulated problem hypotheses, rigorous mathematical proofs, and extensive comparative and ablation experiments. The narrative is logical and coherent, ensuring a robust structure. Key strengths include being the first to identify heterogeneity in continuous-time temporal graph interactions and its impact on link prediction performance. The proposed TAMI framework includes Log Time Encoding, which rescales time differences via logarithmic transformation, and Link History Aggregation, which preserves historical interactions for target node pairs. Existing temporal graph neural networks can be seamlessly integrated into TAMI, enhancing their effectiveness.\n\nWeaknesses\nIn Figure 4, when describing historical interactions between node pairs, node v5 appears twice as an interaction node for node u, which may be a typographical error and warrants author clarification. The paper does not discuss limitations or shortcomings of the proposed method. Although the authors conducted thorough experiments, they did not clarify whether the hyperparameter in Equation (4), used to measure the forgetting rate of historical interactions, varies across different datasets; if so, further explanation is needed.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for recognizing our paper with **precise formula definitions, well-formulated problem hypotheses, rigorous mathematical proofs, and extensive comparative and ablation experiments** as well as **being the first to identify heterogeneity in continuous-time temporal graph interactions and its impact on link prediction performance**. Below, we respond to the concerns raised.\n\n- **W1. In Figure 4, when describing historical interactions between node pairs, node v5 appears twice as an interaction node for node u, which may be a typographical error and warrants author clarification**\n\nThanks for pointing this out. We will update the figure to use two links, instead of two nodes, to represent the two interactions between node $u$ and node $v_5$ for better clarity.\n\nAs you may have noticed, our intention behind Figure 4 was to show that a target node may engage in frequent interactions with a small set of nodes, dominating its recent interaction history. For example, a student may have hamburgers on both Monday and Tuesday, resulting in two distinct links between the \"student\" node and the \"hamburger\" node, each associated with a different timestamp. In this case, node $u$ frequently interacts with node $v_5$ at timestamps $t_3$ and $t_4$. As a result, historical interactions between the target node pair, i.e., node $u$ and node $v$, may be forgotten in their latest temporal node embeddings, thereby leading to degraded link prediction accuracy.\n\n- **W1 (cont'd). The paper does not discuss limitations or shortcomings of the proposed method**\n\nThanks for the comment. We would like to point out that the limitations are discussed in Appendix C.3.\n\n- **W1 (cont'd). Although the authors conducted thorough experiments, they did not clarify whether the hyperparameter in Equation (4), used to measure the forgetting rate of historical interactions, varies across different datasets; if so, further explanation is needed.**\n\nThanks for the comment. We would also like to point out that the setting of $\\gamma$ in Equation (4) is mentioned in the section Implementation Details in Appendix B.3. We set $\\gamma = 0.9$ for most experiments. In addition, we examine the performance of underlying TGNNs with different values of $\\gamma$ and present the results in Table 14 under the section Analysis of the Hyperparameter $\\gamma$ in LHA in Appendix C.4.\n\n- **Q1. In Section 4.6, the authors state that \"UC and EN datasets require only 2 and 32 historical interactions for attention calculation\", indicating that the choice of the number of historical interactions (k) varies across datasets. Is this variation related to dataset characteristics such as sparsity, scale, or interaction depth?**\n\nThanks for the comment. We would like to clarify that the number of historical interactions discussed here is not the same as the number of **historical edge embeddings ($k$)** stored and used in our LHA module. **The value of $k$ is set to 1 and remains fixed in the experiments.** Existing TGNNs (e.g., GraphMixer and DyGformer) utilize historical interactions **of a node** to compute its temporal node embedding. However, in addition to those node embeddings, TAMI stores and leverages $k$ historical edge embeddings in LHA to ensure that the mutual interaction history between a target node pair is preserved when predicting their future links. Figure 7 in Section 4.6 demonstrates that, when integrated with TAMI, the underlying TGNN (e.g., DyGformer) can compute temporal node embeddings based on significantly fewer historical interactions compared with its vanilla counterpart, i.e., from 256 to 32 on the **EN** dataset and from 32 to 2 on the **UC** dataset, to achieve comparable or even better performance.\n\n- **Limitations: The authors’ contributions focus on improving temporal encoding and information aggregation, offering a novel perspective and effectively enhancing existing models. However, the approach lacks methodological innovation, resulting in somewhat limited overall contributions**\n\nThanks for the thoughtful comment. We would like to point out that we are the first to identify the presence of heterogeneity in temporal interactions and investigate its impact on the performance of TGNNs thoroughly. This heterogeneity leads to two problems in temporal link prediction for existing TGNNs, i.e., ineffective temporal encoding and forgetting of past interactions for a pair of nodes that interact intermittently. To cope with these problems, we propose **two novel modules, namely, LTE and LHA**, respectively. We then develop **an end-to-end framework named TAMI** to integrate LTE and LHA with various types of existing TGNNs in order to boost their performance in temporal link prediction. In other words, we do not propose yet another TGNN, but we propose TAMI to be used as an add-on to existing TGNNs for improved temporal link prediction. Comprehensive and extensive experiment results on 13 classic datasets and 3 large TGB datasets with three negative sampling strategies under both inductive and transductive settings demonstrate the effectiveness of TAMI.\n\nWe hope this clarifies the motivation and the contributions of our work, and we sincerely appreciate the reviewer’s comment."}, {"review_id": "8ZwUAhO1q8", "reviewer": "Reviewer_LnpK", "summary": "This paper proposes two enhancements to existing temporal link prediction methods: (1) applying a logarithmic transformation to the temporal difference $\\Delta t$, and (2) storing the most recent $k$ embeddings for each edge instead of only the latest one. Experimental results demonstrate the effectiveness of the proposed approach on selected baselines.", "questions": "Please see the weakness part.", "limitations": "The limitations have been provided in Appendix C.3", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strength**\n\n1. The authors propose two enhancements that can be integrated into existing temporal link prediction methods.\n2. Experimental results verify the effectiveness and efficiency of the proposed methods on the selected baselines and datasets.\n\n**Weaknesse**\n\n1. The technical contribution of this paper is limited, as the proposed modifications are trivial and their effects are expected. Specifically, the LTE method introduces a log transformation to the temporal differences—a well-known technique in machine learning to improve the training stability when the feature scales vary. As such, it is unsurprising that this modification improves model performance when the scales of temporal differences vary. Furthermore, the instability of temporal encoding methods is not a novel observation by the authors—it was previously discussed in GraphMixer [4]. Regarding the LHA module, storing more historical embeddings instead of just the most recent one to mitigate information loss is also an intuitive strategy. Similar ideas have been explored in prior work [3], and the reviewer does not find the proposed LHA design to offer significant novelty or technical depth.\n2. The experimental evaluation lacks comprehensiveness and fails to convincingly support the claimed advantages of the proposed method..\n    1. The newest baseline is published on NeurIPS 2023 (DyGFormer). There are lots of newer link prediction methods that are both more efficient and effective than DyGForemer [1,2,3]. The authors are supposed to report the results of TAMI on these methods, like [1], for a fair and thorough evaluation of the proposed TAMI method.\n    2. The efficiency analysis is limited to two small datasets (i.e., UC and EN) and one computationally expensive baseline. To fully evaluate the efficiency of TAMI, the authors are encouraged to report results on larger datasets and more baselines.\n    \n\n[1] Lu X, Sun L, Zhu T, et al. Improving Temporal Link Prediction via Temporal Walk Matrix Projection[C]//The Thirty-eighth Annual Conference on Neural Information Processing Systems.\n\n[2] Tian Y, Qi Y, Guo F. Freedyg: Frequency enhanced continuous-time dynamic graph model for link prediction[C]//The Twelfth International Conference on Learning Representations. 2024.\n\n[3] Cheng K, Linzhi P, Ye J, et al. Co-Neighbor Encoding Schema: A Light-cost Structure Encoding Method for Dynamic Link Prediction[C]//Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024: 421-432.\n\n[4] Cong W, Zhang S, Kang J, et al. DO WE REALLY NEED COMPLICATED MODEL ARCHITECTURES FOR TEMPORAL NETWORKS?[C]//11th International Conference on Learning Representations, ICLR 2023. 2023.", "was_revised": true, "final_justification": "The rebuttal has addressed my concerns, so I decide to raise my score to borderline accept.", "rebuttal": "Thank you very much for the constructive feedback. Below we respond to the concerns raised.\n\n- **W1. The technical contribution of this paper is limited, as the proposed modifications are trivial and their effects are expected. Specifically, the LTE method introduces a log transformation to the temporal differences — a well-known technique in machine learning to improve the training stability when the feature scales vary. As such, it is unsurprising that this modification improves model performance when the scales of temporal differences vary**\n\nThanks for the thoughtful comment. We believe that **the reviewer's assessment that our modules' effects were as expected, and the performance improvement was unsurprising, confirms the intuitive and effective nature of our design**.\n\nWe emphasize that our design is novel and our contributions are substantial. Specifically, we are the first to identify the heterogeneity in temporal interactions and investigate its impact on the performance of TGNNs thoroughly. This heterogeneity leads to two problems in temporal link prediction for existing TGNNs, i.e., ineffective temporal encoding and forgetting of past interactions for a pair of nodes that interact intermittently. To cope with these problems, we propose two novel modules, namely, LTE and LHA, respectively. We then develop an end-to-end framework named TAMI to integrate LTE and LHA into existing TGNNs to boost their performance in temporal link prediction. Extensive experiment results on 13 classic datasets and 3 large TGB datasets demonstrate the effectiveness of TAMI.\n\nIn particular, **while the logarithmic transformation is a well-known technique in statistics and machine learning, it remains unknown why and where we need to apply it to TGNNs**. LTE is motivated by our observation that temporal differences often follow a highly right-skewed distribution, but existing TGNNs overlook this skewness. Since their time encoding functions are directly applied to the raw temporal differences, their resulting models often struggle with the prediction of infrequent interactions, as shown in Figure 3(b). LTE addresses these issues by transforming a skewed distribution of temporal differences into a more balanced one, thereby effectively improving the link prediction accuracy.\n\n- **W1 (cont'd) The instability of temporal encoding methods was previously discussed in GraphMixer.**\n\nThanks for the comment. Our work is **orthogonal** to GraphMixer. GraphMixer finds that the instability of time encoding functions is due to the fact that the gradients of their trainable parameters scale proportionally with the input temporal differences. Thus, they propose to fix these parameters during training. In contrast, in TAMI, we observe a new problem that **the distribution of the input temporal differences is often right-skewed**, and its negative impact on temporal link prediction. Based on this observation, we propose a novel time encoding function, LTE, to directly mitigate the skewness in the temporal differences to improve the link prediction performance. Note that LTE can be used for both when the frequency parameters need to be learned or fixed.\n\nAs such, LTE can be integrated into GraphMixer to boost its performance. As shown in Figure 3(b), together with LTE, GraphMixer's performance improves substantially, especially for node pairs whose interaction intervals are long. Furthermore, Tables 3 and 16 in our manuscript show that LTE, used with the fixed time encoding function in GraphMixer, consistently improves GraphMixer's performance under both transductive and inductive settings.\n\n- **W1 (cont'd) LHA stores more historical embeddings instead of just the most recent one to mitigate information loss. Similar ideas have been explored in prior work [3].**\n\nThanks for the comment. Again, our work is **orthogonal** to [3]. Our LHA module maintains the historical interactions using edge embeddings for each target node pair, in addition to the node embeddings learned by the underlying TGNN, such as [3], which retains each node's recent historical interactions to learn its temporal node embeddings.\n\nSuppose we are to predict a link between node $u$ and node $v$. [3] first computes the temporal node embeddings of $u$ and $v$ using $u$'s recent interactions and $v$'s recent interactions, respectively. However, it is possible that $u$ and $v$ do not interact recently. Hence, their node embeddings may not capture any of their interaction history and thus be less effective in link prediction. In contrast, LHA explicitly encodes the past interactions between $u$ and $v$ into a dedicated historical edge embedding and directly leverages this edge embedding, together with their node embeddings, for predicting their future link.\n\n[3] Cheng K, Linzhi P, Ye J, et al. Co-Neighbor Encoding Schema: A Light-cost Structure Encoding Method for Dynamic Link Prediction. ACM KDD 2024.\n\n- **W2. The experimental evaluation lacks comprehensiveness and fails to convincingly support the claimed advantages of the proposed method**\n\nThanks for the comment. Our experimental evaluation is comprehensive and extensive, as it is done with various underlying TGNNs on 13 classic datasets and 3 large TGB datasets under both inductive and transductive settings with three negative sampling strategies.\n\nFurthermore, the effectiveness of LTE and LHA is demonstrated separately. The effectiveness of LTE in mitigating the skewness in temporal differences for improved link prediction accuracy is well supported by Figure 3(b) and Tables 3, 13, 16, and 18 in our manuscript. In addition, the effectiveness of LHA in addressing the forgetting issue of historical interactions between target node pairs for improved link prediction accuracy is well demonstrated by Figures 5, 8, and 9, and Tables 3, 13, 16, and 18.\n\n- **W2 (cont'd) Authors are supposed to report the results of TAMI on newer methods, like [1], for a fair and thorough evaluation of the proposed TAMI method**\n\nThanks for the comment. We do not aim to propose yet another TGNN architecture. Instead, we propose TAMI to be used for existing TGNNs to further improve their performance by addressing the heterogeneity of temporal interactions that is overlooked in their learning process. We demonstrate the effectiveness of TAMI through extensive experiments. For example, Tables 1, 2, and 5 in our manuscript show that TAMI consistently improves the performance of various TGNNs over their vanilla counterparts.\n\nTo provide a more thorough evaluation, we have conducted extra experiments on TAMI with TPNet [1]. As shown in Table 1 below, TAMI indeed improves the performance of TPNet.\n\n**Table 1: Transductive AP of TPNet with and without TAMI.**\n| Method \\ Datasets|USLegis (US) |Enron (EN) |Uci (UC) |CanParl (CP) |\n|-|-|-|-|-|\n|TPNet|80.58|92.90|97.35|90.28|\n|**TPNet+TAMI**|**94.15**|**93.87**|**97.61**|**91.03**|\n\n[1] Lu X, Sun L, Zhu T, et al. Improving Temporal Link Prediction via Temporal Walk Matrix Projection. NeurIPS 2024\n\n- **W2 (cont'd) The efficiency analysis is limited to two small datasets (i.e., UC and EN) and one computationally expensive baseline**\n\nThanks for the comment. In Section 4.6, we do not aim to evaluate the computational efficiency of TAMI, but rather to empirically demonstrate that TAMI can improve the training efficiency of underlying TGNNs. As shown in Figure 6, TAMI enables **DyGFormer, GraphMixer, and TGAT** to converge **faster** than their vanilla counterparts, while achieving higher validation average precision with fewer training epochs. We have also conducted similar experiments for three other baselines, i.e., **TGN, JODIE, and CAWN**. Results in Table 2 below show that TAMI indeed improves their training efficiency. The overall trend of training three new baselines is similar to our observation with Figure 6.\n\n**Table 2: Results on the UC (top) and EN (bottom) dataset.**\n|Method|Val AP after 5 epochs|Final Val AP|Total epochs|\n|-|-|-|-|\n|TGN|85.28|88.32|41|\n|**+TAMI**|**94.14**|**94.43**|**36**|\n|JODIE|83.20|84.05|39|\n|**+TAMI**|**89.12**|**89.36**|**24**|\n|CAWN|93.22|93.53|**42**|\n|**+TAMI**|**95.18**|**95.43**|48|\n\n|Method|Val AP after 5 epochs|Final Val AP|Total epochs|\n|-|-|-|-|\n|TGN |80.61|87.01|70|\n|**+TAMI**|**90.21**|**91.10**|**43**|\n|JODIE|77.24|85.77|82|\n|**+TAMI**|**88.71**|**89.52**|**28**|\n|CAWN|88.79|90.10|36|\n|**+TAMI**|**90.19**|**91.18**|**34**|\n\nIn addition, in Figure 7, we demonstrate that TAMI enables existing TGNNs, such as DyGFormer, to compute temporal node embeddings using **fewer** historical interactions to achieve comparable or even better performance. As a result, they consume much less GPU memory during training, require less training time per epoch, and converge faster with fewer epochs. We here use DyGFormer as an example because it is the top performer in all baselines. Together with TAMI, this computationally expensive baseline becomes much more efficient while achieving comparable or even better performance, with less GPU memory usage (up to 53.22%) and less training time per epoch (up to 40.54%).\n\nWe have conducted additional experiments on the large-scale dataset, **UNVote (UV)**, which contains over one million edges, using DyGFormer and CAWN. Results are shown in Table 3 below. Again, TAMI allows them to use significantly fewer historical interactions to achieve comparable or even better performance, thereby reducing both GPU memory usage and total training time.\n\n**Table 3: Results on the UV dataset. The number in the parenthesis indicates the number of historical interactions used to compute the node embeddings.**\n|Method|GPU mem (MiB)|Train time per epoch (second) |Epochs|Test AP|\n|-|-|-|-|-|\n|DyGFormer(128)|4069|490|67|55.88|\n|DyGFormer(32)|2415|341|60|55.15|\n|**DyGFormer(32)+TAMI**|**2435**|**352**|**23**|**59.09**|\n|CAWN(64)|7279|2442|52|52.84|\n|CAWN(4)|2691|330|26|51.69|\n|**CAWN(4)+TAMI**|**2741**|**347**|**23**|**58.47**|"}]

Author rebuttal:

Meta-review: This paper identifies that skewed interaction patterns in temporal graphs hurt link prediction and proposes TAMI, a lightweight framework that uses logarithmic time encoding (LTE) to balance temporal data and link history aggregation (LHA) to prevent models from forgetting past, infrequent interactions.

The paper's primary strength is its novel identification of the interaction heterogeneity problem, supported by an exceptionally rigorous experimental evaluation that validates its simple, effective, and broadly applicable solution. Its main weaknesses are the limited technical novelty of its components and its unproven applicability beyond the link prediction task, with performance gains being modest on some top-performing models.

In the discussion, the authors addressed the main weaknesses to some extent. Initial concerns about limited novelty and an insufficient set of modern baselines were partially resolved when the authors provided new experiments integrating TAMI and conducted a more comprehensive efficiency analysis, convincing the reviewers to raise their score. Similarly, questions about inconsistent performance gains on certain datasets were clarified with deeper data analysis that explained the observed phenomena. The only remaining point is the framework's applicability beyond link prediction, which the authors acknowledged as a valid limitation and a direction for future work.

I also noticed the limited broad usage of the proposed techniques. But the extensive experiments and analysis outweigh the weaknesses. Therefore, I suggest accepting the paper as a poster,
Decision: Accept (poster)