FreeDyG: Frequency Enhanced Continuous-Time Dynamic Graph Model for Link Prediction
Authors: Yuxing Tian
Yiyan Qi
Fan Guo
Venue: ICLR 2024 poster (2024)
Keywords: Dynamic graph; fourier transform; link prediction
Primary area: learning on graphs and other geometries & topologies

Abstract: Link prediction is a crucial task in dynamic graph learning. Recent advancements in continuous-time dynamic graph models, primarily by leveraging richer temporal details, have significantly improved link prediction performance. However, due to their complex modules, they still face several challenges, such as overfitting and optimization difficulties. More importantly, it is challenging for these methods to capture the 'shift' phenomenon, where node interaction patterns change over time. To address these issues, we propose a simple yet novel method called \textbf{Fre}quency \textbf{E}nhanced Continuous-Time \textbf{Dy}namic \textbf{G}raph ({\bf FreeDyG}) model for link prediction. Specifically, we propose a node interaction frequency encoding module that both explicitly captures the proportion of common neighbors and the frequency of the interaction of the node pair. Unlike previous works that primarily focus on the time domain, we delve into the frequency domain, allowing a deeper and more nuanced extraction of interaction patterns, revealing periodic and "shift" behaviors. Extensive experiments conducted on seven real-world continuous-time dynamic graph datasets validate the effectiveness of FreeDyG. The results consistently demonstrate that FreeDyG outperforms existing methods in both transductive and inductive settings. Our code is available at this repository: \href{https://github.com/Tianxzzz/FreeDyG}{https://github.com/Tianxzzz/FreeDyG}

Published as a conference paper at ICLR 2024
FREEDYG: FREQUENCY ENHANCED CONTINUOUS- TIME DYNAMIC GRAPH MODEL FOR LINK PREDIC- TION
Yuxing Tian Yiyan Qi** FanGuo
IDEA Research, International Digital Economy Academy Jjangxx Normal University
ABSTRACT
Link prediction is a crucial task in dynamic graph learning. Recent advancements in continuous-time dynamic graph models, primarily by leveraging richer tem- poral details, have significantly improved link prediction performance. However, due their complex modules, they still face several challenges, such overfitting and optimization difficulties. More importantly. it is challenging for these meth- ods to capture the 'shift' phenomenon, where node interaction patterns change over time. To address these issues, we propose simple yet novel method called Frequency Enhanced Continuous-Time Dynamic Graph (FreeDyG) model for link prediction. Specifically, we propose node interaction frequency encoding module that both explicitly captures the proportion of common neighbors and the frequency of the interaction of the node pair. Unlike previous works that pri- marily focus on the time domain, we delve into the frequency domain, allow- ing deeper and more nuanced extraction of interaction patterss, revealing peri- odic and "shift" behaviors. Extensive experiments conducted on seven real-world continuous-timg dynamic graph datasets validate the effectiveness of FreeDyG. The results consistently demonstrate that FreeDyyG outperforms existing methods in both transductive and inductive settings. Our code is available at this repository: https://github.com/ htwp://uwbb.bo/Taaaaz//////ooo/o Tianxz
INTRODUCTION
Link prediction on dynamic graphs is a fundamental problem various real-world applications, such social media analysis Huo et al. (2018) Alvarez- al. (2021), recommendation systems Song et al. (2019); Yuxiao et al. (2012); Wang et al. (2021b), and drug discovery [Abbas]
al. (02T). Recently, dynamic graph representation learning methods have become widespread in both industry and academia due to their remarkable performance in solving this problem.
Existing dynamic graph learning methods can be divided into two main categories: discrete-time dynamic graph (DTDG) models Pareja et al. (2020); Zhao et al. (2019); Yang et al. and continuous-
time dynamic graph (CTDG) models Kumar et al. (201 Wang et al. (2021c): Xu al. 2020
Trivedi et al. (2019): Yu et al.] (2023): Wang et al. (2021a Jin et al. 2022a): Luo et al. 2023
Chang et al. 2200 Huang et al. (2020). Notably, there has been a growing interest in CTDG algorithms, primarily because of their ability to preserve time information effectively.
Although the above CTDG methods have achieved impressive results, they still have some limita- tions. Firstly, some methods rely on random walks (RW) Wang et al. (2021d); Din et al. (20226), temporal point processes (TPP) Chang et al.] (2020); Huang et al. (2020). or ordinary differential equations (ODE) Luo al. (2023); Liang et al. (2022) to enhance their learning ability. How- ever, these methods are computationally expensive and may lead to overfitting of the historical interactions. Secondly, most existing methods encode interaction sequences independently, disre- garding potential relationships between them. Although Yu et al. (2023) proposes the neighbor
"Equal contribution.
I Work done during internship at IDEA Research.
1 Corresponding author.

---

Published as a conference paper at ICLR 2024
co-occurrence mechanism, it relies solely on the number of historical common neighbors between node pairs to predict the likelihood of node pairs interacting in the future. As a result, is difficult for them to capture the "shift" phenomenon which is commonly hidden behind time series data. For instance, in a co-author network, researchers who are acquainted with each other tend to collaborate during the same period each year, as conferences are typically held around the same time annually. However, when a new and popular research topic emerges suddenly, such as large language models (LLMs), there notable shift in these collaboration patterns. Researchers then show a preference for establishing collaborations with unfamiliar researchers who specialize in that area.
To address the above challenges, we delve into the frequency domain and propose a simple yet novel method called Frequency Enhanced Continuous-Tinn Time Dynamic Graph (FreeDyG) model for link prediction. FreeDyG comprises two essential components: the encoding layer and the Trequency-enhance MIP-Mixer layer. The encoding layer is designed to transform each interac- tion (u, v t) into an embedding sequence, considering the historical neighbors of both node u and In contrast to existing methods, we also introduce new node interaction frequency encoding ap- proach to explicitly capture the interaction frequency between node pairs. Then, we propose a novel Trequency-enhaneee
MLP-Mixer layer efficiently capture the periodic temporal patterns and the "shift" phenomenon hidden in the frequency domain. Specifically, we apply the Fourier Transform to the encoded embedding sequence, followed by multiplication with learnable complex number tensor. This can adaptively enhance desired frequencies, thereby improving the model's ability to capture relevant information.
Our contributions can be summarized as follows: Firstly, we introduce a novel frequency-enhanee dynamic graph model called FreeDyG for the task of link prediction. Instead of the temporal domain, FreeDyG tries to address this problem by delving into the frequency domain. Secondly, in addition to the traditional encoding of node/link features and time information, we propose node interaction frequency encoding approach that explicitly captures the frequency of neighbor nodes from both the source and target nodes. Furthermore, we design a novel frequency-enhanceed MI.P-Mixer layer to further capture periodic temporal patterns and the "shift" phenomenon present in the frequency domain. Finally, we conduct extensive experiments on seven widely used real-world continuous- time dynamic graph datasets, evaluating both transductive and inductive settings. The experimental results demonstrate the superior performance of FreeDyG compared to state-of-the-an methods.
2 PRELIMINARIES
Task Definition. Following prevailing methods Poursafaei et al. (2022) ; Yu et al. (2023); Rossi al. (2020), our study primarily focuses on CTDGs with edge addition events. A CTDG G' can be represented as chronological sequence of interactions between specific node pairs: G == {(v0, (o, 1o), (u1, 71, ... (un, Un, (n)), where i, denotes the timestamp and the times- tamps are ordered as (0 I to < t S I t.). u1 E V denote the node IDs of the i - th interaction at timestamp f,. V is the entire node set. Each node u E V is associated with node feature T. E Rdd, and each interaction (u, u, t) has edge feature e E Rd where dy and dE denote the dimensions of the node and link feature respectively. Based on the above definitions, link prediction in CTDGs can be formulated as: given pair of nodes with specific timestamp aim to predict whether the two nodes are connected at t based on all the available historical data.
Discrete Fourier Transform (DFT), Given a sequence of data {Ix J.V the Discrete Fourier Trans- form (DFT) is utilized to convert the sequence into the frequency domain. The DFT operation can be defined as:
N
X
Σ
Ine
2 nk
ISKSN
(I)
where denotes the imaginary unit and { a sequence of complex numbers in the frequency domain. Thus, X₁ captures the spectrum of the sequence {rr,) at the frequency Wa 2 M. We note that the DFT is one-to-one transformation, enabling the recovery of the original sequence through the inverse DFT (IDFT):
N
In
X
2 nk
(2)
N
k=1
2

---

Published as a conference paper at ICLR 2024
Encoding Layer
Fime moeeiny
Edg mooding
dil +
+b)
<b
Nade
onhanced MI F- F-Mierr Layer
Intunder layer
Milur
AG
Proticiin Lag
4 Sadt sanading
Conticurep-lier dynanicampi
Ede mosdin 4t
Adapttre
Figure 1: The overview of FreeDyG.
The computation complexity of DFT is O(N2). practice, the Fast Fourier Transform (FFT) is commonly used to compute the DFT efficiently, which could recursively express the DFT of a se- quence of length N and reduce the computation complexity from O(N²) to O(N log N). Similarly, the IDFT in Squation2can be efficiently computed using the inverse Fast Fourier Transform (IFFT). Due to the page limit, we omit the details of DFT and FFT, which can be found Bruun (1778).
METHOLODGY
In this section, we provide a detailed description of our FreeDyG. As shown in Figure I the model operates on a node pair (u, v) and specific timestamp (t). Initially, we sample L first-hop histor- ical neighbors for both nodes, ordered based on temporal proximity, to construct two interaction sequences, S and St. If node has fewer than I. historical neighbors, zero-padding is used to fill the gap. Subsequently, we employ an encoding layer to encode the features of each node, link, and timestamp within the sequences. Additionally, we incorporate the frequencies of neighbor appear- ances both St and S to exploit correlations between nodes. The encoded representations are then aligned and fed into crequenccyenhaanee different frequencies. The outputs are aggregated to generate time-aware representations u and v timestamp (i.e., M2 and h.). The final prediction is generated by the link prediction layer.
MLP-Mixer layer, enabling the capture of information at
3.1 ENCODING LAYER
Node/Edge Encoding, In dynamic graphs, both nodes and edges (links) frequently possess associ- ated features. To derive embeddings for interactions, is sufficient extract the intrinsic features of the neighboring nodes and edges based on the sequence S where can be either u or v. Similar to existing works, we encode the nodes and links Z Lxdy and 7,, E IR. Lxdg respectively, where dx, dg are the dimensions of node and edge embeddings respectively.
Time Encoding, Time encoding is employed to map constant timestamps to vectors Z E &Lxxdr, where dr is the dimension of time emheddings. Specifically, we utilize the widely adopted time dr encoding function cos(t,, (t, where w {a ( i=1 is employed encode timestamps. a and 3 are hyperparameters to make fmax x a 11-1 close to when i close to dr. A cosine function is then applied to project these values into the range of [-1,+11] Notably, we use relative timestamps instead of absolute timestamps for encoding. In other words, if the timestamp of the sampled interaction fo. and the specific timestamp for link prediction f, we utilize ((t,(t, -t) as the effective relative time encoding function. It is worth mentioning that w remains constant and not updated during the training phase. This technique makes the model easy to optimize and leads to performance improvement, as demonstrated in Cong al. (2023).
Node Interaction Frequency (NIF) Encoding. Most existing methods encode interaction se- quences, i.e., St and St, independently, disregarding potential relationships between them. Yu et al. (2023) introduces a neighbor co-occurrence scheme, suggesting that nodes with higher number of shared historical neighbors portion of edges in networks exhibit recurring patterns over time, these recurrence patterns vary considerably across different networks and domains Poursafaei al. (002))
more likely to interact in the future. However, while significant
3

---

Published as a conference paper at ICLR 2024
To address this issue, introduce node interaction frequency encoding approach, which takes into account not only the appearance frequency of common neighbors but also the interaction frequency between the node pairs. It recognizes that the potential for future interactions between two nodes influenced by both their common neighbors and their past interactions. Specifically, given the interaction sequences Sy and S. count the frequency of each neighbor in both St and Sy- In addition, We specifically encode for the frequency of the interaction of the node pair and get the node interaction frequency features for u and v, which are represented by Z_,c. For example, suppose the historical interacted nodes of u and are {c, v, c, d, v) and {d. u. d, c, u). The appearing frequencies of d in u/v's historical interactions are 2/1, 1/2 respectively. And the node pair interaction frequency [2, Therefore, the node interaction frequency features of u and v can be denoted by F = [ 1 2, 2]. 2, [1, 2). [2, 2]] and FØ = 11. [2, 2), [1, 2), [2, 12, Then, we encode the node interaction frequency features by:
Z (F:1: o (F%):.1 11) RLxdF
(3)
where * could be or f() represents a two-layer perceptron with ReLU activation. And the dimension of input and output f() are and dp.
Finally, we concatenate all encodings mentioned above to an embedding of dimension d with train- able weight W. E Rd. xd and b, E Rd,, resulting in Z E R² and Zi E R Here, can be N, E, F. Subsequently, the concatenated encodings for nodes are summed as 2 E R2 Zz + Di Zur + Zi and Z E Rixdd De,N Z + Z + Ze, respectively.
3.2 FREQUENCY-ENHANCE MLP-MIXER LAYER
We derive the historical interaction embedding sequences for the node pair by utilizing the encoding layer. Since each interaction embedding is arranged chronologically. it can be viewed as discrete time series data. Consequently, we naturally leverage the Fourier transform series data into their constituent frequencies, which can effectively capture the interaction patterns of nodes across various frequencies. To this end, we introduce the frequency-enhanced MLP-Mixer layer, which contains two sublayers: the frequency-enhancing layer and the MLP-Mixer layer.
decompose of time
Frequency Enhancing (FE) layer. Given the input Z of the 1-th layer and the first layer input is Z. For simplicity, Z is short for Z.. To better identify important frequencies within the historical interaction sequence data, we perform the Fast Fourier Transform (FFT) along the first dimension of Z E IR Lxd to convert into the frequency domain:
z = =(!))
where F denotes the ID FFT. z! cl(+)) denotes the frequency components of Z and C denotes the complex number domain. Then we can adaptively enhance the frequency components by multiplying it with a learnable complex number tensor W E c(4+1)xd
Σ - W-
(5)
where denotes the element-wise multiplication, 21 E Cl +((++) represents the enhanced fre- quency components. Finally, we transform Σ back to the time domain.
7 --
(6)
where F-1() denotes the inverse ID FFT, which converts the complex number tensor into real number tensor. Then we use the residual connection and dropout layer as:
z - Dropout
Z
MLP-Mixer layer. After that, we employ MLP-Mixer Tolstikhin et al. (2021) to further capture the non-linearity characteristics, which contains two types of layers: token mixing MLP layer and channel mixing MLP layer. The computation listed as:
7 Joken %% W token a (W1 loken LayenNorm 2!));
(8)
Z chamnel
Joken + +o (LayerNorm( Z2 (oken) W! channel 2 Z Lia
channel
4

---

Published as a conference paper at ICLR 2024
where a is element wise nonlinearity, W tnken Raxd and W channel R trainable param- eters in the token-min mixing and chinnel-mixing MLPs, respectively. The same channel mixing MLP layer (token-mn miinng MLP layer) is applied to every (column) of input.
Theorem 1. Given an input Z E RRD and a learnable matrix W E RNDD and their correspond- ing frequency domain conversion, the multiplication in the frequency domain is equivalent to the global convolution in the time domain.
Proof. We regard Z RR NxD {zn E RRDN I==1 and W E ERND as {w,, E RR Denote the N d-th dimension features of Z and W as (2 Jnmi and {wo ( n== Then, the global convolution can
be defined as:
N
W n n=1
[w
( In=l =
Σ
wm
mn-m) mad N
(9)
where @ and mod denote the convolution and integer modulo operation, respectively. Then, ac- cording Equation | the multiplication in the frequency domain can be written as:
N
DFT( (4) 1 ( )= = w
(1)
(d)
25
n
e
n=1
N
(d)
(d)
2
kn +
(d) -g gg kn
k
e
n=1
12 N mm
N
m
(d)
Σ
(d)
2x1 ((n
+
-nm++ye
(d)
2x1 k(n
n=m
N
2c5 km
(d)
2x4 k()
W
e
(n-m)%No
m=1
==1
DFT
(1 jn=1
(d)
(w fn=1)
(d)
(10)
3.3 AGGREGATOR
Different from most previous methods that simply aggregate the sequence of each appearing element using average pooling, get the time-aware representations ht E IRd and ht Rd of node u and v at timestamp t by the following weighted aggregation equation:
h = (Waag Z_.aammee) Z cchanel)
(11)
where Wegg 1 is a trainable vector designed to adaptively learn the importance of various interactions.
3.4 LINK PREDICTION LAYER
The prediction ý is computed by adopting 2-layer multilayer perceptron (MLP) and using softmax to convert it into link probability on the concatenation of the above two nodes' representation:
ý = S.ff
(12)
3.5
Loss FUNCTION
For link prediction loss, we adopt binary cross-entropy loss function, which defined as:
Cp = K Σ Ilogg + (1 y:) * log (c)),
(13)
where K is the number of positive/negative samples, y; represents the ground label of i-th sample and the Si represents the prediction value.
5

---

Published as a conference paper at ICLR 2024
100
100
24
99
99.5
TEAT
IMANT
CATES
97
98.5
CAMN
96
98
200ND
97.5
94
Traising (Akppec))
Triining (Nipooc))
Figure 2: Comparison model performance, parameter size and training time per epoch on WIKI and Reddit.
EXPERIMENTS
4.1 DATASETS
We utilize publicly available real-world datasets: Wiki, REDDIT, MOOC, LastFM, Enron, Social Evo, and UCI, in our study. Appendix[A]prove provides detailed description and the statistics of the datasets shown in Table 4 The sparsity of the graphs is quantified using the density score, [v]] where E and [V] represent the number of links and nodes in the training set, respectively. To facilitate training, validation, and testing, we split these datasets into three chronological segments with ratios of 70%-1555 11%655...
calculated E
4.2 BASELINES
To evaluate the performance of our method, conduct experiments comparing it with previous state-of-the oftt---ars.. including JODIE (Kumar et al. 2019). DyRep (Trivedi et al. 2019). TGAT (Xu et al. 2020), TGN (Rossi et al. 2020), CAWN (Wang al. 2021d). EdgeBank (Poursafaet al.
2022
TCL (Wang et al.) 2021a). GraphMixer (Cong et al. 2023). DyGFormer
(Yu et al. 2023
Detailed descriptions of these methods be found in Appendix B
4.3
EVALUATION METRICS AND EXPERIMENTAL SETTINGS
For evaluating method, employ Average Precision (AP) and Area Under the Receiver Op- erating Characteristic Curve (AUC-ROC) as the evaluation metrics. The link prediction task en- compasses settings: 1) transductive setting, which focuses predicting future links between nodes observed during training, and 2) inductive setting, which involves predicting future links between unseen nodes. To ensure comprehensive comparisons, three negative sampling strategies as Poursafaci et al. (2022). i.c., random (md), historical (hist), and inductive (ind) negative sampling, which the latter two more challenging. The evaluation details of all three strategies can be found in Appendix C
also evaluate each method with
All models are trained for a maximum of 200 epochs using the early stopping strategy with patience of 20. The model that achieves the highest performance on the validation set is selected for testing. For all models, we employ the Adam optimizer and set the learning rate and batch size to 0.0001 and 200, respectively. The hyperparameter configurations of the baselines align with those specified in their respective papers. For our FreeDyG, we set the dr to 100,and both a and 3 to 10. The number of frequency-enben enhannce MLP-Mixer layers are 2. We conduct ten runs of each method with different seeds and report the average performance to eliminate deviations. All experiments are performed on an NVIDIA A100-SXM4 40GB GPU.
4.4 COMPARISON WITH SOTA
In this section, compare FreeDyG with the previous SOTA in both transductive and induc- tive settings. Table 10 and Table show the AP of all datasets in these two settings respectively. To provide comprehensive study of our FreeDyG, we show results among all three negative sampling strategies. Due to the limitation space, we put similar results on AUC-ROC in Table [5 and Table 6 in Appendix. We note that EdgeBank Poursafaei et al. (2022) is only designed for the

---

Published as a conference paper at ICLR 2024
NSS
Datasets
JODIE
DyRop
TGAT
TGN
CAWN
EdgeBank
TCL
GrapaMiieer
Dytifoemer
FreebyyG
Wiki
96.69 + 0.25
95.23 a 0.50
96.55 6:17
98.42
98.65
0.04
90.37
96.47
97.17
9882 0.02
99.26 = 0.01
Rodia
97.82+00.11
98.17-0002
984776003
98.6750044
99.11:00001
94.6-00000
97.37900011
99.11+40002
90.80-00011
MOOC
17:00:1:130
79.9710883
85.44:1.766
19.33:2:955
78.6610:311
57.9710000
81.124043
K2.73:0::16
67:2311:45
19.6..01199
rod
Last-M
68.544::996
10.79-::117
75.76::0.55
78.0+2271
86.56:00.10
79:29:00:0
65.64:2:222
75.64:0:23
92:00+00222
92.15-:0.116
Enron
79.101:0.85
K2.00:1:307
72.58::0.79
15:33:::005
89 89.56::0009
83.53::0000
79.70::0.1
01.08:0::33
92.47::0.13
12.51::005
Social Evo.
88.12400774
88.57-0030
93.164:0.17
93.57+00117
84.964:0009
74.95-0000
93.13460.16
93.57
94.73--0000
94.91-00001
UCT
8765-1::88
30.04::::33
79.55:::833
90.6911:45
94.35:2:011
76.00:1:0000
88,
88.12:22.73
93, 03.50:2::49
95 95.76:1:0015 15
96.28:00.11
Rank
7.14
6.08
5.85
2.94
8.79
4.56
2.14
Wiki
81.19:0.48
78.3240.71
87.0110.19
86.9610.36
72.38+1.85
73.35
0.00
88.75+0.27
90,87+0.08
62.23 + 2,54
91.59a0.57
Redki
50.05 0.36
1 0.31
79:35 a 0.20
4..35
8082
0.445
a 0.00
77.14
78.44
81.02
0.59
85.67
1.01
MOOC
78.94 + 1,25
75.60
1.12
83.19
87.06 a 1.93
74.05
0.95
60.71
0.00
77.06
0:41
77.77
0.92
85.85
0.66
86.71
0.81
hist
LastFM
24.35
74,92
2.46 71.39
0.24
76.87 + 4.64
09.86
0.43
73.03
a: 0.00
59.30
2.31
72.47 a 0.49
81.57 à 0.48
79.71
09.71:: 0.51
Firon
60.85
71.19
2.76
64.73
0.36
76.53
0.00
0.39
77.98
75.63
0.73
78.87
0.82
Social Evo.
67.44
93.29 a 0.43
95.01 + 0.44
94.45 A 0.56
85.53
0.38
50.57
4 0.00
94.74
0.31
94,93 4 0.31
90.38 0.14
97.79 a 0.23
UCI
4.80
4.11
68.27
à 1.37
80.43
2.12
4..43
65.50
0.00
80.25
2.74
84.11
$2.17
$6.10 1.19
Rank
5.46
5,08
5.08
3.85
7.54
5.92
5.46
4.00
2.85
1.28
Wiki
15.65 0.779
70.21
1.58
87.00
85.62
0.44
74.06
2.62
80.63
0.00
86.76
0.72
$8.59
78.29
5.38
90.05
0,79
Reddit
$6.98 0.16
86.30
0.26
80.59
88.10
91.67
0.24
85.48
0.00
85.26
21.11+0.40
0.17
MOOC
65.23 2.19
61.66
0.95
75.95
0.64
2:91
73.51
0.94
49.43
0.00
74.65
0.54
74.27 a 0.92
$1.24 0.69
83.01 0.87
ind
LisPFM
62.67 4.449 64.41
2.70
0.17
65.95
67.
0.77
75.49
0.00
0.89
68.12
73,97
0.50
7 0.34
Erzon
68.96 1: 0.98
67.79
1.53
63.91
1.36
70.89 à 2.72
75.15 0.558 73.89
0.00
71.29
a 0.32
75.01 à 0.79
27.41 a 0.89
77.81 0.65
Social Evo.
89.82
4,11
93.28
0,48
99.84
0.44
95.13
0.56
88.32
0.27
83.69
0.00
94.90
+ 0.36
94.72 0.33
97.68 + 0.10
27.57
UCI
1,40
54.79
1,76
70.94
64.61
57.43
$0.10
72.25 1.71
82.35
Avg.Rant
6.62
6,38
4,15
4,38
3:46
5.62
4.69
4.46
2:43
1.71
Table 1: AP for transductive link prediction with three different negative sampling strategies.
Datasets
JODIE
DyKap
TGAT
IGN
CAWN
TCL
GraphMiser
DyLiformer
PreeL
Wikipedia
94.82 0.20
92.43 0.33
96.22 0.07
97.83
0.04
98.24 a 0.03
96.22 0
96.65 a 0.02
98.59 a 0:03
98.97:0.01
Reddii
96.50 0.13
96.09
97.09
97.50
0.07
98.62
94.09
0.07
95.26 0.00
98.84
98.91 0.01
MOOC
1.59 1.92
81.07
A 0.44
85.50 0.19
89.04 1.17
0:24
50.60 a 0.22
81.41 022
36.96 a 0.43
82.75 0.22
md
1asFFM
81.61 3.82
83.02
1.48
78.63 + 0.31
$1.45 4.29
89.42 + 0.07
73.53 1.66
82,11 0.42
94.23 0:09
94.89 0.01
Enron
80.72
74.55
3.95
67.05
77.94
86.35 à 0.51
76.14.0079
75.88
89.76
0.17
Social Evo.
91.96 0.48
90004 0.47
91.41
90.77 0.86
79.94 à 0.18
91.35 0.09
91.86 0.06
2..14
94.76 0.05
UCT
57.48
1.87
09
92.73
0.06
2.03
91.19+0.42
0.15
94.85 0.10
Rank
6.14
6.14
5.45
3.71
3.14
4.56
4.45
1.21
1.14
Wikipodia
68.69
0.30
62.18
1.27
8..17+0.22
81.76
0.32
67.27
1.63
$2.20 2.18
87.60 0.30
4.44
82.78
=0.30
Reddit
6..31:10.51
0.54
63.47 0.72
61.85
0.85
63.67 à 0.41
60.83 à 0.25
6 0.26
65.37 0.60
66.02 0.41
MOnC
63.22 1.55
62.93 + 1.24
76.73
77.07
3,41
74.68 + 0.68
74.27 + 0:53
74.00 e 0.97
$0.82
81.63 0.33
bist
LaslMM
1.33
71.45 à 1.76
66.65
6.11
71.33 a
76.42
76.35
77.25 0.21
Enrca
3.71
62.08 + 2.27
61,00 1.31
62.91
1.16
60.70 + 0.36
67.11 + 0.62
22:37 1:37
67.07 + 0.62
73.01 0.88
Social Evo.
88.72 8.87
1.10
93,97
0.51
90.66
1.62
79.83
91,10 0.31
96,01
0.17
96.82 0.16
96.69 0.14
UCI
52.47 6.211
2.06
70.52
70.78
0.78
0.47
76.71 +
82.66
0.49
72.13 1.87
82.35 0.39
Rank
5.71
6.14
3.46
3,71
4.14
3.85
2,85
2.57
1.28
Wikipedia
68.70 0.39
62.19 à 1,28
84.17 0.22
81.77
0.32
67.24 à 1.63
$2.20 à 2.18
87,60 a 0.29
71,42
a: 4.43
87.54 0.36
Reddin
62.32
0.72
63.9
6484
0.84
63.65 + 0.41
60.1 0.26
64.49
0.25
0.25
65.35 + 0.60
64.98 0.20
MOOC
1.52
62.92
1.24
76.72
77.07
74.69
0.53
73.99
0.97
LaakM
4.53
71.45 + 1.75
0.25
09:46 4.65
71.53 + 0.47
63.78 + 0:65
022422022
76.35 + 0.52
77.01 = 0.43
Eunon
65.86
62.08
2.27
61,40 1.30
62.90
1.16
60.72 a 0.36
67.11 a 0.62
72.37 1.38
67.07 a 0.62
72,85 0.51
Social Evo.
38.51
88.72 1.10
93.97
0.54
90.65
79.83 a 0.39
0.332
94.01
0.47
96.82 0.17
UCT
6.21
52.47 e: 2.09
70.49 a 0:93
70.73
0.79
64.54 a 0.47
76.65 a 0.99
81
61.64 0.49
13 1.86
N2.06 0.58
Asg.Rnn
6.62
7.14
4.14
4.57
3.57
1.28
Table 2: AP for inductive link prediction with three different negative sampling strategies.
transductive setting and thus in Table 2 and 6 we omit this method. We find that our FreeDyG out- performs other baselines in most scenarios and the average ranking of FreeDyG is close to 1, which is far superior to the second one, i.e., DyGFormer. In addition, it's notable that DyGFormer, when adopting the "hist" negative sampling strategy, experiences a much more significant performance decline compared to the "md" strategy. This underscores a limitation in DyGFormer's 's neighbor encoding approach, which simply relies on the historical co-occurrence counts of neighbors to de- termine the likelihood of node interactions. In contrast, our method takes into account the historical frequency patterns of node interactions, leading to improved performance.
7

---

Published as a conference paper at ICLR 2024
WIKI
Reddr
MOOC
42
FraddyG Ewel NIF
FFvellyf
Bib NIF
mino FX
Enron
27
277
2.4
9
Freefy
WS 100 FE
FFvelyyG
Figure 3: Ablation study of FreeDyG under historical negative sampling setting, where w/o NS and w/o represent FreeDyG without node interaction frequency encoding module and layer, respectively.
We also conduct comparative analysis of performance, training time per epoch (measured in sec- onds), and the size trainable parameters (measured in Megabyte, i.e., MB) between FreeDyG and baseline methods the WIKI and Reddit datasets. The results are shown in Figure 2 It is evident that the RW-based method, i... CAWN, not only requires longer training time but also has a sub- stantial number of parameters. On the other hand, simpler methods such as JODIE may have fewer parameters, but there a significant performance gap compared to the hest-performing method. In contrast, FreeDyG achieves the best performance with small size of trainable parameters and moderate training time required per epoch.
4.5 ABLATION STUDY
Next, we conduct an ablation study our FreeDyG model to assess the effectiveness of the pro- posed node interaction frequency encoding approach and FreeDyG without these two modules w/o NIF and w/o FE respectively. Figure 3 illlusttates the performance comparison under the historical negative sampling setting. Due the page limit, performance comparison under the random negative sampling setting is shown in Figure 5 in the Appendix. Both variants exhibit inferior performance across all datasets, highlighting the efficacy of the modules proposed. It is noteworthy that under different negative sampling settings, dis- tinct modules exhibit varying degrees of importance. In the historical negative sampling setting, the performance decreases
layer. We refer to
significantly without FE layer. Whereas, under the random negative sampling setting, the NIF encoding module has a more pronounced impact on performance.
The reason behind this phenomenon is follows: under the random negative sampling setting. where the target node of negative edge is sampled from the entire graph, the negative samples are significantly easier to distinguish, and the model tends predict future interactions between node pairs that have previously interacted. Consequently, the NIF encoding, which captures neighbor interactions. assumes more critical role. However, in the historical negative sampling setting, where the target node of negative edge is sampled from the source node's historical neighbors, the NIF encoding may introduce more false positives on negative samples. This necessitates the importance of FE layer in capturing temporal or shifting signals. It is evident that the FE component holds greater significance this setting.
4.6 HYPERPARAMETER STUDY
In this section, we analyze the impact of the number of sampled neighbors (L) on the performance of FreeDyG. The results presented in Table We observe that the performance tends to be subop- timal when only a few neighbors are sampled (e.g.. L = 10), primarily due to the lack of sufficient information. Furthermore, the optimal number of sampled neighbors varies across different datasets.
8

---

Published as a conference paper at ICLR 2024
L-10
1-30
L-32
L-64
L-100
AUC-ROC
AUC-ROC
AUC-ROC
AUC-ROC
AP
AUC-ROC
Wikipedia
99.15 0.01
99.30 0.01
99.36 a 0.01
99.41 a 0.01
99.23 0.11
99.14 99.40
0.08
99.19 0.02
99.01 0.04
99.10 0.09
Redki
a 0.00
99.06
0.01
99.22
99.37
99.48 0.01
MDUC
07.3
57.44 1.01
88.01 = 0.30
88.17 0.39
89.26 0.20
89.37 + 0.61
89.61 = 0.19
89.93 = 0.35
88.57 0.52
88.71 =
LasFM
a
86.64 0.20
87.14 0.20
87.82 0.26
90.71 0.117
0.20
90.71 0.22
91.53 a 0.19
92.15 0.16
93.42 0,15
Enron
92.09 0.19
0.11
92.24
0.09
92.43
94.00 +
91.93 0.31
Social Evo.
94,01 a 0.04
95, 74 0.05
94.58 : 0.00
46 a 0,4
94.91 a 0.01
96.59 a 0.04
94.03
0.07
95.70 a 0.13
94.06 0.20
94.91 0.18
UC1
94.21 9.35
96.23
94,97 0.26
96.28
95.00
9 94.46
95.99 94.52
Table 3: Performance comparison of different sample numbers of historical neighbors.
Specifically, datasets characterized by a higher frequency of the "shift" phenomenon (e.g., MOOC, LastMM) Poursafaci et al. (2022) require a larger number of samples to effectively capture hidden patterns and achieve optimal performance. This arises from the essential requirement of FreeDyG to learn from more nuanced frequency domain features of interaction behaviors over time (i.e., DFT of longer time sequence comprises a greater number of refined frequency domain components).
RELATED WORK
Dynamic graph representation learning. Existing methods can be roughly grouped into two cate- gories: discrete-time dynamic graph (DTDG) models and continuous time dynamic graph (CTDG) models. DTDG models typically generate discrete snapshots of graphs and fuse information ex- tracted from different snapshots Pareja et al.] (2020); (Zhao et al. (2019): Yang et al.) However, they often suffer from information loss, as time discretization can miss out on capturing important inter- actions. To solve it, there is growing interest in designing CTDG models that treat dynamic graph data as link streams and directly learn node representations from interactions that occur continu- ously. Specifically, existing CTDG models Xu et al. (2020); Wang al. (2021c a commonly em- ploy Recurrent Neural Networks (RNNs) or Self Atte-Atttniion mechanisms as their basic mechanism. Building upon this, some methods incorporate additional techniques like memory networks Rossi eet al. (2020); Kumar et al.) (2019); Trivedi et al.] (2019), ordinary differential equations (ODE) Luo al. ) Liang et (2022). random walk (RW) Wang et al. (2021d); [Jin al. (2022b), tem- poral point processes (TPP) Chang et al. (2020): Huang et al. (0200) to better learn the continuous temporal information.
Fourier Transform. Fourier Transform is a basic technique in the digital signal processing do- main Reddy (2018); |Pitas| (2000) over the decades. Numerous studies have integrated it into areas like computer vision Huang et al. (2023): Wang et al. (2020) and natural language processing Tamkin al. (2020); Lee- horp et al. (2021). ). More recent research efforts have sought to harness the power models for tasks such as long-term time series forecasting Wu et al.)
of Fourier transform-enhancee
2023 Zhou et al. (0022 2021).
CONCLUSION
In this paper, we present FreeDyG, a frequency-enhanced continuous-time dynamic graph model designed specifically for link prediction. Our approach includes novel frequency-enhance MLP- Mixer layer, which effectively captures periodic temporal patterns and the "shift" phenomenon ob- served in the frequency domain. We also introduce node interaction frequency encoder that simul- taneously extracts the information of interaction frequency and the proportion of common neighbors between node pairs. Extensive experiments conducted on various real-world datasets demonstrate the effectiveness and efficiency of our proposed FreeDyG model. In future work, we plan to extend our method to handle continuous-time dynamic graphs with both edge insertion and deletion events.
ACKNOWLEDGEMENT
The research presented in this paper is supported in part by the National Natural Science Foundation
of China (62372362).

---

Published as a conference paper at ICLR 2024
REFERENCES
Khushnood Abbas, Alireza Abbasi, Shi Dong, Ling Niu, Laihang Yu, Bolun Chen, Shi-Min Cai, and Qambar Hasan. Application of network link prediction in drug discovery. BMC Bioinformatics, 22, 04 2021. doi:
-20-00--......
Unai Alvarez-Rodriguee Federico Battiston, Guilherme Ferraz de Arruda, Yamir Moreno, Matjaž Perc, and Vito Latora. Evolutionary dynamics of higher-order interactions in social networks. Nature Human Behaviour, 5:1-10, 05 2021. doi: 00.1038/44116--000-00000
G. Bruun, z-transform dft filters and fft's. IEEE Transactions on Acoustics, Speech, and Signal Processing, 26(1):56-63. 1978.
Xiaofu Chang, Xuqin Liu, Jianfeng Wen, Shuang Li, Yanming Fang, Le Song, and Yuan Qi. Continuous-time dynamic graph learning via neural interaction processes. In CIKM, 2020.
Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi. Do we really need complicated model architectures for temporal networks? In ICLR, 2023.
Zhipeng Huang, Zhizheng Zhang, Cuiling Lan, Zheng-Jun Zha, Yan Lu, and Baining Guo. Adaptive frequency filters as efficient global token mixers. In ICCV, 2023.
Zijie Huang, Yizhou Sun, and Wei Wang. Learning continuous system dynamics from irregularly- sampled partial observations. In NeurlPS, 2020.
Zepeng Huo, Xiao Huang, and Xia Hu. Link prediction with personalized social influence. AAAI, 32, 04 2018.
Ming Jin, Yuan-Fang Li, and Shirui Pan. Neural temporal walks: Motif-aware representation learn- ing continuous-timg dynamic graphs. In NeurIPS, 2022a.
Ming Jin, Yuan-Fang Li, and Shirui Pan. Neural temporal walks: Motif-aware representation learn- ing on continuous timm dynamic graphs. In NewrIPS, 2022b.
Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in tem-
poral interaction networks. In KDD, 2019.
James Lee-Thorp. Joshua Ainslie, Ilya Eckstein, and Santiago Ontanon. Fnet: Mixing tokens with fourier transforms.
10:::2205503824 2021.
Bo Liang. Lin Wang, and Xiaofan Wang. Autoregressive gnn-ode gru model for network dynamics. wrXiv:2211.10594 2022.
Linhao Luo, Gholamreza Haffari, and Shirui Pan. Graph sequential neural ODE process for link prediction on dynamic and sparse graphs. In Tat-Seng Chua, Hady W. Lauw, Luo Si, Evimaria Terzi, and Panayiotis Tsaparas (eds.), WSDM, 2023.
Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma. Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao B. Schardl, and Charles E. Leiserson. Evolvegen: Evolving graph convolutional networks for dynamic graphs. In AAAI, 2020.
Ioannis Pitas. Digital Image Processing Algorithms and Applications. John Wiley & Sons, Inc., USA, 1st edition, 2000. ISBN 0471377392.
Farimah Poursafaci, Shenyang Huang, Kellin Pelrine, and Reihanch Rabbany. Towards better eval- uation for dynamic link prediction. In NeurlPS, 2022.
G. Obi Reddy. Digital Image Processing: Principles and Applications. Springer International Publishing. 2018.
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti. and Michael M. Bronstein. Temporal graph networks for deep learning on dynamic graphs. CoRR, abs/2006.10666 2020.
10

---

Published as a conference paper at ICLR 2024
Weiping Song, Zhiping Xiao, Yifan Wang, Laurent Charlin, Ming Zhang, and Jian Tang. Session- based social recommendation via dynamic graph attention networks. WSDM, 2019.
Alex Tamkin, Dan Jurafsky, and Noah Goodman. Language through prism: A spectral approach for multiscale language representations. In NeurIPS, 2020.
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Un- terthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. Mip-mixer: An all-mlp architecture for vision, 2021.
Rakshit Trivedi, Mchrdad Farajtabar, Prasenject Biswal, and Hongyuan Zha. Dyrep: Learning representations over dynamic graphs. In ICLR, 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017.
Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P. Xing. High-frequency component helps explain the generalization of convolutional neural networks. In CVPR, pp. 8681-8691, 2020.
Lu Wang, Xiaofu Chang, Shuang Li, Yunfei Chu, Hui Li, Wei Zhang, Xiaofeng He, Le Song, Jingren Zhou, and Hongxia Yang. TCL: transformer-based dynamic graph modelling via contrastive learning. CoRR, abs/2105.07944 2021a.
Wen Wang, Wei Zhang, Shukai Liu, Qi Liu, Bo Zhang, Leyu Lin, and Hongyuan Zha. Incorporat- ing link prediction into multi-relational item graph modeling for session-based recommendation. IEEE Transactions on Knowledge and Data Engineering, PP:1-1, 09 2021b.
Xuhong Wang, Ding Lyu, Mengjian Li, Yang Xia, Qi Yang. Xinwen Wang, Xinguang Wang, Ping Cui, Yupu Yang, Bowen Sun, and Zhenyu Guo. APAN: asynchronous propagation attention network for real-time temporal graph embedding. In SIGMOD, 2021c
Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks. In ICLR, 2021d.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang. and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In ICLR, 2023.
Da Xu, chuanwei ruan, evren korpeoglu, sushant kumar, and kannan achan. Inductive representation learning temporal graphs. In ICLR, 2020.
Menglin Yang, Min Zhou, Mareus Kalander, Zengfeng Huang, and Irwin King. Discrete-ting tem- poral network embedding via implicit hierarchical learning in hyperbolic space. In KDD.
Le Yu, Leilei Sun, Bowen Du, and Weifeng Lv. Towards better dynamic graph learning: New architecture and unified library. CoRR, abs/2303.13047 2023.
Dong Yuxiao, Jie Tang, Sen Wu, Jilei Tian, Nitesh Chawla, Jinghai Rao, and Huanhuan Cao. Link prediction and recommendation across heterogeneous social networks. In ICDM, 2012.
Ling Zhao, Yujiao Song, Chao Zhang, Yu Liu, Pu Wang, Tao Lin, Min Deng, and Haifeng Li. T-gen: A temporal graph convolutional network for traffic prediction. IEEE transactions on intelligent transportation systems, 21(9):344-3388 2019.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In ICML, 2022.
11

---

Published as a conference paper at ICLR 2024
A
DETAILS OF DATASETS.
Dataset
Nodes
Edges
Unique Edges Node/Link Feature
Time Granularity
Duration density
WIKI
9227
157474
18257
0/172
Unix timestamp
1 month 4.30E-03
REDDIT
10984
672447
78516
0/172
Unix timestamp
1 month 8.51E-03
MOOC
7144
411749
178443
0/4
Unix timestamp
17 month 1.26E-02
LastFm
1980
1293103
154993
0/0
Unix timestamp
I month
5.57E-01
Enron
184
125235
3125
0/0
Unix timestamp
years
5,538+00
Social Evo.
74
2099519
4486
0/2
Unix timestamp
months 5.36E+02
UCI
1899
59835
20296
0/0
Unix timestamp
196 days
3.66E-02
Table 4: Dataset statistics
A dataset tracking user edits on Wikipedia pages, is represented as a bipartite interaction graph that contains interactions(edits) between users and Wikipedia pages. Nodes represent users and pages, and links denote the editing behaviors with timestamps. Each interaction is associated with a 172-dimensional Linguistic Inquiry and Word Count (LIWC) feature. This dataset addition- ally contains dynamic labels that indicate whether users are temporarily banned from editing.
Wik [
A dataset tracking users posting in Reddit, is bipartite and records the posts of users under subreddits for one month. Users and subreddits are the nodes, and links are the timestamped posting requests. Each link has 172-dimennn 12--dimnsiionll LIWC feature. This dataset also includes dynamic labels representing whether users are banned from posting.
Reddi 2
MoOd is bipartite interaction network online sources, where nodes are students and course content units (e.g., videos and problem sets). Each link denotes student's access behavior to specific content unit and is assigned a 4-dimensional feature.
LastFM is bipartite and consists of the information about which songs were listened to by which users over one month. Users and songs are nodes, and links denote the listening behaviors users.
Enron is an email correspondence dataset containing around 50K emails exchanged among em- ployees the ENRON energy company over a three-year period. This dataset has no attributes.
Social E mobile phone proximity network that monitors the daily activities of entire undergraduate dormitory for a period eight months, where each link has a 2-dimensional feature.
UCT is Facebook-like, unattributed online communication network among students of the Uni- versity of California Irvine, along with timestamps with the temporal granularity seconds.
B DETAIL DESCRIPTIONS OF BASELINES
link ( the link feature between V; f2 timestamp and m, the timestamp that U; latest interact with other node. When an interaction between 0s vy happens at timestamp 1, JODIE updates the temporal embedding using RNN by x; (t) = (N,((, (m;(m.) 2, (mz), link (t), - m²), Then, the embedding of node ;; at timestamp lo is computed by h²(4o) = (1 + (1o m.)w) (ma).
JODIE is an RNN-based method. Denote (t) as the embedding of node v at timestamp f, x
TGAT is seff-attention-bunt based method that could capture spatial and temporal information simul- taneously. TGAT first concatenates the raw feature x; with a trainable time encoding =(t), i.e., x, (1) [r x(t] and zt) cos(tw (tw + b). Then, self-attention is applied produce node represen- (h.,)) Nr, (i)). where N₂, (i) denotes the neighbors of node ;; lation (o) = 5 AM (z;(4)), AM.
Download from http://sunn.ttaafoodddddddddoiit
2 DDownload from http://wwnaaaaaffootccccc///lled
Downloadd from http://sag..aamooodddedd//ddeeen
"Download from http://wwwgssaaaoorucccc////lllt
Download from http:///ennnddooorrcccc/m///11/f
Download from http:///eonahooooocccmm////////1
7 Download from http:://eennbb.orgg/cccm///////1 Y106688300
12

---

Published as a conference paper at ICLR 2024
at time to and hn denotes the timestamp of the latest interaction of node u. Finally, the prediction on any node pair at time to is computed by MLP((h(tto)] hy (to))).
TGN is mixture of RNN- and self-attention-busee method. TGN utilizes a memory module to store and update the (memory) state (t) of node i. The state of node is expected to represent i's history in compressed format. Given the memory updater as mem, when an link ezj (t) connecting node i is observed, node i's state is updated as st(t) mem mem(ss(t") (s,(t), 8 memory state of node just before time L. is the concatenation operator, and node is i's neighbor connected by eij (4). The implementation of mem is a recurrent neural network (RNN), and node i's embedding is computed by aggregating information from its K-hop temporal neighborhood using self-attention.
Ilee, (t)). where s; s,(t)) is the
DyRep is an RNN-based method that updates node states upon each interaction, It also includes temporal-attentive aggregation module to consider the temporally evolving structural information in dynamic graphs.
DyGFormer is a self-attention based method. Specifically, for node n, DyGFormer just retrieves features of involved neighbors and links based on the given features to represent their encod- ings. Instead of learning at the interaction level, DyGFormer splits each source/destination node's sequence into multiple patches and then feeds them to transformer (Vaswani al. 2017).
GraphMixer is a simple MLP-based architecture, that uses fixed time encoding function rather than the trainable version and incorporates into a link encoder based on MLP-Mixer to learn from temporal links. A node encoder with neighbor mean-pooing is employed to summarize node features.
TCL is self-attention based method. It first generates each node's interaction sequence by per- forming a breadth-fing first search algorithm on the temporal dependency interaction sub-graph. Then, presents graph transformer that considers both graph topology and temporal information to learn node representations. also incorporates a cross-attention operation for modeling the interdepen- dencies of two interaction nodes.
CAWN is a mixer of RNN- and seff-attention-based method that proposes to represent network dynamics by extracting temporal network motifs using temporal random walks (CAWs). CAWs replace node identities with the hitting counts of the nodes based on set of sampled walks to establish the correlation between motifs. Then, the extracted motifs are fed into RNNs to encode each walk as representation and use self-attention to aggregate the representations of multi-walks into a single vector for downstream tasks.
EdgeBank is a pure memory-based approach without trainable parameters for transductive dynamic link prediction. It stores the observed interactions in the memory unit and updates the memory through various strategies. An interaction will be predicted as positive if it is retained in the memory, and negative otherwise.
C EVALUATION DETAILS OF SAMPLING STRATEGIES
In the evaluation stage, we combine the original test set, treated as positive samples, with additional negative samples. Specifically, the ratio of positive samples to negative samples is set to 1:1. We employ three different negative sampling strategics, as described in Poursafaci et al. (2022): 1) Ran- dom Negative Sampling Strategy: This strategy randomly samples negative edges from nearly all possible node pairs within the graphs. 2) Historical Negative Sampling Strategy: Here, negative edges sampled from the set of edges observed in previous timestamps but are absent in the cur- rent step. 3) Inductive Negative Sampling Strategy: Negative edges are sampled from unseen edges that were encountered during the training phase.
13

---

Published as a conference paper at ICLR 2024
NSS
Datasets
JODIE
DyRep
TEAT
TGN
CAWN
EdgeHank
TCL
GroplMiieer
DyGiFormer
FreDDyy
Wikipedia
96.33 = 0.07 94.37 + 0.09 96.67 a 0.07 98.57 a 0.00 98.54 0.04 90.78 0.00 95.84
0.18 96.92 a 0.05 98.91 a 0,02 99.41 0.01
Reddit
98.31 a 0.05 96.1 17 0.05 98.47 0.02 98.60 0.06 99.01 a 0.01 95.37 a 0.00 97.42
0.02 97.17 0.02 99.15 0.01 99.50 0.01
MOUC'
83.81
a 0.58 87.11 0.19 91.21 1.15 80.38
0.00 K3.12
0.18 84.01 0.17 87.91
mi
LasFM
70:49 =: 1.66 71.16 a 1.89 71,59 0.18 78:47 a 2.94 85.90 a 83.77
0.00 64.06
1.16 73.53 0.12 93.05 a 0.10 93.42 0.15
Enron
$7.96
84.89 = 3.00 68.89 1.30 88.32
90.45
$7.05 0.00 75.74
0.72 34.38
98.33 0.13 94,01 0.11
Social Evo.
92.03 =: 0.46
90.76 a 0.21 94.76 0.16 0.11 87.34 0.08 81 a 0.00
94.84
0.17
95.23 0.07
96.30
a 0.001 96.59 0.04
UCI
90.44 0:49
68.77 2.34
78.53
0,74
92,03
1.13 93.57 a 0.08
77.30
0.00
87.82
1,36 91.81 a 0.67 94.49
0.26
95,00
0.21
Aug.Rank Rank
8.57
6.14
3.78
4.56
6,14
4.78
2.14
1.14
Wikipodiia
80.77
0.733 77.74
0.33
82.87
0.22
82.74
0.33
67.84 77.27 + 0.00 85.76
+
0:46 87.68 a 78.80 1.95
82.76+ 0.30
Reddit
50.52
79.33
0.16
0.11
0.30
76.19
77.80
85.92
0.30
MODC
82.75 0:88
81.06
0.94
80.81
0.67
88.00
1.30
71.57
1.07
61.00
10:0000000
0.00 72.09
0.56 76.68 + +.40 37.04 + 0.35 H8.32 0.99
bist
LastEM
75.22 2.36 74.65
1.98 64.27 0.225 77.97
67.88
78.09
0.00
3.11
64.21
2.73 78.78
0.35
2 0.12
Faron
75.30 2.37
27.09.42.:2
65.10
79.59
0.00
67.95
0.88
75.27 1.14
75.74 0.72
Social Evo.
90.06
2.115 93.12
0.344
93.08
a 0.59
0..53
67.43
0.15
85.81
0.00
93.44
0.068
94.39
0.31
92.28
0.07
97.42 0.00
tCC
3.54
3.12
58.89
1.57
77.25
57.86
0
69.56
72.25
3.46
77,54
76.97
80.38 0.26
Avg. Rank
4.78
5.85
6.01
3.85
6.54
7.14
4.85
4.14
2.85
2.14
Wikipedia
70.96
$1.93
0.22
$0.97
81.73
$2.19
Reddit
83.51 0.15 K2.90
0.31
87.13
0.20
01.24
88.04
0.29
85.93 + 0.00 84.67
+ +.22
82.21
0.15
86.25
84.38 0.21
MOOC
66.63
2.30 63.26 à 1.01 1
77.44
70.30
1.43 48.18
0.00 70.36 0.37 72.45 0.72 80,76
0.76
7
ind
LasFFM
3.49
63.09 0.21
4.24
67:92=0:44
77.37 0.00 46.93 2,59 60.22 0.32
Enron
70:92 à: 1.05 68.73 a: 1.34 60.45 2.11
71.34
= 0.50 75.00 1: 0.00 67.64 2:8.6 71.53 a 0.85 74.07 à. 0.64 77.27 0.61
Social
90.01
93.07
0.38 92.94 0.61
95.24 (1.56
0.15
87.88
0.00 93.44
0.72 94.22
22.51
0.00
98.47 0.02
UCI
64.14 1.26 54.25 à 2.01
60.80 a 1.01
64.11 à 1.04
58.06
0.26
58.03
0.00
70.05 à: 1.86 74.59 a 0.74 65.96
1.18 75.39 0.57
Asg. Rank
6.0
5.77
4.77
3,46
4.0
5:35
5.46
4.57
2.85
2.14
Table 5: AUC-ROC for transductive dynamic link prediction with different sampling strategies.
NSS
Dasasess
JODIE
DyRep
TGAT
TGN
CAWN
TCL
GroghMiser
DyGisommrr
FryeDyyG
Wikipedia
94.33 0027 91.49
0,45 95.90 1 0.09
97.32 a 0.03
98.03 2004
95.57 0.20
96.30 0.04 98.48 2.03
99.01 : 0.02
Reddit
0.152+0113
96.05
0.12 96.98 + 0.04
97.39 -.07
98.42
0.02
93.80
0.07
94.97
0.05
+
08.84 0.01
MOOC
13.
0.49
86.8410.17
$1.56
81.43
82.77
0..62
0.74
rad
LaslM
3.21
82.24
1.51
76.99
0.22
82.61 3.15
87.82
70.84
0.55
80.37
a 0.18
94.08
0.08
94.32 0.03
Enrcn
81.96
4.22
64.33
78.83 1.11
87.02
0.50
22.33
0.99
76.51
0.71
90.69
0.26
89.51 0.20
Sucial Evo.
9.29
91.18
0.49
93.41
0.19
93.43
0.59
84.73
0.27
13.71:0.11
94.09
0.07
95.29
0.03
96.41 0.07
UCI
=8.80
58.08
1.81
77.64 e
86.68
2.29
90.40
84,40
1.82.
89.30
0.57
92.63
93.01
0.08
Aank
4.69
5.85
5.31
2,85
3.38
5.31
6.01
186
1.43
Wikipedin
0.58
57.54
1.09
78.38 0.20
75.75 0.29
62.04 1 0.65
79.79
0.96
82,87
0.21
70.33
0.25
$2.08
0.32
Reddit
11.89:0.99
0.37
64.55
64.94
61.43
0.26
64.27
0.13 66.08
0.31
MD0C
64.48 1.64
64.23
1.29
74.08
0.27
77.69 3.55
71.08
0.94
69.82
0.32
72.55
a 0.84
80.7
0.63
81.52 0.37
LasFM
3:26
68.79
1.08
69.89 1 0.28
66.99 5.62
67.69
0.24
55.83
1.85
70.07
0.20 70.73
0.37
72.63 0.16
Enron
6.57
+1.50:22.50
57.84
62.68 1.09
62.25
0.40
64.06
1.02 68.20
0.42
70,09
Sucial Evo.
0.55
87.93
1.05
91.87
0.77
92.10
1.22
83.54
0.24
93.28
0.60
93.62
0.35
96.91
0.09
96.94 0.17
UC1
60.24 1.94
51.25
2.37
62.32
62.69
0,90
56.39
20146
1.94
75.98
0.84
65.55
1.01
76.01
=0.75
Avg. Rank
5:08
6,00
4.23
4.38
5.38
3.69
3.08
2.85
1.14
Wikipedia
0.53
$7.54
78.38
0.20
0.7776
0.29
62.00
0.65
79.79
0.96
82.88
0.21 68
2.82
83.17
0.31
Reddit
0.39
60.44
0.57
64.39
0.27
64.35
0.50
64.91
0.21
61.36
0.26
64.27
0.13
64.00
0.25
64.51 0.19
MOOC
64.48
1.64
64.22
1.29
74.07 a 0:27
77.68
3.55
71.69
0.94
69.83
0.32
72.52 0.84
80.77 0.63
75.81 0.69
LastFM
63.44:3.22
68.79
0.28
66.99
67.68 0.24
55.88 a 1.35 70.07 0.20
20.73
0.37
Ennon
61,50
57.83 e 2.18
62.68 t 1.09
62.27 0.40
64.05 e 1.00
68. 19 1.63
65 (9.79 0.42
68.79 a 0.91
Social Eivo.
88.53
87.93
1,05
91.88
0.72
92.10
1.22
83.54 0.24
93.28
0.00
93.62 0.35
96.91
0.09
96.79 0.12
UCT
60.27
1.94 51.26 2,40
63.79
1.17
62.66
0.91
56
0.11
70.42 e 1.93
75.07 0.85 65. 65.58 1.00
73:41
0.88
Avg. Rank
4.60
5.85
5.31
2.85
3.38
5.31
6.0
1.86
1.43
Table 6: AUC-ROC for inductive dynamic link prediction with different sampling strategics.
14

---

Published as a conference paper at ICLR 2024
D SUPPLEMENTARY MATERIALS
D.1 THE PSEUDO-CODE OF FREEDYG
In A we show the pseudo-code of the training process of FreeDyG. In addition, following the suggestion of the reviewers, we briefly describe the procedure of FFT in Algorithm
Algorithm Training pipeline for FreeDyG
Input: CTDG G, node pair (u, v) with a specific timestamp (t), the neighbor sample number I, maximum training epoch of 200, early stopping strategy with patience 20.
Output: The probability of the node pair interacting at timestamp initial patience 0
2: for training epoch = 2... do
3: Acquire the most recent first-hop interaction neighbors of nodes u and from prior to timestamp / as 5 and S::
4: for S% and S parallel do
5: Obtain node encoding Z and edge encoding Z Z,E
6:
Obtain time encoding Z as Cong et al. (2023);
7:
Obtain NIF encoding Z from Equation 3;
8:
Z - Z + - + Z + -
9:
for Frequency-ehanced MiP-Mixer Layer
10:
z + F(Z,) with Equation! I;
2 W Z
12:
7! F-1
2 with Equation 2:
13:
7/ LayerNorm (44 Dropout
7!
Feed z into MI.P-Mxer Layer with Equation 10; end for
15:
16:
Weighted aggregation with Equation 11;
17:
end for
18:
Conduct link prediction with y + (Mo
19:
Loss Lp + - E (y: * logū, + (1 w1) log(1 - ú.))
if current epoch's s metries worse than the previous epoch's 's then
21:
patience patience +1
22:
else
23:
Save the model parameters from the current epoch end if
24:
25:
if patience = 20 then
26: Exit training process
27:
end if
28:
end for
15

---

Published as a conference paper at ICLR 2024
Algorithm 2 The Pseudo-Code of Fast Fourier Transform
1: Input: Sequence X of length L
2: Output: Y (FFT of }
3:
4: Function NextPowerOf2(L)) {Return the next power of greater than or equal to I.)
5: N < 1 {Initialize n to the smallest power of 2}
6: while < do
7: N N x (Double until it is greater than equal to L}}
8: end while
9: return N
10:
Function FFT(X, N)
12: if N = then refurn X
14: end if
Xeven f- even-index elements of X(O, .0,,...... N 1]
16: Xodd + odd-index elements of X ...,, N 1] (Divide the sequence into two parts: even-
indexed and )
17: Yoee FFTe(eeven N/2) {Recursively apply FFT even part)
18: You t FFT 2/2) (Recursively apply FFT to odd part)
19: Y < new array of size N {Initialize array Y of size N}
20: for = == N/2- 1 do
21: t 22ikk/N (Compute the twiddle factor]
22: Yιk] < Yoven [k] + t Youd [k] {Compute the FFT value for index k)
23: Ylk [k++N/2) + Yeven [k] - t Ykaakk] {Compute the FFT value for index + N/2]
24: end for
25: return Y
27: Procedure:
NextPowerOf2(1) (Pad the sequence length to the next power 2}
29: Xpddee X
30: while length of Xpadded < N do
31: append 0 to X Xpadded (Pad the sequence with zeros}
32: end while
33: Y t FFT(Xpadded) N) (Apply FFT to the padded sequence)
D.2 THE EXPERIMENTAL RESULTS ON MICRO-F1 AND MACRO-FI METRIC
In Table 7-10, conduct experiments all datasets under three negative sampling strategies with both transductive and inductive link prediction. We use Micro-F1 and Macro-F1 scores as the metric. We note that the results similar to those on AP and AUC-ROC. Specifically, our FreeDyG outperforms other baselines in most scenarios and the average ranking of FreeDyG is close to 1.
16

---

Published as a conference paper at ICLR 2024
NSS
Datasets
JODIE
DyRap
TGAT
TGN
CAWN
TCL
GraphMixer
Dytikomer
FreellyG
Wikipedia
57.47 = 0:99
0..42
89.09 20.15
95.09 2.05
94.63 0.08
88.0
+0.00
0.22 94.00 a:00:00 0.53
95.92 + 0.09
moditt
91 0:49
93.03
0.2.1
0.11
95.55
0.03
91
0.07 91.24 a 0.04
95 0.20
9683 0.07
MOUC
7800
78.26 +
70.05
0.32
74.76
76.74
md
LasFMM
3.99
61.01
5.668
65.01 a 0.26
$4.93
3:44
80.94
0:47
0:47
59.50
0.33
66.96 a 0.44
$8.89 a 0.38
Enron
0.874
77.01
63.99
2.222
84.53
2.79
76.47
88:37 0.33
58.91
Social levo.
4:226
83.06
#.662
90.42
0.12
20225
0.36
90.56
0.188
90067
0.14
2:126
9387
0.07
UCT
67.61 2.97 63.49
9.1.7
70.29
1,18
79.39
0.67
0.184
76.15
0.58
83.30
0.99
89:34 0.07
90.25 0.08
Avg. Rank
7.71
6.43
5.71
4.28
4:43
7.00
2.57
Wikipodia
65 1.27
63.35
62.53
1.53+044
50.67
1.26
50.18 18
0.23
67.61 + 0.88
69.94
+
66.73 + 1,32
Reddit
59.76:0.26
6.740:0.76
59.27
60.55
5..0
0.10
62.74
MO0C
63.74+9.08
0.55
72.00 + 0.37
77.77
0.87
64.20
1.42
59.00 + 2.06
65.58 0.12
75.10 5.73
76 18 0.87
hist
LasFM
1..71 22.52
55.26
50.86
1.06
59.85
0.68
38.98
2.52
56.29
68.59 0.55
67.81 0.40
Fanon
6.31
56.38
67.63
0.94
0.34
61.48
1.79
66.97 1.28
Social Evo.
0.25
86.41
0.38
83.47
81.81 1.28
86.81
0.17
87.70
$4.96 a 0.96
99.42 0.66
90.02 0.83
UCT
6.31
9..65
0.444
5672
44
0.02
78.66
2.00
39
Avg. Avg.Ronk Rank
6.14
3,86
6.57
4.71
7.00
6.00
4.00
4.71
2.00
Wikipedia
5.79
1.011
57.25
5
6.11
58.
Reddit
58.07 0.12
0..52
65.52
0.65
58.57
0041
58
58.76
66.09
0.03
63.94 = 0.19
48.65 0.38
63.09 0.78
MOOC
51.69
57.96
0.33
61.54 0.64
61.75
2,58
6
60.11
1.55
62.50
2.26
70,06 à 1.45
LaeFM
1.55
51.57
0:45
39.30
53.42
66:31 0:30
66.57 +
Enron
0.45
61.26
2.27
$4.75 à 1,25
60.51
0.68
61.57
1.18
65.66 a 0.50
58.24
0.73
56.80 2 0.74
60.09 à: 1.43
Social
7.442
0.440
83.68
86.13
1.02
8..72
+.33
84.94
1.05
89.40
0.65
89.87
UCT
1.64
53.07
2.71
$1.69
à 1.25
49.75
0.46
44
0.02
50.76 à 0.84
69.43 à 0.21
46.53
0.84
60.00 a: 2.61
Avg.Rank
7.14
5.28
4.86
6.00
5.00
4.28
4.14
5.71
2.57
Table 7: micro-F1 for transductive link prediction with three different negative sampling strategies.
NSS
Duiasets
JODIE
DyRep
TGAT
IGN
CAWN
TCL
GraphMixer
DyGioommer
FreeDyyG
Wikipodia
87.45 a 0.99
99.69 0.155
93.09 à 0.05
94.63 a 0.08
$8.06 i 0.50
40.57 4
95.92 0.09
redkit
91.99 + 0.49
93.03 0.21
93.90
0.17
94.14
0.07
95.55 0.03
91.68 0:07
91.24 0.04
0.28 0.20
96.03 x 0.07
MO0C
72.13
77.89
78.23
84.29
1
md
LastFM
55.61 4.77
55.77 45.49
6.220
a.422
$0.86 a 0.51
58.93
0.60
66,
66.70 0.44
$7.95 à 0.60
88.02 a 0.47
Fnron
1.25
76.55
2.95
77.75
3.63
84.23 0:20
69.66 3.13
76.04
58.01
8883
Social Evo.
78:47 3.55
83.58 0.66
90,41 1.112
0.22
00.5810.29
90.55
4 1.188
90.67 4 0.14
0.10
93.87 1 0.07
UCT
63.43 4.37
61.57 11.71
70.19
78888
0.76
87.76 0.13
76.01 + 0.62
82.87 1.07
89.24 0.09
90.22 0.08
Aug. Runk
5.71
6.57
2.52
Wikipadia
62.92 + 1.80
61.21 +.799
59.41 8.55
55.02 + 1.99
38.00
0.58
66.34 1.02
68.70 1.74
45.72 1.45
61.52 1.25
Reddit
$4.11 0.332
56.89
55.45
0.95
1.
36.58
58.71
MOOC
59.01 14.15
7..14
72.73
0.46
77.44 0.80
63.98
1.37
58.61
2.32
65.07 0.15
74.92
5.87
76.03 0.87
hist
LaslMM
48.17 à
53.69 2.28
54.23
35.63
58.75
0.94
35.31
55.68
67.28 0.33
67.48 0.42
Enron
61.92 + 0.06
66.87
55.35
2.12
2.12
59.69 +
64.98 1.91
05.33 1.70
Social Evo.
71.95 a 0.07
96.37 0.388
83.37 a 0,50
$4.69 à. 1.35
$7.75
1.34
87.66
5.21
$4.89 à 0.98
6..73
90.17 0.64
UCT
57.97 5.887
48.76 12.80
46.15
49.27
0.75
35.98
0.05
42.58
78.43
$7.90 2.75
65.07
Avg. Runk
6:00
4.00
6:57
4.85
685
5.71
3.85
4.85
2.28
Wikipodia
46.58
47.95 0.05
55.48
51.12
0.58
42.85
0:57
0.18
36.76
Reddit
51.34
0.41
62.43
0.94
52.29
0.75
52.47
0.51
63.4
61.04
0.29
33.66 0.30
61.25 225.4000 0.81
MOOC
40.70 60.70 9.95
54 0.92
63.63
0.98
58. 48
2.91
62.73
0.88
59.11
1.72
61.73 0.45
66
a:20
68.31
LasEM
40.70
54.07
24.61
1.62
63.29
0.89
52.41
51.37
Furon
50.58 + 1.45
58. 24 2.84
53.31
1.42 56.00 + 0.59 62.66
1.94
64.93 a 0.70
55.67 0.51
51.37
1.04
55.79 1.06
Social
72.07
86.45 0:440
83.60
86.05
1.07
88.31
0.16
2.38
$4.88
89.25
0.70
$9.94 0.47
UCT
54.41
48.49 6.799
49.11
0:07
35.97
0.32
37.56
0.05
48.18
2.13
69.24
0.21
38.10
2.10
58.47
Aagk
7.38
5.57
4.38
6.71
4.57
3.85
4.14
6.00
2,57
Table 8: macro-Fl for transductive link prediction with three different negative sampling strategics.
17

---

Published as a conference paper at ICLR 2024
NSS
Dotssets
JODIE
DyRep
TGAT
TGN
CAWN
TCL
GrapMaaeer
Dytisamer
FreebyyG
Wikipedia
84.03 = 0.89
0.26
58.20 0.14
91.22 20227
9..6
87.9
#0227
89233 2.09
94.42 0.05
redii
88.84
88.99
0.48
91.06
0.19
0.024
86.39
2.21
87.88
0.15
94
95.33 0.09
71.19 1.66
76.89
78.00
0.32
72.01
0.27
73.53
75.24
77.81 1.12
nd
LasFM
70.71 a 4.65 70.68
6,38
70.10 0.36
S8. 14 6.06
83.55
0.08
65.86 a
0.48
72.93 a 0.85
90.00
90.49 a 0.23
65.50
66.01
3.88
59.29
2.13
71.54
80.29
0.24
09.50
85-15 0.24
$6.34
Social Evoo.
84.78 + 1.22
83.54
1.94
58.30
0.12
$8.49
0.53
09.4220.41
89.74
0.11
89.67
= 0.08
22
2
92.59 + 0.07
UCT
61.25 2.27
56.51
5.09
68.41
0.89
69
2,12
85.26
0.55
71.13
0.7.8
82:44
0.68
88.40 0.07
89.11 0,15
Avg. Rank
7.00
5.8
4:57
6.43
2.43
Wikipediin
48.81 +0.11 48.41
0.06
06376
$2.80
0.24
0.137
61.00+025
6111
48.48
0.27
$7.43 0.75
0.06
5033
54.44022
51.07
MOnc
52.08 + 4.84
59.00
-
0.65
65.59
0.25
63.13
1.90
64.37 1.07
59.76
1.56
62.65
0.24
66.80 1.86
66.2 0.75
hist
LasFM
52.09 23115
53.35
57.99
0.61
50.37 2.555
5..11
4284
57.88
69.75 0.53
0.15
0.83
55.27
1.11
$2.03
0.60
2.40
58.71 0.75
63.3
0.57
57.31 0.47
1.07
Social Evo.
69.77 0.74
21.25
$1.07
0.94
81.29
2.48
0.774
57.09
83.82 à 0.79
89.01
0.53
2.553
56.71
52.37
0.34
$2.72+0009
0.49
44.71
0.05
50.54
0..8
71.02
46.79
62.19 2.09
Arg.Rank
6.14
7.00
4.71
5885
5.43
4.49
3.57
4.86
3.00
Wikipedia
48.81
48.41
0.36
49.37
48.48
Reddit
0,15
51.64
54.07 +003
51.67 0.23
50.86 0.30
54.16 0.24
54.48 0.16
48.10
0.38
51.07 0.36
MOOC
52.98 4.85
59.02 0:65
65.60
63.13
61.37 1.07
59.76
1.56
62.66
66.80 1.87
66.02 0.75
LacFM
53.35
57.99
50.37
0.34
0.53
Enron
60.29 0.883
55.27 a: 1.11
$2.04
0.60
$6.85 2.40
6.77
63.39 a 0.58
$6.06 a 0.35
57.31 0.47
58.23 1.07
Social
0.774
71.25
3.58
81.73
0.92
88.20
0:75
87.70
4.442
83.82
10.01
0.53
90.13
UCI
56.71 à 1.53
52:29
0.255
52.62
0.07
49.
16.16:6
0.51
44.71 0.05
50.58 a 0.90
71.01 & 0.11
46.79 0.99
62.
Avg.Rank
6.14
7.00
4.57
6.00
5:43
4.43
3.57
4.85
3.00
Table 9: micro-F1 F1 for inductive link prediction with three different negative sampling strategies.
NSSS
Dutasets
JODIE
TGAT
TGN
CAWN
TCL
GmppMixer
DyGöoomer
FreeDyyG
Wikipedia
84.02 &
82.55 à 0.27
$8.19 1.144
0.22
92.86 0.11
$7.09 a 0.27
89.23 0.009
23.3
94.42 : 0.05
redfit
88.85 0.72
88.99 + 0:48
91.06 0.42
91.60 0.11
94.24
0.04
86.39
19.99.00 0221
87.88 + 0.15
94.62 0.22
95.33 a 0.09
76.80
27.97
71.78
0.24
73.43
77.71
nd
LastUM
66.82 5:98
67.76 10.04
70.01 0:36
47.46 9.00
83.42 0:008
63
0.52
72.80 0.87
9.22
90.65 1 0.25
0.67
64.21 4.73
58.61
70.97
67.34 3.43
0.224
0.16
Social Evo.
85.00 à 1.14
83.29 2.06
58.78 0.12
$8.43 à 0.56
79.161 0.40
89.74 0.13
89.67 1 0.088
92.16 à 0.08
92.59 0.07
UCT
55.66 + 3.90
54.23 7.99
68.23
6.88
67.33
2.91
85.21
0.53
7.884
82,
12.10:00::5
58.31
0.00
89.05 0.04
Aug. Rank
6:57
Wikipadia
40.56 0.66
41.01 +0:21
$1.08 è 0.68
44.34 +
49.37
#011
57.98
35.52 0.87
50.49 + 0.59
0.57
43.77
4..7
41.72
0.54
44.95
0.25
49.15
MO0C
44.48 = 12.08
56.69 + 1.20)
64.92
0.50
60.30
2.21
64.31
1.06
58.90
1.73
02.02 +
66.39 1.80
63.63
hist
LaslMM
49.01 3.41
47.68 a: 5.06
56.77 0.93
34.73
1.83
$2.05
0.85
38.60
56.11 0.22
68.61 0.37
68.11 0.40
Enroa
49.32 1.57
52.43
1.54
0.68
$2.70 0.87
Social Evo.
67.53 à 1.25
69.55 1.61
$0.90 à 0.97
$0.88
3.84
$8.09
0.57
87.65
4.46
4.46
$8.92 0.46
89.28 0,51
UCT
4.588
47.47 5.98
50.45
35.17
35
00077
2.05
70.33
#.10
38.54
50.05
Avg. Rank
6.14
7.14
4.29
7.43
4.71
4.14
3:57
5.00
2.57
Wikipodiaa
40.55 0.64
0.69
44.34
0.77
37.36
57.97
0.34
35.52
Reddit
42.53 0.57
43.77 + 0.15
46.65 + 0.27
41.72
0.54
39.42
0:38
48.58
0.35
33.16 0.21
49.81 0.76
MOOC
44.48 e 12.08
56 69 1.20
64.92 0.50
60.30
64.31
1.06
58.90
1.73
62.03 0.30
66.39 1.80
65.63 1.53
ind
LastFM
34.73
52.05
38.60
6.90
Faron
55.43 e
32.3 1.57
50.45 0.62
$2.44
3.86
55.05
1.55
62.67
0.69
53.54 0.12
$2,70 0.87
$4,70 0.82
Social
67.53
69.55
81.58
80.88
88.12
0.511
4.46
0.81
58.92
89.28
UCT
50.76 4.56
6
50.32
0.37
35.17
0.02
35.49
0.06
45.5
2.05
70,82
6.11
38.56
7.35
50.05 3.15
Ragk
5.86
6886
4.29
7.14
5.57
4.14
3.57
5.00
2.57
Table 10: macro-Fl for inductive link prediction with three different negative sampling strategies.
18

---

Published as a conference paper at ICLR 2024
D.3 HYPERPARAMETER STUDY
To enhance the presentation of the results in Table 3, we employ Figure to visually depict the trends observed across varying numbers of sampled historical neighbors.
Performmnc on Biki
Performance on Avddl
Performance on MMOX
062
966
162
162
90
Lode
1x30
1437
1420
1-22
Leds
19100
1435
14:5
1+42
(rdd
E4500
Rud82
Performance on LorFM
Perfomancc on Enron
Performance Social
10.
342
02.3
924
00.0
904
824
1422
1um
GAM
1400
1.30
1.30
1-37
1.300
Lus
SteW Evo
Performonce on U.T
Figure 4: Performance comparison on AP of different number of sampled historical neighbors
19

---

Published as a conference paper at ICLR 2024
D.4
ABLATION STUDY OF FREEDyG UNDER RANDOM NEGATIVE SAMPLING SETTING
WIKI
MOOC
AV9
87
+FreshyG
NIF mode FE swl.ALL
/PivllyG
will MP
BIW FE
1I ALL
8 FredyG
WAA WKA NI
HIR FE
uND ALL
LasFFM
Enron
2.14
Preatyt KM
A.T.
+Frestylii
*FrexDyG
Figure 5: Ablation study of FreeDyG under random negative sampling setting, where w/o NS and w/o FE represent our FreeDyG without node interaction frequency encoding module and FE layer respectively.
20


Reviews:
[{"review_id": "SFZxddrVct", "reviewer": "Reviewer_Wgp4", "summary": "The authors propose the FreeDyG graph neural network (GNN) model for continuous-time dynamic graphs. It incorporates frequency-based representations of the nodes to attempt to capture periodic patterns in the dynamic graph. It also contains a novel node interaction frequency encoding approach. The authors demonstrate impressive link prediction accuracy in both transductive and inductive settings compared to other GNNs for dynamic graphs on a variety of real data sets. They also perform favorably in terms of training time and size of the trained model when compared to other methods.\n\n*After author rebuttal:* The authors have partially addressed my concerns in their rebuttal and revision, particularly regarding the usefulness of the frequency encoding. After reading through the other reviews, I am still in support of this paper.", "questions": "1. Is there a way you could inspect your trained model to identify whether any type of periodic patterns are being captured by your frequency-enhanced MLP-Mixer layer?\n2. Why is the NIF encoding more important than the frequency-based representations for improving link prediction accuracy?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Proposed FreeDyG model contains several novel elements, including the design of the node interaction frequency (NIF) encoding and incorporating frequency-based representations.\n- Comparison of accuracy, training time, and model size shown in Figure 2 is a nice inclusion. This shows that improvements in accuracy are not at the cost of significantly increased training time or a very large model.\n- Strong improvements in accuracy compared to other approaches. These improvements hold over different negative sampling strategies and evaluation metrics.\n- Mostly well written and organized paper. In my opinion, the authors made good choices on which results and details should be presented in the main paper rather than the appendices.", "weaknesses": "- The positioning of the paper is a bit deceiving. From reading the paper, it would appear as though the main contribution is incorporating the frequency information. However, from the results of the ablation study in Figure 3, we see that the NIF encoding plays a much bigger role in improving accuracy than the frequency-based representations.\n- The authors present no evidence that the frequency-based representations are actually able to capture periodic patterns, which they used as their motivation for using the FFT.\n\nMinor concerns:\n- Table 3 is probably not the best way to present the hyperparameter study. Typically, one would be looking for trends as you vary the number of historical neighbors $L$. Such trends are difficult to pick out from the table. I would suggest instead using plots with AP or AUC-ROC on one axis and $L$ on the other.\n- Page 7, second last paragraph: \"neigh encoding\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YesuCJe589", "reviewer": "Reviewer_Gxoa", "summary": "This paper proposes a novel method called FreeDyG for link prediction in dynamic graphs. The method devised a novel frequency-enhanced MLP-Mixer layer to learn the periodic temporal patterns and the ”shift” phenomenon present in the frequency domain. The effectiveness of the FreeDyG model was validated on several real-world datasets, showing performance improvement in AUC-ROC against baselines.", "questions": "1. Why does this work only focus on the link prediction? How is applying this work applicable to other graph mining tasks like node classifications?\n2. For the LastFM, what is the summit of the performance when sampling more neighbor nodes? Could you please clarify the experiment details about training every baseline using the same amount of information as FreeDyG in the experiments?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. The proposed frequency-enhanced MLP-Mixer is novel and effective.\n2. The experiments of link prediction are comprehensive. It is conducted on seven datasets and compares the performance against 9 baselines in two dynamic settings, which is solid and comprehensive to validate the effectiveness of FreeDyG in link prediction.\n3. The paper is well written, especially the problem formulation and methodologies.", "weaknesses": "1. The motivation of delving into the frequency domain needs to be further clarified. I am wondering about the intuitions behind capturing the ”shift” phenomenon hidden in the frequency domain.\n2. The authors claim that FreeDyG is the first work that considers the frequency information for dynamic graph embedding, which is overclaimed.\n3. The authors argue that random walk based approaches are computationally expensive. However, conducting Fourier transform are also very computationally expensive. In addition, I think FreeDyG also relied on some random walk based approach to obtain the continuous-time dynamic graph from the raw graph data.\n4. The proposed FreeDyG seems computationally expensive. However, there is no time complexity analysis. The authors are suggested to present the time complexity empirically or theoretically.\n5. In a dynamic graph, some nodes will have more edges, but others will have fewer. Using AUC-ROC as the evaluation metrics cannot tell how good the performance of link prediction for minority nodes. I suggest reporting the Micro- and Macro-F1 scores in the link prediction tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Rvsa1NInrO", "reviewer": "Reviewer_VGzm", "summary": "This paper introduces a new GNN for CTDG. The concept of Node Interaction Frequency (NIF) Encoding appears to be a simplified version of SEAL, a link prediction technique for static graphs, and it further introduces a frequency-enhanced MLP-Mixer layer that has functions of Fourier transform and inverse transform with weight learning. Evaluation is conducted in various experimental settings, including transductive/inductive and three negative sampling strategies.", "questions": "Q1. It would be valuable to discuss how the proposal outperforms other approaches, such as using RNN or transformers, which are known to capture some temporal patterns. This comparison can provide insights into the superior performance of the proposal.\n\nQ2. Could you please clarify whether the optimization is conducted in an end-to-end fashion?\n\nQ3. What is the mean of the circle sizes in Figure 2?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. The overall architecture is well-designed. Of particular interest, the Node Interaction Frequency (NIF) Encoding and frequency-enhanced MLP-Mixer layer are novel and highly effective.\n\nS2. The proposal achieves performance higher than the state-of-the-art. Particularly, achieving high efficiency and high quality is impressive. The ablation study verifies that each technical component is effective for high accuracy. \n\nS3. The experimental settings are detailed, encompassing evaluation experiments across 9 methods, 7 real-world datasets, and various settings including transductive/inductive and three negative sampling strategies.", "weaknesses": "W1. Since the proposal's effectiveness varies across datasets, it is essential to discuss the impact of Node Interaction Frequency (NIF) Encoding and the frequency-enhanced MLP-Mixer layer by investigating the characteristics of each dataset. For instance, if a certain dataset is known to exhibit periodicity, it would be reasonable to understand the benefits of the frequency-enhanced MLP-Mixer layer. Similarly, an analysis should be conducted to determine which data characteristics justified the effectiveness of NIF Encoding.\n\nW2. Some design decisions are not clear. For example, while it is crucial that F^t_* represents common neighbors and their past interactions, further explanation is needed regarding the idea behind Equation 3.\n\nW3. The equation transformation involving w_k^{(t)} in Equation (9) is not clear. Additional clarification is necessary to understand this transformation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lqqT7ZCded", "reviewer": "Reviewer_VgR1", "summary": "In this paper, the authors considered the temporal link prediction task on continuous-time dynamic graph (CTDG) and proposed a novel FreeDyG method, which to my knowledge, is the first work to use a frequency-based discrete Fourier transform (DFT) to capture the evolving patterns of CTDG. The overall presentation of this paper is clear. The experiments are also comprehensive and sufficient that can validate the effectiveness of FreeDyG.", "questions": "According to my background knowledge, a significant property of CTDG is that the difference between two successive time steps can be irregular. However, as shown in Table 4, each dataset has an item 'Duration'. What does this item mean? Does is mean that the time steps of all the datasets are still regularly spaced?\n  \n  In some previous studies, the inductive settings include the prediction between (i) one previously observed node and one newly added node as well as (ii) between two new nodes. It is unclear that the inductive setting in this study refers to which case?\n  \n  According to my understanding, the inductive inference of the proposed method and other baselines relies on the availability of graph attributes (i.e., node and edge attributes in this study). Consider an extreme case, when attributes are unavailable, can the proposed method still support the inductive temporal link prediction?\n  \n  In addition to the commonly used settings of temporal link prediction in this study (i.e., the prediction of unweighted feature links), there are some other studies considered the advanced temporal link prediction tasks for weighted dynamic graphs [1-4], which should not only determine the existence of a future link but also the corresponding edge weight. Can the proposed method be extended to handle such an advanced settings?\n  \n  [1] GCN-GAN: A Non-linear Temporal Link Prediction Model for Weighted Dynamic Networks. IEEE InfoCOM, 2019.\n\n  [2] An Advanced Deep Generative Framework for Temporal Link Prediction in Dynamic Networks. IEEE IEEE Transactions on Cybernetics, 2020.\n\n  [3] High-Quality Temporal Link Prediction for Weighted Dynamic Graphs via Inductive Embedding Aggregation. IEEE TKDE, 2023.\n\n  [4] Temporal link prediction: A unified framework, taxonomy, and review. ACM Computing Surveys, 2023.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "S1. The overall presentation of this paper is clear, which is easy to grasp the key ideas.\n  \nS2. Using the frequency-based Fourier transform to capture the evolving patterns of dynamic graphs is novel and interesting.\n\nS3. The authors conducted comprehensive and sufficient experiments covering both transdutive and inductive settings of temporal link prediction.", "weaknesses": "**W1. From my perspective, some of the motivations regarding model designs need further verification or validation.**\n  \nIn Section 1, the authors argued that RW, TPP, and ODE are computationally expensive. However, the proposed method includes a sampling procedure that samples $L$ first-hop historical neighbors for both source and target nodes. It seems that such a sampling procedure has a complexity similar to those of conventional methods (e.g., RW-based and PPT-based), according to my background knowledge. To verify this motivation, is is recommended to add the comparison of time complexity for sampling/feature extraction in both training and testing phases.\n  \nIt is also suggested to add pseudocode of each procedure (e.g., first-hop historical neighbors sampling, FFT, extraction of node interaction frequency, etc.) even in the appendix since the details of some modules in current version of manuscript are still unclear.\n  \nMoreover, the authors claimed that self-attention acts as persistent low-pass filter and the utilization of DFT can tackle its limitation. How this superiority of the proposed method is validated in the experiments?\n\n***\n\n**W2. As stated in Section 2, the authors only considered CTDG with edge addition events. It seems that the proposed method cannot handle the deletion of edges.**\n  \n***  \n\n**W3. It semes that there are some inconsistent and unclear statements.**\n  \nIn Eq. (1) $n$ starts from 0 but in the 2nd paragraph of Section 2, the authors defined that ${x_n}_{n=1}^N$, where $n$ starts from 1. It is also similar for $ {X_k} _{k=1}^N$.\n  \nIn the 2nd paragraph of Section 3.1, the definitions of $\\alpha$ and $\\beta$ are not given.\n  \nIn Eq. (11), what is the dimensionality setting of $W^{agg}$? It is still unclear how to derive a vector $h_*^t$ based on a matrix $Z_*^l$. Moreover, there is no $t$ in the right side of Eq. (11) but how can we know $t$ in the left side?\n \n***\n\n**W4. There are also some minor errors.**\n\ne.g., 'In addition, We specifically encode' > 'In addition, we specifically encode'", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)