SALoM: Structure Aware Temporal Graph Networks with Long-Short Memory Updater
Authors: Hanwen Liu
Longjiao Zhang
Rui Wang
Tongya Zheng
Sai Wu
Chang Yao
Mingli Song
Venue: NeurIPS 2025 poster ()
Keywords: Graph neural networks
dynamic graphs
temporal information
Primary area: deep_learning

Abstract: Dynamic graph learning is crucial for accurately modeling complex systems by integrating topological structure and temporal information within graphs. While memory-based methods are commonly used and excel at capturing short-range temporal correlations, they struggle with modeling long-range dependencies, harmonizing long-range and short-range correlations, and integrating structural information effectively. To address these challenges, we present SALoM: Structure Aware Temporal Graph Networks with Long-Short Memory Updater. SALoM features a memory module that addresses gradient vanishing and information forgetting, enabling the capture of long-term dependencies across various time scales. Additionally, SALoM utilizes a long-short memory updater (LSMU) to dynamically balance long-range and short-range temporal correlations, preventing over-generalization. By integrating co-occurrence encoding and LSMU through information bottleneck-based fusion, SALoM effectively captures both the structural and temporal information within graphs. Experimental results across various graph datasets demonstrate SALoM's superior performance, achieving state-of-the-art results in dynamic graph link prediction. Our code is openly accessible at https://github.com/wave5418/SALoM.

SALoM: Structure Aware Temporal Graph Networks with Long-Short Memory Updater
Hanwen Liu
Longjiao Zhang Zhejiang University Hangzhou.Zhejiang zhl ahljJoat Joan ju. edu. cn
Zhejiang University Hangzhou,Zhejiang 22451 11111111jj...... edu.
Rui Wang"
Tongya Zheng
Zhejiang University
Zhejiang Provincial Engineering Research Center for Real-Time SmartTech in Urban Security Governance, School Computer and Computing Science,
High-Tech Zone (Binjiang) Institute of Blockchain and Data Security
Hangzhou.Zhejiang rwang210zju. edu. cn
Hangzhou City University doujiang_zheng@de www.eeeenneee.cccm com
Sai Wu Zhejiang University Zhhjzan wusaiezju edu. cn
Chang Yao Zhejiang University Hangzhou,Zhejiang changyēzju. edu, en
Mingli Song Zhejiang University Hangzhou, brooksongozju edu. en
Abstract
Dynamic graph learning is crucial for accurately modeling complex systems by integrating topological structure and temporal information within graphs. While memory-based methods are commonly used and excel capturing short-range temporal correlations, they struggle with modeling long-range dependencies, har- monizing long-range ong-range and short-range correlations, and integrating structural in- formation effectively. To address these challenges, we present SALoM: Structure Aware Temporal Graph Networks with Long-Short Memory Updater. SALoM features memory module that addresses gradient vanishing and information forgetting, enabling the capture of long-term dependencies across various time scales. Additionally, SALoM utilizes long-short memory updater (LSMU) to dynamically balance long-range and short-range temporal correlations, preventing wver-generalization. By integrating co-occurrence encoding and LSMU through information bottleneck-based fusion, SALoM effectively captures both the struc- tural and temporal information within graphs. Experimental results across various graph datasets demonstrate SALoM's superior performance, achieving otate-of- the-art results in dynamic graph link prediction. Our code is openly accessible https://gihuub.coowwwes.tt/ wwwew41wee544
Introduction
Dynamic graphs [35 are essential for modeling intricate systems such as traffic planning 33 ge-
nomics [1],, financial analysis (22). and environmental science 22. Continuous-tinn dynamic graph learning networks 66 17 25 operate in a continuous stream of events, enabling the understanding of
entity interactions. These networks excel in analyzing and predicting data patterns by incorporating
Rui Wang is the corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

temporal information and topological structure ( B11, making them valuable in social network
analysis 26 fraud detection, recommendation systems [O], and predictive maintenance [9], etc.
To capture temporal correlation information between nodes over time, two common approaches used: memory-based methods and neighbor-based sequence methods. Memory-based methods, like JODIE (17), DyRep [28), and TGN (25), assign vector-based memory data to each node to represent historical interaction sequences. These memory data are continuously updated by new events using RNN architectures, preserving temporal information related to all events associated with the respective nodes. On the other hand, neighbor- baaoerr---dddd- sequence methods like DyGFromer [34] and CNEN 7 leverage sequential models such as attention mechanisms and transformer encoders. These methods integrate historical event lists with features from neighboring nodes and time intervals to effectively capture temporal correlations from neighbor sequences,
To characterize topological structural features of a subgraph formed by each node and its historical neighbors, various approaches are commonly used, including message passing, random walk, and specialized structural encodings. Message passing, a traditional method in graph neural networks like TGN (25], encodes the topological structure of the graph into node representations by iteratively aggregating information from neighboring nodes at each layer. Random walk-bused methods, such as CAWN (3, generate random paths in the graph to extract information and contextual relationships from neighboring nodes. Specialized structural encodings like path counting 27 and co-occurrence neighbor encoding [34] enhance structural features and tackle issues like oversmoothing [18] and over-squashing [T] In particular, co-occurrence neighbor encoding computes the co-occurrence of historical neighbors, providing notable advantages in various tasks.
Despite progress in the field, current methods still faces challenges in effectively capturing both long sequence temporal information and graph structure features, hindering model performance. Memory- based techniques struggle with capturing complete long-range neighborhood temporal correlations, due to issues like vanishing gradient and information forgetting in RNN architectures (8). Neighbor- based sequential methods, on the other hand, face difficulties in efficiently balancing long-term and short-term neighborhood features. Moreover, integrating long- long-term temporal and topological structural features poses a challenge. given the issues of over-squashing and difficulty node differentiation [7]. While co-oceurrence neighbor encoding offers a potential solution, directly integrating it with temporal features may lead to conflicts and reduced performance.
To address the above challenges, we introduce Structure Aware temporal graph networks with Long- Short Memory (SLoter(SALoMM,, a dynamic graph learning framework aimed capture long-range temporal features, harmonize long-range and short-range dependencies, and integrating structural features within graphs. SALoM enables the model to discern subtle differences among isomorphic nodes and preserve critical temporal patterns. Our contributions can be summarized as follows:
To address the challenge of capturing complete long-range neighborhood temporal correla- tions, we introduce a memory module that mitigates the gradient vanishing and information forgetting issues and effectively captures dynamic features across various time scales.
Expanding on this memory module, we propose Long-Short Memory Updater (LSMU) to balance long-range and short-range temporal dependencies. LSMU utilizes a sparse mixture- of-experts module to integrate memory and encoded interactions for gate calculations, effectively adapting both influences and mitigating over-generalization issue.
To tackle the lack of structure information, we propose to incorporate co-occurrence encod- ing into LSMU via information bottleneck-based fusion, effectively capturing and adaptively balancing the impact of temporal and structure encoding.
We implement the prototype of SALoM and conduct extensive experiments to demonstrate its effectiveness. Our results show that SALoM outperforms the state-of-the-art dynamic graph learning methods on most benchmark datasets for link prediction. In particular, SALoM demonstrates significant improvements in prediction accuracy on the USLegis, UNtrade, and UNvote datasets where previous methods fell short, with enhancements of 14. 18%, 15.89% and 32.48% over its closest competitors, respectively.

---

2 Background and Related Works
2.1 Temporal Correlation Encoding in Dynamic Graph
Memory-based Methods. Memory-based methods use specialized memory modules to update evolving node representations through sequential interactions, theoretically preserving complete historical patterns. Jodie [17] pioneered this with RNNs and t-Batch, excelling in recommendation systems but lacking generalizability on other tasks. DyRep [28] extended memory-based methods to general dynamic graphs, emphasizing the interaction between graph structure and temporal dynamics. Innovations like TGAT's
32 attention mechanisms and Temporal Graph Networks (TGN) (25 combine memory modules with Multi-Head Attention, achieving early state-of-the- art results. However, memory updated iteratively based on RNN face practical limitations, such as gradient vanishing and information forgetting, which hinder long-range dependencies. And implementing discrete models on irregularly-sampe
events will cause intra-batch information loss.
Weighbor-Based Sequential Methods. Some recent studies 7 7,4 abandon iterative memory updates and directly process long historical neighbor feature sequences through sequential models, such as linear layers and transformer layers, to extract temporal correlations and structural features. While circumventing RNN-related gradient issues, information forgetting, and intra-batch loss, this approach imposes theoretical constraints on long-range dependency modeling through finite neighbor sampling and incurs memory overheads that scale quadratically with sequence length (e.g.. O(L2) for iitentioonbbase models), posing computational bottlenecks in practical applications.
2.2 Structural Encoding in Dynamic Graphs
Message Passing Methods. These methods leverage neighborhood aggregation without explicit structural encoding, implicitly capturing local topology through aggregating information from neigh- bors and propagating layer by layer (25 32). These methods often combined with memory- based
techniques, such as GCN [6] and MHA [29]. which exemplifies the application of this approach.
Random Walk- Methods. Inspired by static GNNs, early efforts to explicitly capture structure encoding in continuous-tin dynamic graphs explored methods based on random walks. For instance, CAWN extracts multiple causal anonymous walks for each node and employs RNNs to encode these walks and aggregates them to form the final node representation, with a particular emphasis on capturing causality in dynamic graphs
Specialized Structural Encodings, As research advanced, various explicit structure encoding methods were developed (7 [1 34], with co-occurrence neighbor encoding proving to be the most expressive and generalized. This approach, serving as a relative structure encoding, measures the frequency of common neighbors in the historical neighbor lists of two nodes. When integrated with temporal and node-specific features and processed by sequential models such as Transformers, demonstrates significant improvements on continuous-time dynamic graph tasks, showing robust expressiveness and generalizability across diverse datasets.
2.3 Limitations of Existing Methods
Although some achievements have been made, the existing methods still have limitations in capturing long sequence information and graph structure information.
Limitation#1:
Difficulty capturing complete long-range neighborhood temporal correlations.
Memory-based methods, although theoretically capable of storing information from all historical interactions in memory data, struggle to effectively handle long-term dependencies in practice. This is because RNN architectures used for capturing temporal correlations and updating memory are susceptible to issues like vanishing gradient [8] IT [4] and information forgetting [8] [ Neighbor- sequential methods, on the other hand, can capture long-rann rongerange temporal correlations of historical interactions by sampling a long neighbor list. However, this approach leads to redundant computational overhead due to duplicated interactions related to the same node in the long neighbor lists, and it faces limitations on the interaction history due to constraints on neighbor list length. For detailed motivation study of long-range temporal correlations capture, refer to Appendix B.]
Limitationi22: Challenge in balancing long-range and short-range temporal neighborhood features. In dynamic graph learning, it is crucial to capture both long-tern trends and short-term
3

---

Neighboy Sample
Message Encode Aggregate
LSMU (Section
Message
Temporal Memory
Multi-head Attention
Encoder
Temporal Memory
Dispatch
gate
IB Encoder (Section 3.3)
IB Decoder
(Section
Node Emheddings
C O
GRUI
GRU2
CFC1
CFC2
Update
Co-occurence Encoder
Hash Map
Updated
Structure Memory
Memory
Temporal Encoder
Fusion
Design LSMU
Figure 1: Model overview of SALoM
changes neighborhood feature data, because long-term memory helps identify evolving trends and persistent patterns, while short-term events reflect local dynamics and immediate changes [12] (6).. However, existing memory-based methods struggle to adaptively balance the influence of different historical interactions across different time periods. Although neighbor-based sequence methods can assign uniform weights for historical events with different time periods [23), their complexity grows quadratically with the neighbor list length. They also struggle adaptively determine the importance of short-range versus long-range correlations. Detailed motivation study of short-range temporal correlations capture and long-short term temporal correlation fusion refer to Appendix B.2 and B.3
Limitation#3: Hard to integrate long-term temporal and topologiaal structural features. For topological structural features, traditional message passing methods struggle to effectively handle long-term neighbors, often resulting in over-squashing and the inability to differentiate between homogeneous nodes. This is due the over compression of distant node information, leading to a loss of crucial structural features 3 24. As a result, these methods primarily focus on local neighborhoods. While random walk-based methods
capture long-range dependencies, they can be time-consuming or limited by cached historical neighbors, hindering the utilization of long- range information. Co-occurrence neighbor coding offers a solution but is restricted by the reliance on neighbor lists, limiting its ability to incorporate long-term information. Additionally, directly fusing temporal and topological structural features may result in conflicts and ultimately degrade performance, For details on structure encoding and feature fusion, see Appendix B.4 and B.5
Methodology
The SALoM framework is a cutting-edge approach designed to tackle key challenges within memory- based architectures, achieving three primary objectives that distinguish from existing methods. An overview of SALoM is provided in Figure SALoM excels at capturing long-range temporal dependencies through its Continuous- Time Memory Module, which uses ordinary differential equations to model continuous memory. The module effectively identifies long-term patterns in highly connected nodes, avoiding the discontinuity issues typical of batch-updated memory units. Additionally, SALoM features the Long-Short Memory Updater(LSMU), which combines long- range dependencies from the Continuous-Tiin Memory Module with short-range dependencies from RNNs. By adjusting its focus based on input characteristics, the model becomes more responsive to changes, improving prediction accuracy. SALoM also enhances structural information using HashMap Memory and Co-occurrence Encoder. It integrates temporal dependencies and structural features through the IB Encoder and Decoder, resolving potential coding conflicts and reducing the over-squeezing problem found in traditional methods. This allows for better differentiation between similar nodes, boosting the model's representational power and understanding of data relationships.
3.1 Continuous- Time Memory Module
Memory Update Based on Neural Ordinary Differential Equations. We propose a continuous- time memory module based on Neural Ordinary Differential Equations (ODEs) to capture long-range temporal dependencies in nodes with extensive neighborhoods. Unlike discrete models, Neural ODEs

---

describe the dynamics of hidden states using differential equations, allowing for continuous memory evolution that aligns with irregular temporal interactions in real-world graphs
The continuous time graph, which is modeled as a sequence of time-stamped events, denoted as = ( 11, f2), (42, 02, ..),..., (Mn, Uny f. ). representing the addition or change of node or interaction between pair of nodes at times 0 S t S l2 I ... < tn. An interaction event between nodes and at timestamp is associated with temporal edge feature e E 12 Each node at timestamp t is associated with raw feature x E R Temporal memory of node at time t before update, denoted as Zem (t), is updated with message msgf with dimension d-mag to M tem (i). We treat node memory updates as a continuous flow, expressed as:
d. (i) = J(M (1), (,)) dt
where 0 represents the training parameters. The function J(.) captures both the node's dynamics and the influence of neighboring nodes over time, mitigating the loss of temporal continuity from long-range batched updates. This approach effectively captures long-range dependencies complex graphs by modeling the derivative of the target function rather than direct input-output mapping, making it ideal for understanding temporal relationships.
Efficient ODE Solvers. However, the high computational overhead of ODE solvers presents significant barrier 4 13]. To address this issue, we explored closed-form solution variant of Neural
ODEs [3] for memory updates, which is quoted as CFC Cell in the rest of the paper. The memory update can be formulated as:
Mien (i) =0(== (i), msg' 0f) f) @ g(M temm (1), msgi 0g)
(2)
+[1 - (-f (t), msgi, 0y) +)] @ h(M (i), msg? 0x))
Here, a is the sigmoid activation function. The 0y, fg. and On trainable model parameters. Denote M)rm() (i) and msg; as the memory of node before timestamp and the aggregated message of node i at time t, respectively. The function (M (1), msg' 0r) serves as the liquid time constant for the sigmoidal time gates, while g g(M (i). 0g) and h h(Me (t), msg;. 0r) construct the potential memory of node at time f. These functions are instances of neural networks, such as multilayer perceptrons(MLPS) @ represents the Hadamard product. This formulation allows for efficient memory updates while maintaining the advantages of continuous time modeling.
3.2 Long-Short Memory Updater as Temporal Memory Updater
Bottleneck in only long-range dependencies. While continuous- time memory modules effectively capture long-range temporal evolution, they underperform certain datasets, as illustrated in Figure 3. For instance, replacing the memory updater with a continuous USLegis dataset led decreased performance. This decline arises because interactions in USLegis are more influenced by recent discrete events than by long-term patterns. Overemphasizing long- range dependencies
module on the
cause node representations to overly generalize, neglecting recent events. Thus, we incorporate methods for both long- and short-range dependencies to balance long-range correlations with short-term influences.
Adaptive Memory Backbone Selection. The optimal balance between long-range and short-range temporal dependencies exhibits dataset-specine and entity-specifit characteristics in continuous time dynamic graphs. To address this heterogeneity, we implement a Sparse oofture-off-Exerre (MoE) framework [5] with adaptive expert selection based on temporal interaction patterns. As shown in Figure I| our architecture integrates three complementary components CFC Cells for full-capacity long-term dependency modeling, GRU Cells for robust short-term pattern capture, and Sparse MoE Controller for context-aware backbone selection. SALoM dynamically routes input samples using both temporal messages and node memory states. For each interaction event (u, U, t), we generate time-aware messages through:
msg's Linear (ww.ncoooooccoaoooc (M), (v),, (v), TE(At),
(3)
where (F(-) denotes the temporal encoder 25 and et is the raw edge feature. The message msgg follows symmetric generation. Let (-) denote the q-th expert network. The memory update
5

---

combines expert outputs through gated aggregation:
Miem (u) (g) Ug (msgu, Miem (u))
Here, Q is the total number of experts. Assume that r experts are activated for learning. The routing weights wy are determined by:
Score = MLP (concat(msgl.. Miem (u))) A,
(5)
[Score (q)
if q E top-r(Score.,) Score (g) 0, otherwise,
Score (q)
(6)
0 w Softmax(Score.))
where A is trainable matrix that computes raw scores for each expert based on mixed messages.
The top-r selects the r experts with the highest scores. Then, we use a one-layer multi-head attention
mechanism to aggregate neighbor memory, forming the temporal encoding of node u. The neighbors of u are represented by (u, u', t') E G for t' < t. with x' as the feature of node at time and Φ(-) as the time encoding [32]. The temporal encoding can be formulated as follows:
= (u) Lineara (MHAAQ = (4t + Miem (u))llo(0) K = K V = =V)))
(8)
K2 Vi stack ((Mte (ol (w')ll 11o(t (`) (u, ,', 9), dim 0).
3.3 Combining Structural and Temporal Insights in Feature Fusion
To improve topological structural features while preserving long-ten temporal characteristics, we present a new learning architecture that combines long-term temporal memory with specific structural encoding, known as the Co-occurrence Neighbor Encoder.
Co-occurrence Neighbor Encoder as Structure Enhance Encoder. We discuss how to capture structure features with co-occurrence neighbor encoding. Let the k-hopp neighbors of node before timestamp f be denoted as 5% = {u' (u, v', (') E G, < 1). We maintain two structural memory arrays for each node u, M [u, 1 and M i,, which are used to store long-range and short-range historical neighbor hash tables, respectively. The lengths of these units correspond to the dimensions of the long-term memory unit da, and the short-term memory unit ds.s, where ds, = 4 da... The long-range co-occurrence neighbor count of node u with respect i can be formulated as:
da.1-1
(,,(,,,) = Σ I(Matruclle
M (trrecl..)))
(10)
which quantifies the number of common nodes between and in their hash memory. Similarly, the short-range co-occurrence neighbor count of node with respect to i can be estimated using similar approach. The structural encoding of node u derived from the structure memory as:
Ct (u) = (O,,(u u Oh,(e, (v, u'), Of.a (u, u'), Ov, (v,"") w' e (t))), Extrao (u) (F)(C
(11)
(12)
where FFN(.) is Feed-Forward Network. Appendix C.I details structure memory updating process.
Information Bottleneck Based Feature Fusion. We employ an information bottleneck-bases feature fusion approach integrate temporal and structural encodings, as simple concatenation fails to address modality conflicts. Fiiuure 3 highlighs the performance decline from direct concatenation. This fusion is achieved using the IB Encoder and IB Decoder architectures.
Denote the unified node embedding of node u at time i as 2 Temporal and structure encoding are denoted as =fem (u) and =struc(() respectively. The label of node u at timestamp / is denoted as vl.
IB Encoder obtains hybrid encoding of structural and temporal information through a linear layer, and then applies variational approximation with a standard normal distribution to derive the representation of the unified node embedding = The specific computation proceeds follows.
u Linear, (concal (zimm =ttruc()))
(13)
, -Linear((E))
(14)
a = log(1 + elui -5)).
(15)

---

The IB Decoder generates the final representation of the unified node embedding based on the approximate distribution, denoted as z (N (.).
z = randn(j -=
(16)
The optimization objective and loss function are designed to maximize the information corresponding to v within concat ( (u), = *struc (u)), also denoted as #coneat (u). We opt to minimize the mutual information between "concat (u) and zyg which can measure the correlation between two variables.
arg min -I vl)) + B (1 (éoncat(())
(17)
=/
In this expression, -I vi)) represents the mutual information between the intermediate repre- sentation z,, and the labels yu. while 1 (z =comoat (u)) represents the mutual information between the intermediate representation and the original representation. The former is aimed at maintaining representation relevant the labels, the latter is aimed at denoising the representation. This leads to intermediate features that are easily distinguishable, preserving the original information while enhancing predictive power.
As mutual information is difficult to calculate, by employing variational approximation, we scale the target function to derive the mathematical form of its upper bound. This upper bound can then be transformed into a form combining Binary Cross-Entropy (BCE) loss and KI. divergence. Due to space constraints, the detailed mathematical derivation for solving the upper bound of the optimization objective is provided in the appendix C.2
[I v')) + B- [= =coneat( (u)) S y- +=)ooo(()))++1 + -il----)-ool0 -og((1 p((())(())+++ 3 KLL 1)].
(18)
Based on the upper bond of the original optimization objective, the loss function for each node exemplified by node u is as follows, where ý is the model prediction.
loss = BCE vih) B KKL( 1)).
(19)
4 Experiments
4.1 Experiment Settings
Datasets and Baselines. We evaluate on thirteen datasets (Wikipedia, Reddit, MOOC, LastFM, Enron, Social Evo., UCI, Flights, Can. Parl., US Legis., UN Trade, UN Vote, and Contact) obtained from Edgebank 22.. Following DyGLib [34], we compare SALoM against ten popular dynamic graph learning methods, including JODIE 17]. DyRep 128]. TGAT (32,, TGN 25). CAWN [I],
EdgeBank 21). TCL [30], GraphMixer 19. DyGFormer [34, and CNE-N 7 The details of the
above datasets and baselines are elaborated in Appendix A.. and Appendix A.2 respectively.
Evaluation Metrics. We evaluate the performance on the dynamic link prediction task, which involves predicting the presence of link given time. This task includes two settings: the transductive setting predicts future links among nodes observed during training, while the inductive setting assesses link prediction for unseen nodes. We measure performance using Average Precision (AP) and Area Under the Receiver Operating Characteristic Curve (AUC-ROC) as evaluation metrics.
Model Configuration. Our SALoM is built upon the classic memory- based model TGN. We set structure memory dimension as dz,1 = 64, dg.1 = 16 and temporal memory dimension as d; = 172. We choose 2 experts from a total of GRU units and 3 CFC units. For fusion, we set le-
Implementation Details. To ensure a fair comparison, we evaluate the baseline model TGN, the state-of-the-art model CNE-N, and our SALoM on the same machine with identical settings. We conduct the evaluation on an Ubuntu machine featuring an Intel(R) Xeon(R) Platinum 8352V CPU @ 2. 2010HHz and an NVIDIA GeForee RTX 4090 GPU with 24 GB memory. The models are trained for 100 epochs with early stopping using a patience of 20. The model exhibiting the best performance on the validation set is chosen for testing. A uniform learning rate of 0.0001 applied to all methods across all datasets. Batch sizes are set to 10 for memory-based methods and 200 for neighbor-based sequence methods. Each dataset divided into rraiiingvalidaaiionnessii sets a 70%/15%//55% ratio. We conduct five runs of each method with seeds ranging from 0 to 4 and report the average performance. For the remaining baselines, we refer to the reported best performance in DyGFormer to maintain consistency, following similar approach as in CNE-N 77..
7

---

Table I: AP&AUC-ROC (%) for transductive and inductive link prediction.
Metrics
Dalasels
JODIE
LyRep
TOAT
TGN
CAWN
EdgeBank
TCL
GraphMixer
NAT
DyGFormer
CNE-S
SALOM
Wikipechi
94.55
99.03
99.03
Reddit
98.1
98.22
98.52
1/5.47 477
19:11
94.86
97.53
97. 31
99.10 10
99:22
99.26
99.27
MO0C
80.23
81.97
85.84
93,21
80.15
57.97
82
82.78
87.21
87.52
581.16
92.42
LaxtMM
71.92
93.00
9/2
93.14
Fnmoa
84.77
82.38
71.12
91.51
89.56
83.53
79.70
82
90.81
92,13
94.08
Social Evo.
89.8
58.87
93.16
89.83 83
84.96
74.95
93.13
93.37
91.23
94.73
94,50
94.73
Trans-AP
UCI
89.43
65.14
79.63
92.94 04.944
95.
20
89.57
93. 25
94.26 26
M5. 79
95. 64.64
96.36
Flights
95.60
95.29
90.99
Can. Parl.
69.
66.54
70.73
96.29 29
69.82
64.55
68 67
77.04
83.83
97.36
81 84
99.11
US Legis.
75.05
75.34
68.52
78.09
70.58
58.39
69.59
70.74
77.56
71.11
72.58
92.27
UN Trade
61,94
63.21
61.47
68.3
65.59
60.41
62.21
62.61
72.32
66.16
77.97
93.86
63.91
62.81
52.84
58.10
Contact
9
95,98
96.28
95,00
90.26
92.58
92.44
91.92
97.25
98.23
98.28
98.53
Avg. Rank
7,76
8.51
8.38
4.92
7.07
T0:53
9.76
K.NN
4.53
3.15
3.61
1,07
Wikipedia
96.33
V6.
US
96.92
98.91
V8.87
Reddit
98.31
98.17
98.47
98.32
99.01
95.37
97.42
97.17
99.02
99.15
99.19
99.2
MOOC
83.81
85.03
87.11
93.56
80.38
60.86
83.12
84.01
88. 38
87.91
91.42
92.52
LasFM
70.49
71,16
71.59
62.66 66
85.92
83.77
64.06
73.53
86.94
93.05
92.21
91.32
87.96
$4.89
90.45
87.05
84.38
92.77
95.11
Social Evo.
92.05
90.76
94.76
90.36 36
87 87.34
81.60
94.84
95.23
93.22
DOL3
96.20
96.36
ArC
Flight
UCI
96.21
90.44
68.77
95.95
94.13
78,55
92.17
.9
98.45
93.87
90.23
77.30
87.82
91.13
91.81
97..22
93.02
$1.49
94.32
98.74
98.99
95.53
Can. Parl.
78. 21
73.35
75.60
97.17
75.7
64.14
72.46
83.17
87.
V7. 76
84.49
99.18
US Legis.
82.85
82.
75.84
84 66
77.16
62.57
76.27
76.96
N4 64.6H
77.50
7M 58
93.75
UN Trade
62
67.44
61.01
69:41
68.54
66.75
64.72
65.52
76.76
70.20
79.64
93.23
67.18
$2.46
57.12
60.67
87.87
Contact
96.66
96.48
96.95
95.37
89.99
94.34
94.15
93.94
97.64
98.53
98.62
98.69
Avg. Rank
7.07
8.38
8.69
5:53
7.15
10:15
10:07
8.61
1.30
3.25
3.53
1.23
Wikipedia
94.82
92.43
A4
V8.
96.22
96.65
9$
97.76
99.49
Reddit
96.5
96.09
97.09
26
62
94.09
95
56
98.93
MOOC
79.63
81.07
85.50
91.86
81.42
80,60
81.41
83.59 59
86.96
88.71
90.53
LasHMM
81.61
83.02
78.63
87.18
89
73.53
82.11
86.87
94
944000
94.56
74.55
84.53
75.88
39.76
87.59
91.67
Social Evo.
91.96
90.04
91.41
82,85
79.94
51.55
91. 91.86
91.22
93.14
92.70
92.84 24
Ind.AP
Flights
UCI
79.86
91.74
92.88
57.48
85.73
79.54
95.03
$2.04
97.06
92.73
87.36 36
83.11
83.03
91.19
96.59
87.30
94.54
97.79
97.34
93.58
97.85
94.36
Can. Parl.
54.02
78.75
55.91
65.01
96.20
US Legis.
$4.93
57.28
51.00
55.74
53.17
$2.59
50.71
57.54
54.28
59.54
68.38
UN Trade
59.65
$7.02
61.03
77.86
65.
62.21
67.17
69 29
64.55
09:84
85.46
UN Vote
56.64
54.62
52.24
65.67
49.94
51.6
50.68
66.35
55.93
57.57
62.37
Contact
94.34
92.18
95.87
KX.56
89.55
91.11
90:59
98.03
97.58
97.79
Avg. Rank
7.92
3.69
8.15
3.38
6.33
8.33
X:15
3:07
2.84
3:23
1.53
Wilipedis
94.33
91,49
95.9
97.08
98.003
95.37
96.30
54,74
98.48
97.45
96.26
Reddit
96.05
94.97
98.85
MOOC
83.16
84.03
86.84
92.02
81.86
81.43
82.77
86.13
87.62
89
90.09
LasFM
81.13
62.24
76.99
85.58
87.82
70.84
$0.37
83.07
94.08
93.62
93.77
Enme
81.96
76.34
64.63
83.58
87.02
72.33
76.51
90.69
88.24
V1.57
Social Evo.
93.30
91.18
84.73
InC-AUC
Flight 95.21
UCI
78.80
93.56
58.08
88.64
77.64
95.92
86.48
91140
96.86
84.49
82.48
89. 30
$2.27
96.36
83.81
97.80
92.63
97.20
91.31
97.93
97
Can.
53.81
39:33
US Legis.
58.12
61.07
48.27
58.87
$1.49
50.43
47:20
62.85
53.21
60.10
65.56
UN Trade
62.28
58.82
62.72
75.70
67.05
63.76
63.48
72.56
67.25
71,40
83.04
UN Vote
58.13
55,13
51.83
61.64
48.34
50.51
50.04
66.26
56.73
58.85
62.44
Contact
95.37
91.89
89.07
92.83
98.30
97.91
97.98
Avg. Rank
7.76
3.53
8.07
3.45
633
8.76
8:15
3.00
2.09
3.45
1.33
4.2
Performance Study on Model Accuracy
In this section, we analyze the model accuracy in terms of the AP & AUC-ROC metrics for the ten baseline models and our SALoM in transductive and inductive dynamic link prediction tasks. The results are presented in Table I highlighting the best and second-best performances using bold and underlined fonts. Note that EdgeBank is evaluated only in the transductive setting and its results for the inductive setting are not included. Our SALoM consistently achieves top performance across most datasets, with average rankings ranging from 1.07 1.46 for different metrics. Specifically, notable improvements are observed in the USLegis, UNtrade, and UNvote datasets, with enhancements of 14. 18%, 15.89%, and 32.48 over its closest competitors, respectively. This superior performance can be attributed to SALoM's ability to effectively capture both long-range and short-range tempo- ral dependencies using LSMU, enabling adaptive retention or forgetting of temporal correlations. Additionally, the information bottleneck-based fusion in SALoM allows for structural and temporal awareness without conflict, enabling nuanced node distinctions from multiple perspectives. We provide the version with standard deviates in Appendix D.I
4.3 Ablation Study
Effectiveness of LSMU. this section, we evaluate SALoM using various temporal memory updaters, including the traditional GRU from TGN, CFC ($..1 MoE-GRU (GRUs in MoE architecture),
8

---

Results on tested AP are shown in Figure LSMU consistently outperforms other updaters, with an improvement of 3-5% AP compared to GRU, This superiority is due to GRU's limitations in capturing long-range temporal features due vanishing gradients and forgetting information issues. CFC captures long- long-range features struggles with short-range ones and over- generalization issue. Simply using the MoE. method does not solve these issues. LSMU excels by dynamically determining retention or forgetting of long-range and short-range temporal correlations using sparse MoE module. This adaptive approach effectively balances influences, mitigating over-generalization for superior performance. We further experimented the training expense and universality of LSMU in Appendix D.2 and D.3
and LSMU (:3.2
Effectiveness of Feature Fusion. In this section, we evaluate SALoM using various fusion methods for temporal and structural embedding. including w/o SE (without structure encoding), Concat, Linear, and our proposed IB method ($3.3), and show the results in Figure 3 Simple Concat and Linear fusion methods offer advantages for UCI, Can.Parl, and UNtrade datasets, but struggle with the USLegis dataset due to potential conflicts between temporal and structural embeddings. This highlights the need for effective fusion methods. On the other hand, our IB fusion method effectively balances the influence temporal and structure encoding, consistently delivering superior performance.
4.4 Performance under different numbers of historical neighbors.
In this section, analyze SALoM's performance with varying numbers of sampled historical neighbors aggregated per iteration, as shown in Figuure 4 Since SALoM builds memory based methods that update node memory iteratively, it does not require processing a large number of historical neighbors each iteration. Having 10 historical neighbors per iteration is sufficient for SALoM to perform well. This because our proposed LSMU effectively retains valuable long-range features in memory data and dynamically balances temporal dependencies across different time. Performance declines when the number of neighbors aggregated each iteration exceeds 100, likely due to over-smoothing. Excessive neighbor sampling increases similarity between node embeddings during aggregation, reducing their distinctiveness. Moreover, since the iterative memory update mechanism already preserves long-term information, excessive sampling of historical neighbors disproportionately weights long-term dependencies and may dilute crucial short-term patterns.
4.5 Trade-off Between Accuracy and Efficiency
In this section, we compare the time per epoch and AP of SALoM with baseline methods, as shown in Figure 5) SALoM is evaluated with different batch sizes, while baseline methods maintain their default settings from their respective papers. Larger batch sizes in SALoM allow for more parallel event computations, enhancing computational efficiency. However, this efficiency gain comes at the cost of intra-batch information loss, impacting training accuracy. This trade-off consistent with other memory-basee methods. Our results suggest that SALoM can outperform existing methods with slight edge in performance at comparable computational costs. When computational constraints are relaxed, setting a small batch size significantly boosts SALoM's performance, largely surpassing existing approaches in dynamic graph learning. We further study the impact of varying batch sizes on accuracy in Appendix D44
Performance UCI
[b] Performance Cen.Peri
(c) Performance USLegis
(d) Performance UNtrade
96.3
92
2 95.0
92.8
95.7
92.0
95.4
98.1
80
91.2
97.8
76
GRU
CFC
MoE-GRU GRU
LSMU
GRU
CFC
MaE GRU
LSMU
GRU
CFC
MoF MRE-GRU
LSMU
GRU
MoE-GRU
LSMU
Different Memory Updaters
Different Memory Updaters
Different Memory Updaters
Different Memory Updaters
Figure 2: Ablation study on different memory updaters.
(e)
Performance on UCI
(b) Performence on Can Parl.
(c) Performance on USLegis
(d) Performance en UNtrede
96.4
96.0
98.8
86
90
81
95.2
98.0
84
78
94.8
97.6
72
w/ost SE
Concat
Uinar
(B(ours)
w/a SE
Concat
Linear
(leurs)
w/l SE
Concat
Linear
(ocurs)
w/b SE
Concat
Unear
Blcurs)
Different Fusion Methods
Different Fusion Methods
Different Fusion Methods
Different Fusion Mathods
Figure 3: Ablation study on different fusion methods for structural and temporal embedding.

---

[a] Performance on UCI
(b) Performance on Can. Can.Pari.
(c) Performance on USI USin
(d) Performance on UNtrrade
2
96.2
96.3
99.1
99.0
90
93.0
93.6
96.1
84
92.4
96.0
98.8
91.8
96.7
91.2
10
20
100
200
1000
10
10
100
700
1000
10
20
100
200
1000
10
20
100
200
1000
Number Sampled Neighbors
Number Sampled Neighbors
Number Sempled Neighbors
Sampled Neighbors
Figure 4: Ablation study on different numbers of historical neighbors aggregated each iteration.
UCI
Can.Parl.
USLegis.
UNtrade
100
90
90
90
AP(%)
AP(%)
AP(%)
AP(%)
80
80
80
70
70
70
0
200
0
200
400
0
200
0
1000
2000
Time/epoch(s)
Time/epoch(s)
Time/epoch(s)
Time/epoch(s)
JODIE
TGN
GraphMixer
SALoM (batch_size-10)
SALoM (batch_sin Bticchsize-1100
DeRep
CAWN
DyGFormer
SALoM (batch_size=20)
SALoM (batch_size=200)
TGAT
TCL
CNE-N
Figure The trade-off between efficiency and performance under different settings of batch sizes.
Table 2: Leakage-free performance evaluation.
Metries
Methods
MOOC
UCI
Enron
Can. Parl.
US Legis.
UN Trade
UN Vote
Avg. Rank
CAWN
80.15
95.18
89.56
69.82
70,58
65.39
52.84
3.57
TCL
82.38
89.57
79.7
68.67
69.59
62.21
$1.9
4.86
Trans-AP
GraphMixer
82.78
93.25
82.25
77.04
70.74
62.61
52.11
3.57
DyGFormer
87.52
95.79
92.47
97.36
71.11
66.46
55.55
2
SALoM
91.39
96.39
93.19
98.79
75.79
92.52
68.52
1
4.6 Leakage-Free Evaluation
This section clarifies the information leakage issue in TGNN evaluation and presents our solution. The problem arises because: during batch training, edges sharing identical timestamps might be split across consecutive batches. When processing the second batch, the model's 's memory has already been updated by edges from the first batch that have the same timestamp, thus gaining access to information that should be chronologically unavailable, constituting an information leakage problem.
To ensure rigorous leakage-free evaluation, we implement a time-aware dual memory management system. This approach maintains two separate memory states: one storing the final node state from the previous timestamp, and another for accumulating updates within the current timestamp. During inference for a given edge, the model strictly uses the memory state from timestamps prior to the current edge's time form node representations. Only when advancing the next distinct timestamp are the accumulated updates synchronized, preventing any leakage within the same timestamp.
As shown in Table 2, SALoM maintains state-of- of-te-of-the-art performance under these strict leakage-fiee frree conditions. While the impact of leakage is minimal on most datasets, performance variations are Crucially, SALoM still outperforms its best competitors by significant margins (4.68%and 12.97%in average precision, respectively), confirming its robust SOTA status through fair and rigorous comparison. Further detailed examples and more experimental results are provided in Appendix C.3
observed in USLegis (90%~755% and UNvote (80%~>68
5 Conclusion and Future Work
This paper introduces a continuous-time dynamic graph learning framework that emphasizes capturing temporal correlations and structural relations in graphs. We propose the Long-Shon Short Memory Updater (LSMU) to extract both long-range and short-range temporal dependencies and balance their influence to mitigate over ver-globalization. By integrating o-occurrence encoding into LSMU through information bottleneck-bar based fusion, we unify temporal and structural information, improving model performance and achieving state-of-the-art results on benehmark datasets. Our experiments adhere to the DyGLib framework, enabling reproducibility and comparison with other methods. In the future, potential areas for improvement our framework include exploring automatic feature extraction methods, enhancing neighbor aggregation efficiency, and improving strategies for mitigating intra-batch information loss.
10

---

Acknowledgments
This research is supported by the "Pioneer" R&D Program of Zhejiang (No.2024C01019) the Zhejiang Province "Jianbing" Key R&D Project of China (00.002500,),)), the Hangzhou Joint the
Fund of the Zhejiang Provincial Natural Science Foundation of China NN..HHZS224F02000 Zhejiang Province High-Level Talents Special Support Program "Leading Talent of Technological Innovation of -en-Thussands Talents Program" (N0.2022550046, and the Fundamental Research Funds for the Central Universities (No. (No.2021FZZ) 202 and 226-200000 226-2025-000 20252000000)).. The author gratefully acknowledges the support of Zhejiang University Education Foundation Qizhen Scholar Foundation.
References
[1] ALON, U., AND YAIIAV, E. On the bottleneck of graph neural networks and its practical implications. arXiv preprint arXiv:2006. 05205 (2020).
[2] CHANG, Y.-T., HU, Z., LI, X., YANG, S., JIANG, J., AND SUN, N. Dihan: novel dynamic hierarchical graph attention network for fake news detection. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (2024), pp. 197-206.
[3] CHEN, D., O'BRAY, L., AND BORGWARDT, K. M. Structure-aware transformer for graph representation learning. In International Conference on Machine Learning. ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA (2022), K. Chaudhuri, S. Jegelka, L. Song. C. Szepesvári, G. Niu, and S. Sabato, Eds., vol. 162 of Proceedings of Machine Learning Research, PMLR, pp. 3469-3489.
[4] CHEN, R. T., RUBANOVA, Y., BETTENCOURT, J., AND DUVENAUD, D. K. Neural ordinary differential equations. Advances in neural information processing systems 3/ (2018).
[5] CHEN, Z., DENG, Y., WU, Y., GU, Q., AND LI, Y. Towards understanding the mixture- of-experts layer in deep learning. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Or- leans, LA, USA, November 28- December 2022 (2022), S. Koyejo. S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds.
[6] CHEN, Z., ZHENG, T., AND SONG, M. Curriculum negative mining for temporal networks. Neural Networks (2025), 107858.
[7] CHENG, K., LINZHI, P., YF, J., SUN, L.,, AND DU, B. Co-neighbor encoding schema: A light-cost structure encoding method for dynamic link prediction. In Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining (2024), pp. 421-432.
[8] CHO. K., VAN MERRIENBOER, B., GÜLÇEHRE, Ç., BAHDANAU, D., BOUGARES, F., SCHWENK, H., AND BENGIO, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL. (2014), A. Moschitti, B. Pang, and W. Daelemans, Eds., ACL, pp. 1724-1734.
[9] CONG, W., ZHANG, S., KANG, J., YUAN, B., WU, H., ZHOU, X., TONG, H., AND МAHDAVI, M. Do we really need complicated model architectures for temporal networks? In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 (2023), OpenReview.net.
[10] DING, D., YI, J., XIE, J., AND CHEN, Z. Meta-path aware dynamic graph learning for friend recommendation with user mobility. Inf. Sci. 666 (2024), 120448.
[11] FENG, L., TUNG, F., AHMED, M. O., BENGIO, Y., AND HAJIMIRSADEGHI H. Were mns all we needed? arXiv preprint arXiv:2410.01201 (2024).
[12] GU, A., DAO, T., ERMON, S., RUDRA, A., AND Ré, C. Hippo: Recurrent memory with optimal polynomial projections. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual (2020), H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds.
11

---

[13] HASANI, R., LECHNER, M., AMINI, A., LIEBENWEIN, L., RAY, A., TSCHAIKOWSKI, M.,
TESCHL, G., AND Rus, D. Closed-form continuous- time neural networks. Nature Machine Intelligence 4, 11 (2022), 992-1003.
[14] HOCHREITER, S., AND SCHMIDHUBER, Long short-term memory, Neural computation 9, (1997), 1735-1780.
[15] JING, X., ZHOU, Y., AND SHI, M. Dynamic graph neural network learning for temporal omies data prediction. JEEE Access 10 (2022), 116241-116222.
[16] KIPF, T. N., AND WELLING, M. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings (2017), OpenReview.net.
[17] KUMAR, S., ZHANG, X., AND LESKOVEC, J. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019 (2019), A. Teredesai, V. Kumar, Y. Li, R. Rosales, E. Terzi, and G. Karypis, Eds., ACM, pp. 1269-1278.
[18] LI, Q., HAN, Z., AND WU, X.-M. Deeper insights into graph convolutional networks for semi-supervised learning. Proceedings of the AAAI conference on artificial intelligence (2018), vol. 32.
[19] LI, X., XIE, L., DENG, B., LU, H., ZHU, Y.. YIN, M., YIN, G., AND GAO, W. Deep dynamic high-order graph convolutional network for wear fault diagnosis of hydrodynamic mechanical seal. Reliah. Eng. Syst. Saf. 247 (2024), 110117.
[20] LUO, Y., AND LI. P. Neighborhood-awaan scalable temporal network representation learning. In Learning on Graphs Conference (2022), PMLR, pp. 1-1.
[21] POURSAFAEI, F., HUANG, S., PELRINE, K., AND RABBANY, R. Towards better evaluation for dynamic link prediction. Advances in Neural Information Processing Systems 35 (2022), 32928-32941.
[22] QIAN, H., ZHOU, H.. ZHAO, Q., CHEN, H., YAO, H., WANG, J., LIU, Z., YU, F., ZHANG, Z., AND ZHOU, J. MDGNN: multi-relational dynamic graph neural network for comprehensive and dynamic stock investment prediction. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, T'hitty-Sitth Conference on Innovative Applications of Artificial Intelli- gence, IAAI 2024, Fourteenth Symposium Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada (2024), M. J. Wooldridge, J. G. Dy, and S. Natarajan, Eds., AAAI Press, pp. 14642-14650.
[23] QU, L., ZHU, H., DUAN, Q., AND SHI, Y. Continuous time link prediction via temporal dependent graph neural network. In Proceedings of the web conference 2020 (2020), pp. 3026- 3032.
[24] RAMPÁSEK, L., GALKIN, M., DWIVEDI, V. P., LUU, A. T., WOLF, G., AND BEAINI, D. Recipe for a general, powerful, scalable graph transformer. In Advances Neural Information Processing Systems 35: Annual Conference
Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 9, 2022 (2022), S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds.
[25] ROSSI, E., CIAMMBERLII,, B., FRASCA, F., EYNARD, D., MONTI, F., AND BRONSTEIN, M. Temporal graph networks for deep learning dynamic graphs. In ICML 2020 Workshop Graph Representation Learning (2020).
[26] SONG, W., XIAO, Z., WANG, Y., CHARLIN, L., ZHANG, M., AND TANG, J. Session-basee social recommendation via dynamic graph attention networks. Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM 2019, Melbourne, VIC, Australia, February l 11-15, 2019 (2019), J. Culpepper, Moffat, N. Bennett, and
K. Lerman, Eds., ACM, pp. 555-563.
12

---

[27] SOUZA, A., MESQUITA. D., KASKI, S., AND GARG, V. Provably expressive temporal graph networks, Advances neural information processing systems 35 (2022), 32257-32269.
[28] TRIVEDI, R., FARAJTABAR, M., BISWAL, P., AND ZIIA, H. Dyrep: Learning representations over dynamic graphs. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 (2019), OpenReview.net.
[29] VASWANI, A., SHAZEER, N., PARMAR, N., USZKOREIT, J., JONES, L., GOMEZ, A. N., KAISER, L., AND POLOSUKHIN, I. Attention is all you need. In Advances in Neural Infor- mation Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA (2017), I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. Vishwanathan, and R. Garnett, Eds., pp. 5998-6008.
[30] WANG, L., CHANG, X., LI, S., CHU, Y., LI, H., ZHANG, W., HE, X., SONG, L., ZHOU, J., AND YANG, H. Tel: Transformer-baas based dynamic graph modelling via contrastive learning. arXiv preprint arXiv22105.07944 (2021).
[31] WANG, Y., CHIANG, Y., LIU, Y., LESKOVEC, J., AND LI, P. Inductive representation learning in temporal networks via causal anonymous walks. 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 (2021), OpenReview.net
[32] XU, D., RUAN, C., KORPEOGLU, E., KUMAR, S., AND ACHAK, K. Inductive representation learning on temporal graphs. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 (2020), OpenReview.net.
[33] Yu, B., YIN. H., AND ZHU, Z. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In Proceedings of the Shenty--eventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden (2018), J. Lang, Ed., jjeai.org, pp. 3634 3640.
[34] YU, I.,, SUN, I.., DU, B., AND Lv, W. Towards better dynamic graph learning: New architecture and unified library. In Advances in Neural Information Processing Systems 36: Annual Conference Orleans, LA, USA, December 10 16, 2023 (2023), A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, Eds.
Neural Information Processing Systems 2023, NeurIPS 2023, New
[35] ZHANG, T., ZHENG, T., XIAO, Z.., CHEN, Z.,, LI, L.,, FENG, Z.., ZHANG, D., AND SONG, M. Language mnnhnee-enhanced semantic topology representation learning for temporal knowledge graph extrapolation. In Proceedings of the 33rd ACM International Conference Informa- tion and Knowledge Management (New York, NY, USA, 2024), CIKM '24, Association for Computing Machinery, p. 3227-3236.
[36] Zou, T., MAO, Y., YE, J., AND DU, B. Repeat-aware neighbor sampling for dynamic graph learning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (2024), pp. 4722 4733.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have faithfully describe the contributions and scope of this work in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the potential limitation in conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
Due to space limitation, we provide the theoretical proof in Section C.2
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provided detailed experimental information to ensure the high repro- ducibility of this work in Section 4.1 and
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification: The experimental datasets are publicly available and are referenced with an accessible link. Our code is provided through an anonymous link.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
wtt.:g/dddeeeeeSSuuurutr ssionpolicy) for more details. httpollliigggddde/oooooee
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section describe the training and test details necessary for under- standing the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: The standard deviations of each method are provided in Section D.I
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The utilized computer resources of this work are described in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The work adheres fully to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed broader impacts of this work in Section D
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: This work does not involve risks for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have carefully referenced the original owners the assets used this work, all of which are publicly available for academic use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cow/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We have provided with detailed instructions of our code via an anonymous
link.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
This work does not involve Crowdsourcing or research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This work does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
19

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: This work utilizes LLM soley for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy httppa///eaurpppcccc//ooeerrrres for what should or should not be described.
20

---

A Datasets and Baselines
A.1 Datasets Details
Wikipedia:
This dataset captures edits Wikipedia pages one-month period. Nodes represent editors and wiki pages, while edges denote timestamped edit requests. Each edge is associated with LIWC feature vector derived from the edit text, encoding linguistic and psychological attributes.
Reddit: Spanning one month, this dataset models interactions within Reddit subreddits. Nodes represent or posts, and edges indicate timestamped posting requests. Similar to the Wikipedia dataset, edges are annotated with LIWC feature vectors based on the text of the posts.
MOOC:
This dataset represents a student interaction network within an online course. Nodes are students or content units (e.g.. problem sets, videos), and edges reflect students accessing content units. Each edge has four features, capturing interaction-specife attributes.
LastFM:
This interaction network tracks 1,000 users listening to the 1,000 most popular songs over one month. Nodes represent users and songs, and edges indicate user-listens-to- song relationships. The dataset contains no additional edge or node attributes.
Enron: This dataset comprises approximately 50,000 emails exchanged among Enron employees over three years. Nodes represent employees, and edges denote email correspon- dences. No attributes are included in this dataset.
This mobile phone proximity network tracks interactions in an undergraduate dormitory from October 2008 May 2009. Nodes represent individuals, and edges indicate physical proximity, with each edge having two features describing the interaction.
Social Evo.:
UCI: This dataset models Facebook like communication network among University of California, Irvine students. Edges represent timestamped interactions with second-level temporal granularity. The dataset includes no additional attributes.
Flights: A directed dynamic network illustrating air traffic evolution during the COVID-19 pandemic. Nodes represent airports, and edges denote tracked flights, with edge weights indicating the number of flights between two airports per day. The dataset was specifically extracted and cleaned for this study.
Can. Parl.: This dynamic political network documents interactions among Canadian Members of Parliament (MPs) from 2006 to 2019. Nodes represent MPs, and edges are formed when two MPs vote 'yes' on the same bill. Edge weights reflect the number of shared 'yes' votes in a year.
This dataset captures co-sponsorship interactions among US Senate legislators. Nodes represent senators, and edge weights indicate the number of times two senators co-sponsored bill given congressional session.
USLegis..
UN Trade:
weighted, directed network of food and agriculture trade among 181 nations over 30 years. Nodes represent countries, and edge weights denote the normalized total value of agricultural imports or exports between pairs of countries.
UN Vote: This dataset records roll-call votes in the United Nations General Assembly from 1946 to 2020. Nodes represent nations, and edges are formed when two nations vote "yes' on the same item, with edge weights incremented by one per shared vote.
This dataset tracks physical proximity among approximately 700 university students over four weeks. Nodes represent students with unique IDs, and edges indicate close physical proximity, with edge weights reflecting the strength of proximity.
Contact:
A.2 Baseline Details
JODIE 17 targets bipartite networks with instantaneous user-item interactions. utilizes two coupled recurrent neural networks (RNNs) to recursively update user and item represen- tations. projection operation is employed to predict the future representation trajectory of each user item, enabling the model to capture evolving interaction patterns.
21

---

features a custom RNN to update node representations upon the observation of new edges. It incorporates temporal attention mechanism, parameterized by the recurrent architecture, to assign weights to neighbors at each timestamp, effectively modeling temporal dependencies node interactions.
DyRep 28
TGAT 32] aggregates features from node's toppooallopoooogcaal neighbors using self- attention mechanism to compute node representations. It includes a time-encoding function capture temporal patterns, enhancing its ability to model dynamic network structures.
25) maintains evolving memory for each node, updating it through message function, aggregator, and memory updater when a node participates in an interaction. An embedding module generates temporal node representations, balancing efficiency and expressiveness in dynamic settings.
CAWN[31] extracts multiple causal anonymous walks for each node to explore network dynamics and generate relative node identities. It employs RNNs encode these walks and aggregates them to form the final node representation, emphasizing causality in dynamic graphs.
EdgeBank 21 is memory-based approach for transductive dynamic link prediction. It stores observed interactions memory unit and updates it using various strategies. An interaction is predicted positive if retained in memory and negative otherwise, offering lightweight solution.
[CL generates node interaction sequences via breadth-firs search on temporal depen- dency interaction sub-graphs. It employs graph transformer that integrates graph topology and temporal information, using cross-attention model interdependencies between inter- acting nodes.
GraphMixer[9 leverages a fixed-time encoding function, which outperforms trainable versions. It integrates this function into an MLP-Mixer- link encoder learn from temporal links, while a node encoder with neighbor mean-pooling summarizes node features.
NAT uses a dictionary-type neighborhood representation to aggregate temporal neigh- bors. employs recurrent process with random Fourier feature (RFF)-based time embed- ding to learn node representations, constructing query-induced subgraphs without neighbor sampling to reduce computational costs.
DyGFormer 34 is Transformer-based model that focuses first-hop interactions be- tween nodes. It introduces neighbor co-occurrence encoding scheme via a patching method, feeding these into Transformer to capture long-term correlations between source and destination node sequences.
CNE-N[7] stores historical neighbor in hash-like tables, efficiently calculates co-occurrence encoding and combines other features to form node embeddings.
B Individual motivations
B.1
Motivation of Continuous Models for Long-Term Temporal Correlation Capture
Traditional sequential methods (RNNs/GRUs) treat dynamic graphs as discrete events, struggling with temporal continuity and long-term dependencies due to gradient issues. In contrast, ODE-based methods preserve continuity and avoid gradient issues through second-order derivative integration, demonstrating superior long-term dependency capture with minimal information loss. However, our ablation studies in $4.3 show that over-emphasizing long-term dependencies can cause over- globalization. We therefore propose L.SMU, an MoE-based approach that dynamically balances long/short-term dependencies, adaptively selecting the optimal processing.
B.2 Motivation of GRU for Short-Tern Temporal Correlation Capture
In dynamic graphs, recent neighbor interactions typically provide the most valuable information. While simple RNNs and K-neighbor aggregation fail to effectively capture these patterns (due to vanishing gradients and limited neighbor hops respectively), GRU's gating mechanism enables robust short-term dependency learning while complementing our ODE-based long-term capture.
22

---

Table 3: Motivation for short-term temporal correlation capture.
CanParl
USLegis
UCI
Untrade
RNN
98.81
89.90
96.00
92.82
AggerateNeighbor
98.77
77.31
96.22
92.83
GRU
99.11
92.27
96.36
93.86
Table 4: Motivation for sparse MoE as long-short term temporal correlation fuser.
CanParl
USLegis
UCI
Untrade
Concat 96.90
76.93
96.02
92.00
Avg- Voting
98.65
76.06
96.23
92.99
MoF
99.11
92.27
96.36
93.86
Our extended experiments confirm that simple RNN suffers from gradient vanishing. leading to performance degradation (mitigated by GRU's gating mechanism). Besides, K-neighbor aggregation limited 1-2 hops, as wider aggregation blurs node representations.
B.3 Motivation of Sparse MoE for Long-Short Term Temporal Correlation Fusion
We propose MoE to fuse long-short term temporal correlations while adaptively selecting optimal backbones and weights per input. In temporal graphs, events exhibit mixed dependencies, but naive fusion methods (e.g., concatenation, voting) treat edges uniformly, ignoring their distinct evolutionary patterns. MoE addresses this by dynamically routing events specialized experts based on node/edge features, achieving superior performance (see Table). For instance, on USLegis, MoE improves AP by 15.3% over concatenation and 16.2% over voting, demonstrating its ability to capture varied temporal scales.
This advantage stems from MoE's capability of adaptively fusing long-short term correlations. And learnable weights for feature fusion.
B.4 Motivation for Co-Occurrence Encoding as Structure Encoding
We evaluated model performance with different structure encoding methods, comparing the expres- sivity co-occurrence encoding and traditional methods, such as message passing and vandom-walk based methods.
As shown in Table and Figure co-occurrence encoding shows better expressivity compared with traditional methods. Improving model accuracy, especially 18.44%%-993.866 on UNtrade.
Our co-occurrence encoder efficiently captures structural patterns while maintaining computational feasibility. Unlike traditional methods that trade off expressiveness for efficiency, it encodes neighbor co-occurrence frequencies relative structural features, enhanced by hash-based memory system for accelerated processing. Experiments demonstrate its superiority over basic message passing and random walk approaches, with significantly better performance at manageable computational cost.
B.5 Motivation of Information Bottleneck for Temporal and Structure Feature Fusion
Temporal graphs require effective fusion of temporal and structural features, which often conflict when combined naively. Our solution adapts the Information Bottleneck method to: (1) create unified node representations, (2) perform data-aware dimensionality reduction, and (3) filter noise while preserving critical temporal-structural information. This approach outperforms naive fusion methods by resolving feature conflicts and optimizing representation quality as shown in 3
Table 5: Motivation for co-occurrence encoding structure encoding
CanParl
USLegis
UCI
Untrade
MassgePassing 97.70
86.97
94.08
62.17
Random Walk
97.95
85.56
96.27
63,44
Co-occurrence
99.11
92.27
96.36
93.86
23

---

Table 6:
Ar with different historical neighbor storage.
CanParl
USLegis
UCI
Untrade
Single Hash
98.57
77.87
96.21
67.19
Dual Hash
99.11
92.27
96.36
93.86
Algorithm 1: Structure Memory Update
Data: Link (u, v, t). hash table size H, constants 9, b. p. sets (t)) memory Matrae
Result: Updated Sttru
1 Function Hash (a):
return ((a 9 + b) mod p) mod H;
3 Mitruel uu Hash u) +-;;
// Link u to v // Link u to
M Strrully Hash u u;
for j (v) do
Matrae U, Hash(j)) + j: M J., Hash(u) t- tu; end
// Link u to v's neighbors
// Link u's neighbors to
for i E (2) do 10
M Matrue W Hash(i) <
// Link v to u's neighbors
11
M Matruel Hash(u) (- V:
// Link u's neighbors to
12 end
C Complementary Explanation
C.1 Structure Memory Update
Hash function for hash table with length H is denoted as Hash(a) ((a-q+b) mod p) mod H. where and are constants, p is a large enough constant. When a new link (u, v, () occurs, the structure memory of u and with each other Matrue [u. Hash(v)] t v, M Marrue Hash(u)) t and their neighbors [ut Hash(j)] + j.j E (v)), M And we update the structure memory of neighbor nodes M.trueee Hash(i)] + i, (u). u, E (1))
M (itrooli H Ha))(e)) <- i Si (u), where M atruc and M Strru indicate two hash tables. We elaborate pseudocode of the structure memory update procedure in Algorithm [
For clarity, we further explain why we implement dual-hash strategy. The dual-memory structure (long-term and short-term) effectively captures both historical and recent neighbor records while reducing hash collision effects. A single hash table would degrade feature extraction, worsen collision impacts, and produce less distinguishable embeddings, ultimately harming performance. To verify, evaluated Trans.AP of SALoM using different design for structure memory, including dual-hasi and single-hash. Results are shown in Table 6 Using a dual-hash design for structure memory consistently outperforms single hash design. Verifying the advantage of using dual-hash structure.
C.2 Proof of Upper Bound of Information Bottleneck Loss Function
In this section, we provide proof of the upper bound of the information bottleneck loss function. For clarity, we assume feature denoted as X, label denoted Y, and the intermediate variable denoted Z. We assume the joint distribution p( X, Y, 7) as follows:
Y, 2) = x(Z]
= p(Z
(20)
The IB objective has the form I(Z, Y) (I((,X),, we examine two terms individually. First for I(Z, Y), writing it in full:
I(Z,Y) / dydz p(y. log p(y. 2)
dydd p(y, p(v, zog p(vlz) p(2)
(21)
p(v)p(=)
where p(y|z) is defined by IB Encoder:
p(y|z) = |dx p(z, y/2) = dx p(xpulaa)lala) = f dxx (y)lr)p(=(>)))))
(22)
p(z)
24

---

Let (()(2) be variational approximation to p(yl2), as the IB Decoder. Using the fact that the Kullback-Leilerr divergence is always positive, we have:
XL) /
dd p(u p(vl))loBp((k2) / dy p(vlz)log((u))
(23)
and hence
/ dydz p(u, 2)log g(ulz)
I(Z,Y)
p(v)
/ dydz ply. 2) z)logg(ulz) - dy p(y)logp(y)
(24)
/ dydz ply. =))ogg(ylz) + HY)
Note that the entropy of labels H(Y) is independent of optimization objective, so can be ignored. We rewrite p(y. =) dx p(ar, y, 2) dx p()))/))ll))l))))lll, therefor::
I(Z,Y) 2
dxdydz ptwp)p/plplaaalleeeoleoeolo.
(25)
Then consider XX)
X2, / dzdx p(x, log p(z) = dzdx p(x, =))op((=)
p(2[x)
f dz p(z)logr(z) (26)
As the marginal distribution of is difficult to compute, let r(z) be variational approximation
to the marginal distribution with (O, 1). Since r(Z) 0 = dz p(z))ogp(z) 2
dz p(=))ogr(2). we can summarize the following upper bound:
dxdz p(x)p(2|z)log p22
X(, &
(27)
r(z)
Combining both upper bounds have:
I(2,Y) - -I(,))) 2 dxdydz
(28)
dxdz ((=)p(2(=)1og p(z|z) L
r(z)
In practice, we approximate p(a, v) = p(u)p(u(z) with the empirical data distribution p(a, v)
DM 8a., (x) (z) (w), and hence we have:
N Σ ==1
((2)xn) / du p(olaan)ogg((uel)) - 3p(2 (r, log r(2)
L
(29)
The encoder is of form p(z(x) = (x). where fc an MLP which outputs both K -ddimensional mean and the K K covariance matrix. Then with reparameterization, p(2[x)dz p(c)de, where 2 = f(x, <) is a deterministic function of x and the Gaussian random variable. Assuming our choice of p(zix) and r(z) allows computation of an analytic Kullback-Leibler divergence, we can have optimization objective:
JIB
Σ
Ee Ee-ple) logg(yyn <)) + 8KL (Z))) r(Z)]
(30)
N
1=1
C.3 Leakage-Free TGNNs details
In this section, we give concrete example. Consider that we maintain two memory units and associated mailboxes, ensuring that the timestamps i and (2 of the two memory units satisfy the condition 4 < l2. In this context, ( represents the data information that can be observed in real scenarios, while le stores the memory unit corresponding to the current prediction time frame. This setup allows to effectively update t for future predictions.
25

---

Table 7: Evaluation on leakage-fiee terms on subset of datasets.
Metrics
Datasets
CAWN
TCL
GraphMixer
DyGFormer
SALoM
MOOC
80.15
82.38
82.78
87.52
91.39
UCI
95.18
89.57
93.25
95.79
96,39
Enron
89.56
79.70
82.25
92.47
93.19
Can. Parl. 69,82
68.67
77.04
97.36
98.79
Trans-AP
US Legis.
70.58
69.59
70.74
71.11
75.79
UN Trade 65.39
62.61
66.46
92.52
UN Vote
52.84
51.90
52.11
55.55
68.52
Avg. Rank
3.57
4.86
3.57
2.00
1.00
MOOC
80.38
83.12
84.01
87.91
91.69
UCI
93.87
87.82
91.81
94.49
95.46
Enron
90.45
75.74
84.38
93.33
94.62
Can, Parl.
75.70
72.46
83.17
97.76
98.75
Trans-AUC
US Legis.
77.16
76.27
76.96
77.90
78.84
UN Trade
68.54
64.72
65.52
70.20
91.59
UN Vote
53.09
$1.88
52.46
57.12
65.60
Avg. Rank 3,43
4,86
3.71
2.00
1,00
MOOC
81.42
80.60
81.41
86.96
90.37
UCI
92.73
87.36
91.19
94.54
94.68
Enron
86.35
76.14
75.88
89.76
90,42
Can. Parl.
55.80
54.30
55.91
87.74
95.98
Ind-AP
US Legis.
53.17
52.59
50.71
54.28
62.39
UN Trade 65.24
62.21
62.17
64.55
82.07
UN Vote
49.94
51.60
50.68
55.93
70.74
Avg. Rank
3.29
4.29
4.29
2.14
1,00
MOOC
81.86
81.43
82.77
87.62
90.34
UCI
90.40
84.49
89.30
92.63
92.82
Enron
86.35
76.14
75.88
89.76
91.61
Can. Parl. 58.83
55.83
58.32
89.33
95.65
Ind-AUC
US Legis.
51.49
50.43
47.20
53.21
55.38
UN Trade 67.05
63.76
63.48
67.25
79,20
UN Vote 48.34
50.51
50.04
56.73
67.28
Avg. Rank 3.43
4.29
4.29
2.00
1.00
If for the subsequent event to be predicted (u, v. f3). we have f1 < ts 12, then we will use the state recorded to calculate the memory updater and merge the last event into the memory unit at fz. t < f2 < f3. will use the memory at i2 for model calculations. In this case, the timestamps of subsequent events will be greater than or equal to f3, allowing the memory recorded at l2 to replace that of L,
This ensures that there always memory unit state that is strictly less than the current timestamp and corresponds to the last interaction involved in the calculation. This approach not only avoids the common data leakage issues traditional TGN but also eliminates the memory and computational overhead of searching for updated states in historical memory records. Additional experimental results validating the effectiveness of this approach are presented in Table 7
D Complementary Experiments
D.1 Performance Study With Standard Deviates
As for space constraints, the accuracy of model performance with display of standard deviates is shown in Table g
We observe that SALoM shows improved stability compared with TGN, which stems from LSMUU and Information Bottleneck generating more stable node representations.
D.2 Training Expense of L.SMU
While LSMU does incur higher computational overhead than GRU due to its enhanced capabilities, it maintains reasonable efficiency overall. The GPU memory requirement doubles due to two-expert selection, but we argue this trade-off is justified by the substantial accuracy improvements achieved.
26

---

Table 8: AP&AUC-ROC for transductive and inductive link prediction.
Menies
10BI
Inkko
TGAT
I0N
CAWN
Elyelarc
Net
CNE-
Wiapola
1436
96.31 +:
00.0
08:3--001
00.65 cm
95.5
07.31
04.33 m00
126-000
M00c:
80.23
11377 0.
1323 1.1
91.13 822
10.33
02-78.0.15
1121 m483
81.5 1440
42:42.0000
56.77 0.030
12.30 3.3
11.12.133
01.51
5m
16.1
12.03 1000
Sxi8
0333 6117
0034
19
11gho
10.00 1.37
07 27.04 11.4
08.33 m0
19.33 0m
11.0
08 1001
10.34 6/0
US
12.35
6.300
73.58
11.99
12.58 1022
12.27 BA3
ON Trade
04.94.85 8
63.21 20.95
03.00 137
0.13
00.. 1.000
10:21 +3.06
61.1 20.33
72.50 10.03
06.40-211 1.29
03.6 6.5
Geract
15.21 1:35
09
06.33
+1.04
05.00 11.66
19.36
0.26
V0.5& 0o4
V1.0 00
09:20
M6.36 00
WL.5 841
My Rea.
1.00
1.25
125
RAMI:
10.31
06.17 20.0
55 11.33.300 008
0000 a0
35.31.0000 2:00
00
20.0
93.15 2001
15 200
LeiSM
70.27 8
11
53.224 2:10
113820.11
92.1 D0S
1ame
00:250000
66.72 -31
1111+33 111
6157 1U
9131 -00
Caa Pac
2:223
13.55
13302475
07.11
33.2
64541 200
125
83.72035
$5.02
$448224
128
1S e
6-.0 5m
Tew-n
6
UN Vie
06.
67.18 17:18-100
53 1350-115
63 7676- 234
$1.8 -0
4 cm
65%
$1.15 -m
11:5-000
802--3111
W7 172
8.0.9
99.5
Renc
3.3
1037
123
20
1
TEX 2012
15
M000
70.6
60
$1.0 -0
15:0-011
15
11.56- 1At
1 0.5
50400 +
61.t -0
M. 0.0
88 001
Lan
51.1
13/0
01.42
12:11
43 1350
Soid
91100 4.40
13 15-1-4115
53371110 105
74044 DUTS
52 2230 NNT
15.5
55.0
14.30
0154 unt
Ca Pxc
54.0
5110-131
1.55.000
000
55.0 109
5591 2013
1002:22
2011
UN Tinds
23 2310-1134
05.24 021
60.11 0.12
60.140031
68 68:00 115
6155 103
BAN
12:44:1145
2:5
24.30
02.15.0:4
1727
1:15
Raki:
NN.32 00.12
20.12
56 2454..00 007
42.42.2 000
05.90 00:0
0..0
50 1100
03.11 001
64 0:
05:44
82.77
Earee
81300 01
24.30
1774
53.5.211
0330
1531 20.1
UCI
73.90 20.04
2.29
2011
56.01 12
05302033
15312135
01512011
Tights
101
15 Eeleei:
16.12
61.00
15.77 115
11354 0:4
11
1611-1111 115
40
1730-014
10-00-11
63.56 815
US Vox
38.13 1:43
14.
48.34
14.6.344
55.33 0.5
58.55 11.5
02.44 1/2
95.19
01:99-011
10:0 40
90054000
6667 -0.35
10-003
41-08
Table Evaluation of the implantation of LSMU on TGN.
LSMU
GRU
routing
31.95s
update
23.54s
20.49s
CanParl
combine
1.01s
total
56.5s
20.49s
routing
24.55s
update
12.13s
13.95s
USLegis
combine
0.81s
total
37.49s
13.95s
routing
24.64s
update
19.08s
20.09s
UCI
combine
0.81s
total
44.53s
20.09s
routing
198.09s
update
136.79s
161.78s
Untrade
combine
6.60s
total
341.48s
161.78s
GPU Memory
18.67M
9.25M
27

---

Table 10: Evaluation of the implantation of LSMU on TGN.
CanParl
Untrade
Trans.AP
TLans.UCC
Ind.PP
Ind.AUC
Trans.AF 75,85
Trans.AliC
lod.AP
Ind.AUC
TransAP 60.15 65.4
Trans.Ali
Ind.AP 58.42
Ind.AUC
61:13
69.61
53.8
82.38
1SMU
66.16
74.52
53.51
54.22
X3.69
62.07
65.03
68.93
60.54
Table 11: The impact batch size on model performance.
CanParl
USLegis
Untrade
TGN
SALoM
TGN
SALoM
TGN
SALoM
bs= 10
98.25
99.11
89.8
93.75
69.41
93,86
bs=20
80.95
97.79
85.32
87.99
65.24
90.57
bs= 100
68.02
89.76
76.08
77.1
64.21
79.5
bs=200
61.13
80.13
74.89
75.91
60.15
72.84
bs= 1000
61.87
72.76
70.4
71.36
62.88
65.57
D.3 LSMU as a Universal Memory Updater
Since LSMU is designed for the memory update process. it serves universal memory updater for all memory-based methods. Notably, LSMU was initially developed on TGN; therefore can allow straightforward adaptation other memory-basee methods for accuracy improvements. We also extended our experiments on TGN with the default setting and only alteration of the memory updater support this view as follows.
D.4 The Impact of Batch Size on Model Performance
We evaluated the transductive AP of SALoM and TGN with different batch sizes to verify the impact of batch size on model performance.
As shown in Table 1I model performance degrades batch size increases. Predict accuracy peaks at batch size 10.
This stems from the inner-batch information loss issue shared by memory-based methods. While training optimizations could mitigate this, our focus remains on model design.
D.5
Configurations of SALoM Hlyper-Parameten
Number of sampled neighbors: 10
Dimension of encoder time interval: 100
Dimension of node temporal memory: 172
Dimension of node structure memory: 64
Dimension of output representation: 172
Number of experts in LSMU:
Number of selected experts LSMU:
3 of information bottleneck loss: 1e
E
Broader Impacts
Our research on optimizing continuous-tine dynamic graph models for edge prediction has significant potential to impact various domains where temporal network dynamics are critical. By improving the accuracy and efficiency predicting evolving connections in graphs, this work can enhance applica- tions in social network analysis, recommendation systems, epidemiological modeling, and financial network forecasting. For instance, better edge prediction could enable more precise identification of emerging social trends, improve personalized content delivery, or enhance the tracking of disease spread in real time. In financial systems, could aid detecting fraudulent transactions by modeling dynamic interaction patterns.
28

---

By advancing the in dynamic graph modeling, our research contributes to a deeper understanding of temporal network structures, fostering innovations that are both socially beneficial and ethically responsible.
29


Reviews:
[{"review_id": "3do0HIrZH0", "reviewer": "Reviewer_64Zg", "summary": "This paper introduces a dynamic graph learning framework called SALoM, designed to effectively capture both long- and short-range temporal dependencies while incorporating graph structural information. To overcome limitations of existing memory-based and neighbor-based methods, SALoM introduces a Long-Short Memory Updater (LSMU) that leverages a mixture-of-experts architecture to dynamically balance influences from long-term and short-term memory using neural ODEs and GRUs. It also integrates co-occurrence-based structural encoding through an information bottleneck-based fusion module, allowing temporal and structural features to be combined without conflict. Extensive experiments on multiple benchmarks demonstrate SALoM’s superior performance, particularly in dynamic link prediction tasks, where it significantly outperforms prior methods in both accuracy and adaptability.", "questions": "See weaknesses. Additionally:\n\n1. What is the effect of number of experts with respect to performance?\n2. Why does the performance degrade as the number of historical neighbors increase (Figure 4)?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The idea of using mixture of experts for choosing between long and short-term memory modules in interesting and novel for dynamic graphs.\n2. Significant performance improvements in the transudative setting for UN Trade and Vote datasets.\n\nWeaknesses:\n\n1. Weak motivation for some modules: The authors imply that GRUs are for short-term memory and CFC cells for long-term memory. Why are GRUs for short-term memory? Why not use a simpler module for short-term memory like aggregating last K neighbors as done in TGAT [1]. Rather than using an MoE module for fusion, why not concatenate the outputs of both modules? For the co-occurance encoder, why not use a simple message passing architecture instead to capture the structural information? Your architecture is relatively complicated so every module must be motivated with experiments.\n\n2. Slight performance degradation in the inductive setting.\n\n[1] Xu et al. Inductive Representation Learning on Temporal Graphs. ICLR. 2020", "was_revised": true, "final_justification": "The authors addressed my concerns but the information leakage issue pointed out by another reviewer is significant. I will keep my score the same.", "rebuttal": "We are glad that the reviewer appreciates our work from the perspective of novelty and performance. Thank you for pointing out the weak motivation issue. We have carefully revised the manuscript according to your constructive suggestions. Below we address the main points raised in the review.\n\n\n## W1：Individual motivations:\nWe apologize for any confusion and now provide analytical and experimental motivation for each SALoM component:\n\n### a) ODE-based memory updater for long-term temporal correlation:\nTraditional sequential methods (RNNs/GRUs) treat dynamic graphs as discrete events, struggling with temporal continuity and long-term dependencies due to gradient issues. In contrast, ODE-based methods preserve continuity and avoid gradient issues through second-order derivative integration, demonstrating superior long-term dependency capture with minimal information loss (Table 1, [1]). This experiment is designed to predict the label(-1/1) of the first node with node embedding propagated through a linear graph  of length(n).\nHowever, our ablation studies (Fig. 2) show that over-emphasizing long-term dependencies can cause over-globalization. We therefore propose LSMU, an MoE-based approach that dynamically balances long/short-term dependencies, adaptively selecting the optimal processing.\n\nTable 1: Accuracy under different numbers of sampling neighbors\n| |n=3|n=9|n=15|n=20|\n|---|---|---|---|---|\n|DyGFormer|100.0±0.0|53.02±6.06|42.80±16.25|42.79±19.62|\n|DyRep|100.0±0.0|47.93±2.73|48.60±2.48|50.47±2.88|\n|GraphMixer|100.0±0.0|52.80±5.56|52.49±5.36|52.04±8.20|\n|JODIE|100.0±0.0|**100.0±0.0**|60.0±14.91|50.87±2.46|\n|TGAT|100.0±0.0|47.87±2.72|50.53±2.15|49.07±1.55|\n|TGN|100.0±0.0|48.13±1.63|48.67±2.76|50.13±2.17|\n|ODE_based|**100.0±0.0**|99.93±0.21|**93.47±8.78**|**88.93±12.06**|\n\n[1]GRAVINA, Alessio, et al. Long range propagation on continuous-time dynamic graphs. *arXiv preprint arXiv:2406.02740*, 2024.\n\n### b) GRU for short-term memory updater:\nIn dynamic graphs, recent neighbor interactions typically provide the most valuable information. While simple RNNs and K-neighbor aggregation fail to effectively capture these patterns (due to vanishing gradients and limited neighbor hops respectively), GRU's gating mechanism enables robust short-term dependency learning while complementing our ODE-based long-term capture.\nOur extended experiments confirm that simple RNN suffers from gradient vanishing, leading to performance degradation (mitigated by GRU’s gating mechanism). Besides, K-neighbor aggregation is limited to 1–2 hops, as wider aggregation blurs node representations.\n\nTable 2: Trans.AP under different short-term memory updaters\n| |CanParl|USLegis|UCI|Untrade|\n|:---:|:---:|:---:|:---:|:---:|\n|RNN|98.81|89.90|96.00|92.82|\n|AggerateNeighbor|98.77|77.31|96.22|92.83|\n|GRU|**99.11**|**92.27**|**96.36**|**93.86**|\n\n### c) MoE for long-short term fusion:\nWe propose MoE to fuse long-short term temporal correlations while adaptively selecting optimal backbones and weights per input. In temporal graphs, events exhibit mixed dependencies, but naive fusion methods (e.g., concatenation, voting) treat edges uniformly, ignoring their distinct evolutionary patterns. MoE addresses this by dynamically routing events to specialized experts based on node/edge features, achieving superior performance (see Table). For instance, on USLegis, MoE improves Trans.AP by 15.3% over concatenation and 16.2% over voting, demonstrating its ability to capture varied temporal scales.\n\nTable 3: Trans.AP under different long-short term fusers\n| |CanParl|USLegis|UCI|Untrade|\n|:---:|:---:|:---:|:---:|:---:|\n|Concat|96.90|76.93|96.02|92.00|\n|Avg-Voting|98.65|76.06|96.23|92.99|\n|MoE|**99.11**|**92.27**|**96.36**|**93.86**|\n\n### d) Co-occurrence encoder as structure encoder:\nOur co-occurrence encoder efficiently captures structural patterns while maintaining computational feasibility. Unlike traditional methods that trade off expressiveness for efficiency, it encodes neighbor co-occurrence frequencies as relative structural features, enhanced by a hash-based memory system for accelerated processing. Experiments demonstrate its superiority over basic message passing and random walk approaches, with significantly better performance at manageable computational cost.\n\nTable 4: Trans.AP with different structure encoders\n| |CanParl|USLegis|UCI|Untrade|\n|:---:|:---:|:---:|:---:|:---:|\n|MassgePassing|97.70|86.97|94.08|62.17|\n|RandomWalk|97.95|85.56|96.27|63.44|\n|Co-occurrence|**99.11**|**92.27**|**96.36**|**93.86**|\n\n### d) IB for feature fusion:\nTemporal graphs require effective fusion of temporal and structural features, which often conflict when combined naively. Our solution adapts the Information Bottleneck method to: (1) create unified node representations, (2) perform data-aware dimensionality reduction, and (3) filter noise while preserving critical temporal-structural information. This approach outperforms naive fusion methods by resolving feature conflicts and optimizing representation quality (as justified in Fig.3).\n\n## W2：Performance in inductive setting\n\nAlthough SALoM exhibits a slight performance decline in inductive settings due to its relatively higher reliance on prior knowledge, it still achieves state-of-the-art (SOTA) performance on most datasets. Notably, it achieves an AP above 85 on the UNtrade dataset, and overall, it remains the most effective method on average across all datasets.\n\n## Q1: Num of experts\n\nThanks for your insightful advice, we have extended our experiments on more hyperparameter settings of LSMU module as below.\n\nFor the results, we discover that in general, the hyperparameters of MoE module hold rather small impact on model performance. However, a few anomalies are discovered. First, as observed in the third row, selecting only one expert per iteration—an unconventional setting—leads to performance degradation due to the lack of fusion between long-term and short-term temporal features, resulting from selecting a single backbone for all edges in a batch. Second, the experimental setup in the last row shows poor performance on the USLegis dataset, attributed to an insufficient dataset size to meet the demands of each expert, causing all experts to be underfitted.\n\nTable 5: Trans.AP under different number of selected and total experts\n|n_selected|n_experts|CanParl|USLegis|UCI|Untreade|\n|:---:|:---:|:---:|:---:|:---:|:---:|\n|2|4|98.38|91.94|96.16|92.54|\n|3|4|98.92|91.09|96.42|91.54|\n|1|6|98.88|78.51|96.20|93.18|\n|2|6|99.11|92.27|96.36|93.86|\n|3|6|98.92|92.13|96.48|91.90|\n|2|8|98.70|92.13|96.51|93.19|\n|3|8|99.00|79.40|96.14|92.82|\n\n## Q2: Performance degrade as the number of historical neighbors increase \n\nWe appreciate this important observation. The performance degradation with increasing historical neighbors occurs because: (1) larger neighbor samples lead to higher similarity between nodes during aggregation, reducing embedding distinctiveness and potentially causing over-globalization; (2) for memory updates, iterate mechanisms already preserve long-term information, making excessive sampling counterproductive as it disproportionately weights long-term dependencies at the expense of crucial short-term patterns."}, {"review_id": "TmqfZOmAYk", "reviewer": "Reviewer_sYk3", "summary": "This paper proposes SALoM (Structure-Aware Long-Short Memory), a novel framework for temporal graph learning that aims to balance the trade-off between short-range and long-range temporal dependencies—an issue that is often overlooked in prior work. The authors design a memory module enhanced by a Long-Short Memory Updater (LSMU), leveraging a Mixture-of-Experts (MoE) architecture to adaptively combine different temporal signals. Structural information is integrated via co-occurrence encoding and an information bottleneck (IB)-based fusion mechanism. Experimental results across thirteen dynamic graph datasets show that SALoM achieves strong performance in dynamic link prediction tasks, outperforming existing state-of-the-art models, particularly on datasets like USLegis, UNTrade, and UNVote.", "questions": "1. The color and shape coding in Figure 5 could be more distinct. Currently, it is visually cluttered and hard to differentiate across batch size settings and models.\n2. Please complete or remove Appendix B to avoid confusion.\n3. What would happen if the structural encoder simply used a unified co-occurrence hash map (i.e., no distinction between short- and long-range neighbors)? Would this significantly degrade performance?\n4. MoE hyperparameter sensitivity.\n+ What is the impact of setting r = 1 or using only a single GRU/CFC expert? Does performance degrade significantly?\n+ How does increasing the number of experts or varying the r value affect training speed and accuracy?\n+ A brief analysis or ablation on these hyperparameters would provide more clarity on their necessity and practical trade-offs.\n5. Can the authors further justify the choice of using Neural ODEs over more mainstream approaches (e.g., attention with decay) for modeling long-range dependencies?", "limitations": "1. The scalability limitations revealed in Figure 5, especially with large batch sizes.\n2. The computational cost associated with MoE routing and ODE solvers (even in closed-form variants).", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\nS0: The motivation to simultaneously handle short- and long-range temporal dependencies is well-justified and addresses a practical gap in temporal graph learning.\n\nS1: The design of LSMU using a MoE mechanism (with GRU and CFC experts) is an elegant and potentially effective way to achieve dynamic temporal adaptation.\n\nS2: The experimental section is comprehensive, covering 13 datasets and a broad set of baselines, with ablations on memory modules, fusion strategies, and neighborhood size.\n\nS3: The authors offer a solid discussion on the limitations of prior approaches, such as RNN-based memory methods and neighbor-sequence models, highlighting how SALoM differs in intent and implementation.\n\n**Weaknesses**\n\nW0: While the paper introduces several technical modules (ODEs, IB fusion, hash-based co-occurrence encoding), their individual motivations are not always clearly articulated. Clarifying why each design choice is necessary would improve the exposition.\n\nW1: The overall writing feels verbose and sometimes disorganized, making it harder to follow the technical contributions, particularly in Sections 3.1 and 3.2.\n\nW2: The rationale behind using Neural ODEs (and the CFC variant) for temporal memory modeling could be better justified, especially compared to simpler alternatives like attention or GRUs.\n\nW3: Several inconsistencies and typos were found, including:\n+ In Figures 2–4, the x-axis is uniformly labeled “Number of Sampled Neighbors?”, which appears to be an error.\n+ Appendix B is present only with a heading but lacks any actual content.\n\nW4: From Figure 5, performance degrades as batch size increases, potentially limiting SALoM’s scalability to larger datasets, where larger batches are often necessary for efficiency.\n\nW5: The number of experts (Q) and selected experts (r) in the MoE mechanism are not explored in depth. Understanding their effect on performance and computational cost would be important for users aiming to apply SALoM in practice.", "was_revised": true, "final_justification": "While your response has addressed most of my concerns, I would still recommend taking advantage of the additional page allowed in the final version to further polish the writing and provide clearer explanations for some of the module motivations. This will significantly improve the readability of your work. So I plan to raise my rating to 4, and I hope the authors make corresponding improvements.", "rebuttal": "We sincerely appreciate the reviewer's positive feedback on our model design and experimental rigor, as well as their constructive comments that helped improve our paper. Below we respond to the main points raised.\n\n## W0：Individual motivations:\nWe apologize for any confusion and now provide analytical and experimental motivation for each SALoM component:\n\n### a) ODE-based memory updater for long-term temporal correlation:\nTraditional sequential methods (RNNs/GRUs) treat dynamic graphs as discrete events, struggling with temporal continuity and long-term dependencies due to gradient issues. In contrast, ODE-based methods preserve continuity and avoid gradient issues through second-order derivative integration, demonstrating superior long-term dependency capture with minimal information loss (Table 1, [1]). This experiment is designed to predict the label(-1/1) of the first node with node embedding propagated through a linear graph  of length(n).\nHowever, our ablation studies (Fig. 2) show that over-emphasizing long-term dependencies can cause over-globalization. We therefore propose LSMU, an MoE-based approach that dynamically balances long/short-term dependencies, adaptively selecting the optimal processing.\n\nTable 1: Accuracy under different numbers of sampling neighbors\n| |n=3|n=9|n=15|n=20|\n|---|---|---|---|---|\n|DyGFormer|100.0±0.0|53.02±6.06|42.80±16.25|42.79±19.62|\n|DyRep|100.0±0.0|47.93±2.73|48.60±2.48|50.47±2.88|\n|GraphMixer|100.0±0.0|52.80±5.56|52.49±5.36|52.04±8.20|\n|JODIE|100.0±0.0|**100.0±0.0**|60.0±14.91|50.87±2.46|\n|TGAT|100.0±0.0|47.87±2.72|50.53±2.15|49.07±1.55|\n|TGN|100.0±0.0|48.13±1.63|48.67±2.76|50.13±2.17|\n|ODE_based|**100.0±0.0**|99.93±0.21|**93.47±8.78**|**88.93±12.06**|\n\n[1]GRAVINA, Alessio, et al. Long range propagation on continuous-time dynamic graphs. *arXiv preprint arXiv:2406.02740*, 2024.\n\n### b) GRU for short-term memory updater:\nIn dynamic graphs, recent neighbor interactions typically provide the most valuable information. While simple RNNs and K-neighbor aggregation fail to effectively capture these patterns (due to vanishing gradients and limited neighbor hops respectively), GRU's gating mechanism enables robust short-term dependency learning while complementing our ODE-based long-term capture.\nOur extended experiments confirm that simple RNN suffers from gradient vanishing, leading to performance degradation (mitigated by GRU’s gating mechanism). Besides, K-neighbor aggregation is limited to 1–2 hops, as wider aggregation blurs node representations.\n\nTable 2: Trans.AP under different short-term memory updaters\n| |CanParl|USLegis|UCI|Untrade|\n|:---:|:---:|:---:|:---:|:---:|\n|RNN|98.81|89.90|96.00|92.82|\n|AggerateNeighbor|98.77|77.31|96.22|92.83|\n|GRU|**99.11**|**92.27**|**96.36**|**93.86**|\n\n### c) MoE for long-short term fusion:\nWe propose MoE to fuse long-short term temporal correlations while adaptively selecting optimal backbones and weights per input. In temporal graphs, events exhibit mixed dependencies, but naive fusion methods (e.g., concatenation, voting) treat edges uniformly, ignoring their distinct evolutionary patterns. MoE addresses this by dynamically routing events to specialized experts based on node/edge features, achieving superior performance (see Table). For instance, on USLegis, MoE improves Trans.AP by 15.3% over concatenation and 16.2% over voting, demonstrating its ability to capture varied temporal scales.\n\nTable 3: Trans.AP under different long-short term fusers\n| |CanParl|USLegis|UCI|Untrade|\n|:---:|:---:|:---:|:---:|:---:|\n|Concat|96.90|76.93|96.02|92.00|\n|Avg-Voting|98.65|76.06|96.23|92.99|\n|MoE|**99.11**|**92.27**|**96.36**|**93.86**|\n\n### d) Co-occurrence encoder as structure encoder:\nOur co-occurrence encoder efficiently captures structural patterns while maintaining computational feasibility. Unlike traditional methods that trade off expressiveness for efficiency, it encodes neighbor co-occurrence frequencies as relative structural features, enhanced by a hash-based memory system for accelerated processing. Experiments demonstrate its superiority over basic message passing and random walk approaches, with significantly better performance at manageable computational cost.\n\nTable 4: Trans.AP with different structure encoders\n| |CanParl|USLegis|UCI|Untrade|\n|:---:|:---:|:---:|:---:|:---:|\n|MassgePassing|97.70|86.97|94.08|62.17|\n|RandomWalk|97.95|85.56|96.27|63.44|\n|Co-occurrence|**99.11**|**92.27**|**96.36**|**93.86**|\n\n### d) IB for feature fusion:\nTemporal graphs require effective fusion of temporal and structural features, which often conflict when combined naively. Our solution adapts the Information Bottleneck method to: (1) create unified node representations, (2) perform data-aware dimensionality reduction, and (3) filter noise while preserving critical temporal-structural information. This approach outperforms naive fusion methods by resolving feature conflicts and optimizing representation quality (as justified in Fig.3).\n\n## W1&W3&Q1&Q2: Writing issues\nWe sincerely appreciate your careful review. We are currently: (1) reorganizing key sections, (2) clarifying mathematical explanations, and (3) correcting all identified inconsistencies. A thorough proofreading will ensure these issues are resolved in the final version.\n\n## W2&Q5: Neural ODEs rationale\n\nThanks for your insightful advice. We further justify the introduction of Neural ODEs. Unlike RNNs' discrete Backpropagation Through Time (BPTT), which triggers gradient vanishing. ODE-based methods excel in dynamic graphs by: (1) enabling continuous memory updates for irregular timestamps, and (2) preventing gradient vanishing through second-order derivative integration.\nTo be specific, gradient computation in ODEs employs the adjoint method, where gradients are calculated by solving an augmented ODE backward. For instance, the adjoint state $$ a(t) = -\\frac{\\partial\\mathscr{L}}{\\partial h(t)} $$  has a time derivative given by $$ \\frac{da(t)}{dt} = -a(t)^T \\frac{\\partial f(t, h(t), \\theta_t)}{\\partial h(t)} $$, ensuring gradient stability through integration.\n\n## W4: Batch size impact\n\nThanks for your insightful advice. We acknowledge the performance trade-off with larger batches, a common challenge for memory-based methods. Our experiments show SALoM maintains better accuracy than TGN across batch sizes (see results below), though both are affected by information loss. While training optimizations could mitigate this, our focus remains on model design.\n\nTable 5: Trans.AP comparison under different batch sizes\n|  | CanParl |  | USLegis |  | Untrade |  |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|  | TGN | SALoM | TGN | SALoM | TGN | SALoM |\n| bs=10 | 98.25 | 99.11 | 89.80 | 93.75 | 69.41 | 93.86 |\n| bs=20 | 80.95 | 97.79 | 85.32 | 87.99 | 65.24 | 90.57 |\n| bs=100 | 68.02 | 89.76 | 76.08 | 77.10 | 64.21 | 79.50 |\n| bs=200 | 61.13 | 80.13 | 74.89 | 75.91 | 60.15 | 72.84 |\n| bs=1000 | 61.87 | 72.76 | 70.40 | 71.36 | 62.88 | 65.57 |\n\n\n## W5&Q4: MoE configuration analysis\n\nThanks for your insightful advice. The extended experimental results are as below. Our experiments show that MoE hyperparameters (expert count Q and selection count r) generally have minimal impact on performance, with two exceptions: (1) selecting only one expert degrades performance (e.g., −15% AP on USLegis) due to lost feature fusion, and (2) excessive experts underfit small datasets (e.g., USLegis). Computational costs scale predictably: VRAM grows linearly with Q (+2.7MB/expert), while training time increases sublinearly (~0.02s → 0.027s/iter for 4→8 experts). The optimal balance is 6 experts with 2 selected, achieving peak accuracy (e.g., 93.86 AP on Untrade) with moderate resource use (18.67MB VRAM). We will incorporate these findings in the final version.\n\nTable 6: Trans.AP under different number of selected and total experts\n|n_selected|n_experts|CanParl|USLegis|UCI|Untreade|\n|:---:|:---:|:---:|:---:|:---:|:---:|\n|2|4|98.38|91.94|96.16|92.54|\n|3|4|98.92|91.09|96.42|91.54|\n|1|6|98.88|78.51|96.20|93.18|\n|2|6|99.11|92.27|96.36|93.86|\n|3|6|98.92|92.13|96.48|91.90|\n|2|8|98.70|92.13|96.51|93.19|\n|3|8|99.00|79.40|96.14|92.82|\n\nTable 7: Train time and GPU memory usage under different numbers of selected and total experts\n|n_selected|n_experts|seconds/iter|GPU memory|\n|---|---|---|---|\n|2|4|0.0213|15.98M|\n|3|4|0.0236|15.98M|\n|2|6|0.0238|18.67M|\n|3|6|0.0241|18.67M|\n|2|8|0.0276|21.37M|\n|3|8|0.0272|21.37M|\n\n\n## Q3: Single hashmap impact\n\nThank you for your valuable feedback. Our extended experiments confirm the limitations of using a single hashmap. \nThe dual-memory structure (long-term and short-term) effectively captures both historical and recent neighbor records while reducing hash collision effects. A single hash table would degrade feature extraction, worsen collision impacts, and produce less distinguishable embeddings, ultimately harming performance. The results below support this conclusion.\n\nTable 8: Trans.AP under using SingleHash and DualHash\n|  | CanParl | USLegis | UCI | Untrade |\n|:---:|:---:|:---:|:---:|:---:|\n| Single Hash | 98.57 | 77.87 | 96.21 | 67.19 |\n| Dual Hash | **99.11** | **92.27** | **96.36** | **93.86** |"}, {"review_id": "vHas74MNCD", "reviewer": "Reviewer_vKih", "summary": "This paper proposes SALoM, a dynamic graph learning framework that addresses three important challenges in temporal graph networks:  \n\nThe effectiveness of capturing long-range temporal dependencies,  \n\nThe difficulty of balancing long/short-term correlations, and  \n\nThe consistency of integrating structural information.  \n\nExperiments on multiple datasets show state-of-the-art performance, with particularly large gains on legislative/voting graphs. SALoM consistently outperforms baselines in both transductive and inductive link prediction tasks.", "questions": "Question 1: Can you please quantify computational costs, such as training runtime and memory space occupation, of LSMU’s expert routing versus existing updaters? \n\n \n\nQuestion 2: For sparse graphs, does co-occurrence encoding degrade due to hash collisions?  \n\n \n\nQuestion 3: Can SALoM handle edge feature dynamics, such as evolving transaction amounts? \n\n \n\nQuestion 4: Please also check the weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths: \n\n1(quality) This paper is methodologically rigorous, with claims well-supported by both theoretical analysis and extensive experiments. Comprehensive ablation studies also validate the effectiveness of individual components in a convincing manner. \n\n \n\n2 (clarity) This paper is well-structured, with a clear narrative flow from problem definition to methods and results. This paper also provides sufficient details, such as hyper-parameters and dataset splits. The code is also released in Anonymous GitHub which further contributes to good reproducibility. \n\n \n\n3 (significance) The experimental results of SALoM proposed by this paper clearly outperform typical memory-based and sequence-based baselines, advancing the field by harmonizing existing methods’ advantages while mitigating their limitations. \n\n \n\n4 (originality) Based on Section 2 and Table 1, this paper effectively contrasts SALoM with prior work, highlighting limitations in handling long-term dependency and managing training costs. \n\n \n\n \n\n \n\nWeaknesses: \n\n1(quality) Although efficiency trade-offs are discussed in subsection 4.5, Figure 5 lacks a detailed comparison of Long-Short Memory Updater (LSMU) overhead, such as the routing latency of MoE, versus selective baselines, which is important in real-world development. \n\n \n\n2 (clarity) The neural ODE equations (Equations 1 and 2) and Information Bottleneck (IB) objectives (Equations 17, 18, and 19) could benefit from more intuitive explanations or illustrative examples. For example, a toy case study can be discussed with simple plots to show how ODEs applied in SALoM could mitigate the gradient vanishing issue. \n\n \n\n3 (significance) Concrete examples of deployed applications are encouraged, such as policy change tracking, as they help strengthen the case for real-world significance. \n\n \n\n4 (originality) This paper could better explain how SALoM’s MoE design differs from other MoE-based graph neural networks.", "was_revised": true, "final_justification": "The authors have addressed the issues that I raised in the rebuttal.", "rebuttal": "We sincerely appreciate the reviewer's recognition of our work as a solid contribution and their valuable suggestions for potential extensions. Below we respond to the main points raised in the review.\n\n## W1&Q1: LSMU overhead\n\nWe appreciate the insightful feedback. We have extended our experiments and show the results below. While LSMU does incur higher computational overhead than GRU due to its enhanced capabilities, it maintains reasonable efficiency overall. \nThe GPU memory requirement doubles due to its two-expert selection, but we argue this trade-off is justified by the substantial accuracy improvements achieved.\n\nTable 1: Training time and GPU memory comparison of GPU and LSMU\n|  |  | LSMU | GRU |\n|:---:|:---:|:---:|:---:|\n| CanParl | routing | 31.95s | - |\n|  | update | 23.54s | 20.49s |\n|  | combine | 1.01s | - |\n|  | total | 56.5s | 20.49s |\n| USLegis | routing | 24.55s | - |\n|  | update | 12.13s | 13.95s |\n|  | combine | 0.81s | - |\n|  | total | 37.49s | 13.95s |\n| UCI | routing | 24.64s | - |\n|  | update | 19.08s | 20.09s |\n|  | combine | 0.81s | - |\n|  | total | 44.53s | 20.09s |\n| Untrade | routing | 198.09s | - |\n|  | update | 136.79s | 161.78s |\n|  | combine | 6.60s | - |\n|  | total | 341.48s | 161.78s |\n| GPU Memory |  | 18.67M | 9.25M |\n\n## W2: Equation clarity\n\nSorry for the confusion. We appreciate the feedback and will add illustrative plots in the next version to enhance clarity.\n\nEquation 1 defines Neural ODE, where the algorithm does not directly model the mapping of the independent variables to the dependent variables but instead models the derivative of the corresponding function. Also, this approach computes gradients by integrating the second-order derivatives over the relevant time interval, eliminating the need for discrete chain-rule multiplications and thereby avoiding the vanishing gradient problem. Equation 2 specifies the closed-form solution of the Liquid Time-Constant (LTC) network employed in our method. Building upon the original framework, and also introducing a gating-like mechanism and utilize a trainable network to compute the time-dependent damping coefficient, capturing the temporal influence on events.\n\nThe objective of the Information Bottleneck (IB) is to generate an intermediate representation of the original representation that maximizes information correlating with the training target while minimizing noise in the original representation. Here, correlation is characterized by mutual information. However, as mutual information is difficult to compute in practice, we optimize an upper bound of the target function instead. By employing variational approximation, we scale the target function to derive the mathematical form of its upper bound. This upper bound can then be transformed into a form combining Binary Cross-Entropy (BCE) loss and KL divergence, with the detailed derivation provided in Appendix C. Consequently, when aggregating multiple features using the IB approach, not only is feature fusion achieved, but noise reduction is also inherently accomplished.\n\n## W3: Application examples\n\nWe appreciate the suggestion to include real-world applications. While time constraints prevented us from implementing a fast-deployable case study in this version (as is common in similar works), we recognize their importance and will explore concrete application scenarios (such as policy change tracking) in future work to better highlight the practical relevance of our approach.\n\n## W4: Differences from other MoE-based GNNs\n\nThanks for your insightful advice. Unlike typical MoE-GNNs that apply mixture-of-experts during node aggregation, our approach uniquely integrates MoE exclusively within the LSMU to explicitly separate long-term and short-term dependency learning. This targeted implementation allows MoE to serve distinct functional roles based on contextual requirements, making our architecture both more specialized and adaptable than conventional implementations.\n\n## Q2: Hash collisions\n\nWe appreciate the insightful question about hash collisions. While collisions can theoretically impact accuracy, SALoM's dual hash table design (long-term and short-term) effectively mitigates this by simultaneously capturing behavior patterns at different timescales. Importantly, collisions actually provide a beneficial side effect by facilitating timely updates and replacement of outdated information in the hash tables.\n\n## Q3: Edge feature dynamics\n\nThanks for your insightful advice. The dynamic graph task inherently involves continuously changing edges, the edges with dynamics edge will be viewed as a new edge despite of the same source and destination nodes. Consequently, SALoM is naturally designed to effectively handle these changing edge features."}, {"review_id": "kDrxsv3RmE", "reviewer": "Reviewer_dfEN", "summary": "This paper focuses on temporal graph learning. It proposes a new method, SALoM, to capture long-range neighborhood temporal correlations. SALoM adopts ODE to update memory and introduces a long-short memory updater to balance long-range and short-range temporal dependencies. And to incorporate both structural and temporal information, it adopts information bottleneck-based fusion, where the structural information is obtained by co-occurrence encoding. SALoM outperforms baselines, especially on USLegis, UNtrade, and UNvote.", "questions": "Q1. Can LSMU be adopted to other memory-based models, such as TGN, to improve their performance?", "limitations": "See weaknesses.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "S1. This paper presents a novel approach to capturing long-range neighborhood temporal correlations and balancing the learning of temporal and structural information. The components of SALoM are technically sound. \n\nS2. SALoM demonstrates outstanding performance across several datasets, highlighting its effectiveness in addressing certain predictive scenarios.\n\nW1. A major concern is that the datasets used in the experiments are outdated. Several studies have shown that these datasets are no longer sufficient for evaluating the full capacity of DGNNs. I recommend evaluating SALoM on the more recent TGB[R1] and TGB-Seq[R2] benchmarks to ensure a comprehensive assessment.\n\nW2. There may be a potential flaw in the implementation of SALoM, which is shared by other memory-based methods such as TGN, DyRep, and JODIE. Specifically, when considering two edges, (a,b,t1) and (a,c,t1), these edges are placed in different batches during testing because there are too many edges occurring at t1. However, when testing (a,c,t1), the memory of node \"a\" has already been updated by (a,b,t1), leading to information leakage. This violates the model’s design, as it should not have access to edges at time t1 when the prediction time is t1.\n\n[R1] Temporal Graph Benchmark for Machine Learning on Temporal Graphs\n\n[R2] TGB-Seq Benchmark: Challenging Temporal GNNs with Complex Sequential Dynamics\n\nMinor: \nFigures 2 and 3 mistakenly include the x-label \"Number of Sampled Neighbors,\" which should only appear in Figure 4.", "was_revised": true, "final_justification": "The information leakage in the proposed method leads to an unfair comparison with non-memory-based methods. While I acknowledge the novelty of the model design, I would recommend a thorough revision of the implementation and corresponding experimental results.", "rebuttal": "We are glad that the reviewer appreciates our work as a solid and effective contribution. Thank you for new dataset resource that we can further extend our study on. We have carefully revised the manuscript according to your constructive suggestions. Below, we address the main points raised in the review.\n\n## W1: New datasets\n\nThank you for raising this question. But due to time constraints and huge GPU memory demands, and with access limited to only two NVIDIA RTX 4090 GPUs, we only tested our model on a subset of datasets from TGB-seq to evaluate MRR performance. The results are presented below. We observe that SALoM outperforms all baselines on non-bipartite datasets, especially by a large scale on Flickr. For bipartite datasets, SALoM outperforms all universal baselines, however its performance is slightly inferior to the specially tailored baseline.\n\nThank you again for the new dataset resource, we will further study into them and add evaluations on them in the next version. \n\n|            | Flickr           | YouTube          | ML-20M       |\n| ---------- | ---------------- | ---------------- | ------------ |\n| JODIE      | 46.21 ± 0.83     | 41.67 ± 2.86     | 21.16 ± 0.73 |\n| DyRep      | 38.04 ± 4.19     | 35.12 ± 4.13     | 19.00 ± 1.69 |\n| TGAT       | 23.53 ± 3.35     | 43.56 ± 2.53     | 10.47 ± 0.20 |\n| TGN        | 46.03 ± 6.78     | 55.16 ± 5.89     | 23.99 ± 0.20 |\n| CAWN       | 48.69 ± 6.08     | 47.55 ± 1.08     | 12.31 ± 0.02 |\n| TCL        | 40.00 ± 1.76     | 50.17 ± 1.98     | 12.04 ± 0.02 |\n| GraphMixer | 45.01 ± 0.08     | 58.87 ± 0.12     | 21.97 ± 0.17 |\n| DyGFormer  | 49.58 ± 2.87     | 46.08 ± 3.44     | OOT          |\n| SGNN-HN    | -                | -                | 33.12 ± 0.01 |\n| SALoM      | **68.12 ± 0.09** | **63.27 ± 0.19** | 30.79 ± 0.59 |\n\n## W2: Potential flaw of information leakage\n\nWe sincerely appreciate you identifying this important issue regarding test information leakage, which indeed affects our method and comparable memory-based baselines (TGN, DyRep, JODIE). To maintain fairness in our current comparisons while addressing this concern, we will: (1) retain the existing evaluation setup for this version to ensure consistent benchmarking, and (2) implement crucial fixes in our codebase to properly batch same-timestamp edges during testing, with updated results to be included in our next manuscript version.\n\n## Q1: LSMU adopted to other memory-based models\n\nThank you for your question, we will elaborate below.\n\nSince LSMU is designed for the memory update process, it serves as a universal memory updater for all memory-based methods. Notably, LSMU was initially developed on TGN; therefore can allow straightforward adaptation to other memory-based methods for accuracy improvements. We also extended our experiments on TGN with the default setting and only alteration of the memory updater to support this view as follows.\n\n|      |  CanParl  |           |           |           |  USLegis  |           |           |           |  Untrade  |           |           |           |\n| :--: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: |\n|      | Trans.AP  | Trans.AUC |  Ind.AP   |  Ind.AUC  | Trans.AP  | Trans.AUC |  Ind.AP   |  Ind.AUC  | Trans.AP  | Trans.AUC |  Ind.AP   |  Ind.AUC  |\n| GRU  |   61.13   |   69.61   |   52.70   |   53.98   |   74.89   |   82.38   |   61.87   |   64.47   |   60.15   |   63.69   |   58.34   |   58.11   |\n| LSMU | **66.16** | **74.52** | **53.51** | **54.22** | **75.85** | **83.69** | **62.07** | **65.03** | **65.40** | **68.93** | **58.42** | **60.54** |"}]

Author rebuttal:

Meta-review: The paper discusses a dynamic graph learning approach that captures both long- and short-term temporal dependencies while integrating structural information. The architecture is motivated by the need to address limitations in existing memory-based temporal graph networks and includes a mixture-of-experts mechanism to balance temporal correlations across timescales.

The initial reviews identified the work as methodologically solid, with promising performance across a broad set of benchmarks. However, a major concern was raised regarding a potential data leakage issue, stemming from improper memory updates across edges occurring at the same timestamp. This flaw, inherited from previous memory-based models, significantly impacted the trustworthiness of the reported results and risked undermining the paper’s main empirical claims.

In the post-rebuttal discussion, the authors responded promptly and thoroughly. They implemented a redesigned memory update strategy to eliminate the leakage, and they re-ran key experiments using this corrected protocol. The updated results, reported in the discussion phase, confirmed that SALoM still achieves state-of-the-art performance, even under leakage-free conditions. This strongly validates the method’s robustness and provides a valuable experimental protocol for the broader community, which has been relying on flawed practices.

The discussion was constructive and engaged, and all reviewers acknowledged the authors’ effort to address concerns with rigor and transparency. Multiple reviewers raised their scores accordingly, recognizing that the authors not only fixed the problem but improved the clarity and reproducibility of their work.

This paper delivers both a strong technical contribution and a responsible, community-beneficial correction to a prevalent benchmarking issue. The approach is well-justified, the experimental protocol is now sound, and the results are compelling.

I recommend to accept this paper.
Decision: Accept (poster)