N-ForGOT: Towards Not-forgetting and Generalization of Open Temporal Graph Learning
Authors: Liping Wang
Xujia Li
Jingshu Peng
Yue Wang
Chen Zhang
Yan Zhou
Lei Chen
Venue: ICLR 2025 Poster (2025)
Keywords: temporal graph neural networks; continual learning; generalization
Primary area: learning on graphs and other geometries & topologies
TL;DR: This paper targets balancing knowledge preservation from existing data with swift adaptation to incoming, unseen classes in open temporal graphs.

Abstract: Temporal Graph Neural Networks (TGNNs) lay emphasis on capturing node interactions over time but often overlook evolution in node classes and dynamic data distributions triggered by the continuous emergence of new class labels, known as the open-set problem. This problem poses challenges for existing TGNNs in preserving learned classes while rapidly adapting to new, unseen classes. To address this, this paper identifies two primary factors affecting model performance on the open temporal graph, backed by a theoretical guarantee:  (1) the forgetting of prior knowledge and (2) distribution discrepancies between successive tasks. Building on these insights, we propose N-ForGOT, which incorporates two plug-in modules into TGNNs to preserve prior knowledge and enhance model generalizability for new classes simultaneously. The first module preserves previously established inter-class connectivity and decision boundaries during the training of new classes to mitigate the forgetting caused by temporal evolutions of class characteristics. The second module introduces an efficient method for measuring distribution discrepancies with designed temporal Weisfeiler-Lehman subtree patterns, effectively addressing both structural and temporal shifts while reducing time complexity. Experimental results on four public datasets demonstrate that our method significantly outperforms state-of-the-art approaches in prediction accuracy, prevention of forgetting, and generalizability.

Published as a conference paper at ICLR 2025
N-FORGOT: TOWARDS NOT-FORGETTING AND GEN- ERALIZATION OF OPEN TEMPORAL GRAPH LEARNING
Liping Wang's Xujia Li:: Jingshu Peng?, Yue Wang", Chen Zhang', Yan Zhou?, Lei Chen
'Te Hong Kong University of Science and Technology (Guangzhou) The Hong Kong University of Science and Technology Chuanglin Tech
ABSTRACT
Temporal Graph Neural Networks (TGNNs) lay emphasis on capturing node in- teractions over time but often overlook evolution in node classes and dynamic data distributions triggered by the continuous emergence of new class labels, known as the open-set problem. This problem poses challenges for existing TGNNs pre- serving learned classes while rapidly adapting
unseen classes. To address this, this paper identifies two primary factors affecting model performance on the open temporal graph, backed by theoretical guarantee: (1) the forgeiting of prior knowledge and (2) distribution discrepancies between successive tasks. Building theoretical insights,
propose N-ForGOT, which incorporates two plug-in in modules into TGNNs to preserve prior knowledge and enhance model general- izability for new classes simultaneously. The first module preserves previously established inter-class connectivity and decision boundaries during the training of new classes mitigate the forgetting caused by temporal evolutions of class characteristics. The second module introduces an efficient method for measur- ing distribution discrepancies with designed temporal Weiseeiler-Lehman subtree patterns, effectively addressing both structural and temporal shifts while reducing time complexity. Experimental results on four public datasets demonstrate that our method significantly outperforms state-of-the-al approaches in prediction ac- curacy, prevention of forgetting, and generalizability!
INTRODUCTION
Temporal graph neural networks (TGNNs) are increasingly recognized for their ability to en- capsulate temporal interactions among nodes in temporal graphs (Shi al. 2018 Kumar jet al. 2019b; Wang et al. 2023). Despite their substantial contributions, many current TGNNs (Rossi et al., 2020 Cong et al., 2023) assume static data distribution and unchang- ing class sets, neglecting the reality that new categories frequently emerge in real-world sce- narios. For instance, new topies continually emerge into the topic community graph (Feng et al. 2023; Hamilton et al.) (017)) where nodes represent post and are labeled by their topics. This necessitates the development of TGNN models that accommodate open tempo- ral graphs (OTGs), adapting to continuously expanding class sets and evolving interactions. illustrated in Fig. 1, Open Temporal Graph Learning (OTGL) is conceptualized as a series of chronological tasks, each in- troducing previously unseen classes that can lead to shifts existing data distri- bution (Feng al. 2023). However, isting methods treat emerging unseen data potential disruption, focusing solely on preserving previously learned knowl- edge, which undermines the model's per- formance in generalizing both historical and incoming data.
Poediteed
Tisk
Task
Time
Tost Indes
d' c' (a) An illustration of the Open Temporal Graph
across different tasks
Figure 1: Open Temporal Graph Learning (OTGL): class evolution and distribution shifts
"Corresponding Authors: Xujia Li, lcexujia@ust.hk
Coode available at at/ttps:: //g thub. wtt.:/.coo.............. wwwp://iiaaaaaaa/anoooooooo.

---

Published as a conference paper at ICLR 2025
Motivated by the need to balance knowledge preservation from existing data with the swift adapta- tion to incoming unseen classes. We theoretically analyzed the generalization error of the OTGL and investigated two primary factors: forgettngg of prior knowledge and 2) distribution discrepancies between successive tasks.
Forgetting of prior knowledge. Catastrophic forgetting poses a significant challenge in OTGs, which continuously evolve with new elements and structures (Feng et al. 2023). In OTGs, the ir- regularity of node neighborhoods and the ongoing introduction of new edges disrupt existing node correlations, complicating continual learning (Lu et al., 2022; Yuan et al., (023). Recent research has tried to address catastrophic forgetting within graph-structure
data, yet most studies focus on static graph snapshots (Wang et al., 2022a Zhang et al.) (202a),, missing temporal topological in- formation crucial for effectively managing forgetting in OTGs. Some works (Feng et al., (023) aimed to preserve temporal topological information by sampling triad motifs from temporal graphs and replaying them for continual learning. However, the invariant structure of triad motifs struggles to represent class characteristics that dynamically change over time. For instance, the evolution of cell phones in e-commerce networks from single to dual-screen and foldable models illustrates the temporal inconsistencies. This evolution highlights that class characteristics change over time, exac- erbating the preservation of previously learned information and intensifying catastrophic forgetting in OTGs.
Distribution discrepancies between tasks. To achieve broad generalization across both historical and emerging data distributions, is essential for OTGL to adeptly navigate the parameter space by measuring the discrepancy between task distributions. Quantifying task discrepancies in OTGs can arise from multiple domains, such as structure shifts, which arise from the differences in graph struc- tures between old and new data and time shifts, caused by the continuous evolution of the OTG over time (Zhang et al. 2022b). Fig. ((b) highlights high distribution shifts (blue grid) between the graph introduced by different tasks. Prior studies have utilized integral Maximum Mean Discrepancy- based approaches (MMD) (Gretton et al. [0122] Zellinger et al. 2017) to measure structure shifts at the graph representation--evel (Zhu et al.. 2021 Zheng al.] 2024). However, the computational demands of these approaches, which scale at least O(n²) (Bernton et al. 2019; Gretton et al. 2012) relative to the graph size, pose significant challenges for TGNNs. These networks, known for their limited efficiency, heavily rely on batch training to enhance processing (Rossi et al. 2020 Li et al., 2023). Although batch processing facilitates parallel processing within batches, it often neglects the temporal dependencies across different batches (Su et al., 2024), leading to inaccuracies in measur- ing distribution shifts compared methods that assess the entire graph. This oversight leads two issues: measuring discrepancies between entire graphs becomes infeasible, decelerating the training process; batch-wise training can reduce the accuracy of the discrepancy measurements due to the isolated nature of each batch.
Motivated by these challenges in OTGL, we explore the OTGL from a new perspective by going be- yond merely addressing forgetting. We propose novel OTGL approach, Towards Not-Forgetting and Generalization of Open Temporal Graph (N-ForGOT), including two plug-in modules for TGNNs: a temporal interclass connectivity regularization module (TICR) to minimize forgetting of prior knowledge and a localized temporal graph discrepancy optimization module (LTDO) to address task discrepancies. To mitigate catastrophic forgetting, the TICR module renormalizes pa- rameters to align new data with previous tasks, preserving inter-class connectivity and decision boundaries. Through this, TICR maintains essential class features to prevent overfitting. To en- hance the generalizability, the LTDO module offers an efficient strategy for measuring multi-domain distribution shifts using a designed local structure alongside a linear approximation of MMD. We develop a temporal Weisfeiler-Lehmar (WL) subtree pattern, which is capable of capturing struc- tural and temporal information across different batches. This capability can effectively measure distribution shifts during batch training while reducing the information loss associated with batch data isolation. LTDO module significantly reduces the computational complexity from Onn)) to (n) and lowers the storage complexity from (n)) to (()) (as elucidated in Sec. 4.2).
Our main contributions can be summarized as: 1) We extend beyond the conventional focus on mitigating forgetting, advancing towards balance between forgetting and generalizability in the OTGL. To achieve this, we propose two plug-in modules for TGNNs; 2) We introduce the TICR module to mitigate the forgetting of prior knowledge by leveraging the interconnectivity of the graph; 3) We introduce the LTDO module with a novel temporal WL subtree patterns representation. reducing computational complexities while enhancing model generalizability.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
TGNNs. Temporal graphs represent sequences of time-stamped events. According to the sampling scheme for aggregation process, TGNNs can be categorized into two categories: neighborhood- based pesssag--passing TGNNs (MP-TGNNs) (Xu et al. 2020 Rossi et al. 2020; Kumar et al. 2019a Trivedi et al.) 2019), which aggregate the node's 's temporal neighborhoods information, and walk-aggregating TGNNs (Souza et al. 2022 Wang et al. 2021), which encode temporal walks from the target node. Additionally, eeemory-uddatee TGNNs (Rossi et al., 2020; Li & Chen (023) build on the MP-TGNN framework to incorporate memory state vector that stores historical node features, updating embeddings as relevant events occur. However, these models overlook informa- tion loss during updates and struggle with new, unseen node classes.
Graph Continual Learning. OTGL stems from Graph Continual Learning (GCL) but is specifi- cally designed to tackle the unique challenges of temporal graphs. Recent studies in graph continual learning have aimed to address catastrophic forgetting in graph-structured data, which can be cate- gorized into two types: regularization-bane based and replay-based methods. Regulation-baseed methods focus on maintaining old data distributions by selectively constraining updates to parameters that crucial for previous task performances. Liu et al. (2021) evaluated both parameter gradients and topological feature gradients to identify and preserve parameters vital for maintaining key topologi- cal information. Replay-bat -bssed methods (Zhou & Cao 2021 Wang et al. 2022al Feng et al. 2023), stored portions of historical data in memory buffer to help preserve previously learned knowledge They utilized various sampling strategies or data generation techniques (Wang al. 2022a) to re- play historical graph structures, including walk sequences (Wang et al. (022a)), node sets (Zhou) & Cao. 202I), neighborhood sets, and different motif structures (Feng et al., 2023), to maintain learned knowledge. DeLoMe (Niu et al., 2024) and CaT (Liu et al. 2023) employ condensed graphs as replay data. These condensed graphs are sigmificantly smaller than the original, yet they consist of synthetic node representations that preserve the complete informational content of the original structures. Innovatively, OTGNet (Feng et al. 2023), the only work addressing forgetting in OTGs, employed triad sampling to preserve key structural evolution information from historical data. How- ever, the effectiveness of OTGNet is heavily dependent on the volume of replayed data. Insufficient data buffers can lead to suboptimal results, while increased sampling could significantly prolong the overall training time. In contrast to these methods, which view updates from new tasks as poten- tial disruptions to previously learned knowledge, our approach seeks balance between retaining knowledge from past data and effectively adapting to new, evolving data distributions.
3 PRELIMINARY
In this section, we illustrate preliminary definitions of OTG and the discrepancy metric and provide a theoretical analysis of the generalization error bound within the OTGL.
Problem Definition. In the open temporal graph setting (Feng al. 2023). the graph dynamically evolves with the continuous addition of new nodes, leading to the emergence of new classes and, correspondingly, new node classification tasks. Consider a temporal graph G = (V, E), where V is a set of nodes, T C No is a finite set of timestamps, and È comprises temporal edges. These edges are defined as the set (u, v, t) for all node pairs u, v E V and timestamps t E T where an edge exists between nodes u and v at time f. The open temporal graph is conceptualized as a sequence of tasks K. = (K1,,.., K& ). arranged in chronological order, where k represent the current task. The task can be represented as XC; (G,, Y,), containing temporal graph G, and a corresponding class set Y, Each task K.; introduces new classes that were not present earlier tasks, and G, contains data specific to the timeslice of task K.;
Performance Bound Analysis of OTGL. The objective of OTGL is to develop a TGNN, f. pa- rameterized by 0, that incrementally learns across a sequence of is tasks. We decompose the trained TGNN = go h into the feature extractor G + Rdd (Z- = g (G) and discriminator h Bd + Y Y = A())). Node embeddings are produced by (1) = ( x; E V). These embeddings are subsequently processed by the discriminator. To formulate the expected loss of the model f. over the entire distribution of data, termed the population risk, we first defined the distribution for the current task as Gy and the aggregate historical data distribution as C == [- Formally, with a given loss function, ((..)) + R. the population risk of the current task (Wang et al. 2022b) can be defined as R&a = V.
G). v)]. Likewise, the population risk over the distribution
of previous tasks can be defined as RG -
For simplicity, we use to represent the parameters of functions g. and f
3

---

Published as a conference paper at ICLR 2025
To achieve balance between forgetting and generalizability, method aims minimize the population risk for the current task, RG2, while also reducing the generalization error, defined as RR RG We analyze this generalization error based on PAC-Bayesian theory Li & Bilmes (007 and establish a bound for RG.
Theorem (You et al. 2023 Li et al. 2021 Wang er al.) 2022b) Let H. be hypothesis space of SaopikkChhrvonenkeee dimension de with probability least 1- 8 for any ô E (0. For the hypothesis function f. the population risk of the current task is bounded as follows (please see complete proof in Appendix A):
em R.. (Jæ(Ox)) S RGLA-1 (fx(O))) divn (G Ga) + V log
2d
d )
log +
2m
where { = m-n [1f1x-1(x) [x) E (ian---l-i-i-xxx1 fx(x)]] I is the difference in label- ing functions across the new and old tasks, which we expect to be small. div(.) is a distance metric that formulates the old and new task distributions in Reproducing Kernel Hilbert Space (RKHS).
Motivated by Theorem 1, which is extended from the domain adaptation bound analysis (Li (ct al.] 2021), we identify two components that bound the performance of OTGL: 1) The term RG (fx)) indicates the model's performance on historical data distribution G1æ-11 which determined by the model's ability in managing forgetting; 2) die GG.:-1,, G&) represents the discrepancy between task distributions, which can be quantified using the MMD. Accordingly, this
work target on minimizing RG (fx(0))) and the divergence diex div (G Gy), leading to the corresponding proposal of two specific plug-in modules: TICR and LTDO.
Specifically, the first term, RG ())) is an empirical measure that quantifies the risk associ- ated with the model f over the training data from previous tasks. This measure reflects how well the model performs on data from previous tasks by calculating the average loss across these historical data distributions. It indicates the model's performance on historical data G given that it was trained using the current data G.. Therefore, the value of the first term, Rög, (S& (0x)), reflecting the model's capacity to mitigate catastofic forgetting. Successfully evaluating this component hinges on deep understanding of how historical data influences and constrains the parameter space. The second term divz (G1-A-1. 1, G&) quantifies the discrepancy between the distributions G1 and Gr. which can be formulated by the MMD metric (detailed definition can found in Appendix C). MMD)
can be expressed as MMD (G1*-11 G2) = suppeølic Eg [x<x)] Eynce (vb()))))))))l)))))) (v)]
where ya is a kernel function that maps data from the two distributions RKHS.
4 METHOLODOGY
MMD (
D
chosantiot Ionomntiiialll crdes
Features of Structure and Temporal Suberee Putterss
Temporal (pex batch)
LTDO module
G
G
Time
TGNN f(e) G.)
TGNN f(0 Gz)
TGNN f(0i, Gy)
TICR module
31 (()
22 100 62)
92
y
Pi
Protutypes
Class
P2
P5
Ps
(dPPPP) [d(r
Leasstifiattin
L
Lic
Class Relational Marrices
Figure 2: Overview of N-ForGOT.
The overall architecture of N-ForGOT is illustrated in Figure 2, including two plug-in modules: the Temporal Inter-class Connectivity Regularization (TICR) module (Sec. 4.1), highlighted in red, is designed to mitigate catastrophic forgetting; and the Localized Temporal Graph Discrepancy Opti- mization (LTDO) module (Sec. 4.2), shown in green, focuses on optimizing distribution discrepan- cies across successive tasks.
4

---

Published as a conference paper at ICLR 2025
4.1 TEMPORAL INTERCLASS CONNECTIVITY REGULARIZATION (TICR) MODULE
As the set of classes within the graph expands, performance previous classes often deteriorates due to the forgetting of established class prototype knowledge. As introduced before, the temporal inconsistencies of class characteristics take more challenge to address the forgetting. To counteract this, we preserve knowledge through two key components: decision boundaries and feature-based inter-class connectivity among different classes. Decision boundaries and inter-class connectivity typically represent consistent knowledge. For example, despite the characteristies of the class 'cell phone' evolving with technological development, the difference between it and 'carphones' remains obvious and consistent over time. The model learns task-specific parameters according to these two components and renormalizes current ones align with those from previous tasks.
Decision Boundaries. To preserve tpek-spe decision boundaries, we ensure that the network retains the essential properties of each class and contrasts features from the past with the current. Inspired by the modified cross-entropy loss function (Hinton et al. [015], we apply logistic outputs from the old network serving as soft targets in the loss function for approximating the predicted class probabilities of the current network.
[V6
log ( (o E
(DB(Sk,
E
E
where 9% = ft- G&) and En = fu(,,, G5.) represent the logits from both previous and cur- rent parameter configurations the class set Y, in the ongoing task. The loss function incorporates temporal scaling function adjust the logits, enhancing the model's discriminatory capabilities across temporal transitions, where 7 the temperature factor. The temperature parameter is set as amplify the influence of smaller logit values. This adjustment accounts for potential label imbalance by giving more weight to less frequent categories.
Inter-class Connectivity. To capitalize on interclass connectivity, assess the distances between class relational matrices from networks trained on previous and current tasks. Prototypes for the class are denoted by p = Ejec, ((z)), where C, denotes the node set related to class i. After fully training the current task, prototypes for the newly involved classes are updated in the class prototype matrix, PA Pk Pkk .,.......,,
....., Pk [N-)] Pk (8) - u), p.d, Vn Y 1ikk where
No Yi:kl denotes the number of classes in the class set, encompassing all previous tasks, and v(i is an indicator function used to represent whether a category appears in a task v. This formulation establishes dynamic and evolving class prototype matrix incorporating historical data to preserve and utilize the interclass relationships effectively as the class set expands.
Interclass connectivity of previous and current tasks be formulated by class relational matrices, which are defined as:
(1) R d(p
10) [ R'
1) [pp [ (1))
i.j=1
R and R' are the class relational matrices of the current and previous networks respectively, where .(...) denotes the Fuclidean distance to measure the distance between class prototypes. Each matrix with dimensions X where = Y& represents the number of classes in the current task. P obtained from the class prototype matrix which represents the ground truth features of previous classes retained after fully training each previous task. p and p' represent the class prototypes of new classes obtained from the current task and previous task networks, respectively. The class prototype p' of class derived from the previous network is defined as:
(i)
p p
11 Σ 9k-1 (k-1(0e-2 IC, jEC,~Gx
(3)
where C, represents the set of nodes in class within the graph of the current task. Similarly, the prototype set P for the current network is computed using the current model parameters.
We define the interconnectivity class loss to align the class relational matrices between previous and current tasks:
Li = R R'
||
(4)
This loss function minimizes the distance between class prototypes across models, ensuring consis- tent class understanding and reducing catastrophic forgetting of previous classes.
5

---

Published as a conference paper at ICLR 2025
4.2 LOCALIZED TEMPORAL GRAPH DISCREPANCY OPTIMIZATION (LTDO) MODULE
This module addresses multi domain distribution shifts and efficiency challenges in measuring dis- crepancies between task distributions. The LTDO module quantifies both structural and temporal shifts within open temporal graphs, identifying task variations based on the designed local struc- ture. Leveraging the topology of this local structure, it enables efficient batch-wise task discrepancy measurement, while also minimizing information loss typically associated with isolated batches.
Temporal WL Subtree Patterns. Inspired by the capability of the WL subtree in addressing graph isomorphism challenges (Leman & Weisfeiler 1968). We introduce the temporal WL subtree with structure and temporal patterns as the discriminative local structure, which makes particularly suited for capturing nuanced structural differences in dynamic graph environments.
Definition 1 Temporal WL Subtree Patterns. For temporal graph, G = (V,T.E). The neighbor- hood of a node v E V is denoted as (v)) = {u E V (,,, & E). For node v, its temporal subtree pattern 54w)) = (Sy, St) with depth d contains structure and temporal parts. The temporal WL subtree can be defined recursively in the following way:
(v)) : 5!u=0) (v) =(v}, S(d20) (v) = ---1((0) U {Wd, (N(('))) l E S
S,(0): (( (t'), s( (w) (c) U qit' - t| (u', u,t) E u. E S $ where S denote a set of the leaf nodes in S and tmaz is the maximum observed timestamp of the target node v.
We further develop the representation of temporal WL subtree patterns for MMD measurement by aligning the TGNN aggregation process with the WL algorithm (Leman Weisfeiler 1968). As delineated in Definition 1, the initial pattern (v) represents the categorical feature for the target node. Subsequent patterns, from S (v) to the higher order, encapsulate the topological structure surrounding the target node, along with evolving temporal information. The subtree structure is defined recursively via node neighhorhood, allowing us to utilize neighborhood--ase message- passing TGNNs to capture the features of temporal subtree patterns. We proposed the following definition to formulate the structural and temporal features of our temporal WL subtree patterns.
Definition 2 With the feature extractor g function & T + R", The depth of our temporal WL subtree refers to the number of TGNN layers. Considering sufficient number of TGNN layers, the structure and temporal features for the temporal WL subtree patterns can be defined as (theoretical proof is provided in Appendix B):
Rd in the TGNN, which contains a time encoding
a) Structure feature for t pattern:
Co.. (u) = g(d) (v) = AGG ((()t-)(()))) (u, E (
(5)
where AGG is layer-specific aggregation functiioon If d = 0, C (s) employs the raw feature of the target node z...
b) Temporal feature feature for th pattern:
Cd (v) = ((AI), where A = 5p()) Li, 4 4 (t))
(6)
Definition 2 is based on the theorem that message-passing TGNNs with d layers yield equivalent outputs for nodes that share the same temporal WL subtree pattern of depth d. when the underlying structures are non-isomorphic. This validates the compatibility of computing features for the tem- poral WL subtree patterns across various TGNN architectures. It also demonstrates that our LTDO module can effectively function as a plug-in module within TGNNs. For instance, the time en- coder function in TGNNs enables the computation of temporal features in our temporal WL subtree. TGAT (Xu et al. 2020) employs self-attention mechanism for message passing and incorporates a trainable time-encoding function ((A)) = cos(Iw- + b). Similarly, the GraphMixer (Cong et al., 2023), another variant of TGNN, utilizes a multi-layer perceptrons- based aggregation function com- bined with a simpler, untrained time encoder, Φb At) = (tw).
Batch-wise Linear MMD Approximation. Recall that the primary goal of this module is to cul- tivate task-invariant representations by minimizing the distribution discrepancies across all tasks. MMD as a powerful non-paramttric metric, effectively compares distributions based two data sets. In many practical scenarios, the distribution shift within a graph is traditionally measured using the classical MMD approximation. MMD? (details in Appendix C) across the entire graph (Gret- ton al. 2012 Zellinger al. 2017). However, MMD; involves pairwise similarity calculations,

---

Published as a conference paper at ICLR 2025
leading to quadratic time complexity. And it is unsuitable for batch-wise training due to the incon- sistency of batch data sampling (Yan et al. 2017).
In contrast, our approach leverages the localized structure of the temporal WL subtree. This al- lows us to implement an unbiased approximation of MMD that operates with linear computational complexity (Gretton et al. 2012). Additionally, our temporal WL subtree can capture the structure information from other batches of data with its root patterns, allowing us to measure distribution shifts in batch training required by TGNNs, significantly reducing the information loss typically associated with batch data isolation (refer to the experiment in Sec. 5.4).
Definition 3 Linear MMD Approximation. This linear approximation assumes that the datasets being compared contain an equal number of instances m. The estimator is an unbiased estimate of MMD² Fmmdd P. Q, where p and q denote targets and source data distribution.
m
|M mend, Gk-1, GR]
Smmdd ((x2))) 1: x2
k-1)
)), (x 155--110 2)
1=1
R-1 )₁ (221- x) fmmd (x 222
(x 1:
k-1
x + 2/s (x 2i
-x k- 11 . -12
The LTDO module addresses the challenge of measuring distribution shifts between adjacent tasks by focusing on local structures within each batch. By comparing batch data from successive tasks, we ensure that the data volumes align with the assumptions required for linear MMD computation. Additionally, batches are organized in chronological order to define the time domains, enabling comparisons within the same temporal contexts, such as comparing data from spring to spring, to enhance the accuracy of our discrepancy measurements. The measurement of distribution discrep-
ancies between successive tasks is defined as follows.
[8]
diu(Gk-1,, G1)
div Bi Bi); dio(B; (Bj BB)
Σ MMD: (B" (B,))
B
(7)
Here, [B] denotes the number of batches in the current task, Bi represents the 7th batch in task k. and [B] denotes the size of batch representing instances m in the MMD/. The function C' (B₁) (o)))C E Bj] represents the features of the temporal WL subtree patterns for batch data, where features of structural and temporal patterns are concatenated. The discrepancy loss function
is defined as follows.
LDIV div(GG div(Gk=11,L.)
(8)
Time Complexity and Effectiveness Analysis. MMD; requires only ((1) memory, and (()) time complexity to compute the kernel on all interacting pairs. Conversely, the classical MMD approx- imation, MMD2, demands (n)) memory and O(n²) computational time, making it less efficient (details in Appendix C). Given the topology of the temporal WL subtree, the time complexity of the LTDO module is ((V) ())) in the worst case, where the batch size equals the size of the input graph. |S)] denotes the average size of the subtree, depending on the neighborhood size and depth. Since |$| is small constant, the LTDO module reduces the computational complexity of discrepancy measurement from ((V)2) to a linear time complexity ((V)).
According to Hoeffding (1963), the deviation bound of MMD; is PIG.-I,GG (MMD)(G&-1. Gx)
assuming that (0 S v)(1,,z)) S K). This states the probability that the empirical MMD deviates from the squared population MMD by more than t, where is a predefined threshold. MMD? N. has the same Hoeffding's bound as MMD Ad- dtionally, the asymptotic variance for MMD is given by Var,, (fmma(2, z²)] when MMD} con- verges in distribution to a Gaussian. The asymptotic variance for MMD?, i is the variance of Ez[ famdd2z, =²)] (Gretton et al. 2012) in the same converges scenario.
-MMD² GG) > 1 < exp
4K*
The analysis demonstrates that the LTDO module significantly reduces computational complexity while maintaining comparably high accuracy relative to classical quadratic time MMD estimators. Further empirical validation of these theoretical advantages is detailed in Section 5.4.
4.3 N-FORGOT
The overall optimization objective of the whole framework N-FORGOT is designed as follows:
L Ler. aLLDB +Ic) + BLDrV
7

---

Published as a conference paper at ICLR 2025
Here the classification loss, LCL. represents the categorical cross-entropy loss. The hyperparameters a and B can adjust the balance between preventing forgetting and generalization. We provide the pseudo-code of N-ForGOT in Appendix D.
5 EXPERIMENTS
In this section, assess N-ForGOT in terms of its overall effectiveness, its capability to manage forgetting, and its ability to generalize unseen data distribution.
5.1 EXPERIMENTAL SETUP
Table 1: Dataset Statistic
Yelp
Reddit Taobao Amazon
Datasets. We assessed N-ForGOT
# Nodes
20,992
10,845 114,232 $61,965
using four real-world datasets:
# Edges
234,459 216,397 455,662 4,172,700
Yelp (Sankar et al.] 2020). Red-
# Tasks
6
3
dit (Baumgartner et al. 2020p.
# Timespan / task 1 year
1 month 2 days 3 months 30
Taobao (Du al., (2019), and Ama- zon (Hou et al.) (0024). We summarize # Classes task
2
the statistics datasets in Table 1. We adopt the Reddit and Taobao datasets from previous OTGL work (Feng al. 2023) and follow their conventions for dataset construction in the other two datasets. More information about the datasets can be found in Appendix E.I.
Evaluate Metric. For each task, nodes for testing. To comprehensively evaluate our method, we use three widely-used metrics (Wang al. 2024) in continual learning to evaluate method: Average Performance (AP), Average Forgetting (AF), and Backward Transfer (BWT). AP measures the mean accuracy of the model across all tasks, calculated as AP K Σx= ai where a; denotes the ACC (Accuracy) on the 7th task, K is the total number of tasks. AF quantifies the loss of previously acquired knowledge, defined by the difference between the maximum historical performance of a task and its current K-1
80% nodes for training, 10% nodes for validation, and 10%
performance: AF = K-1 LØ=1 (max;e (I...,K (I..., -1 (ax.) (Ix.xx, - ds,z), Vj I K. BWT reflects the average influence of learning a new K-th task on the performance of all previously learned tasks.
It is defined as BWT K-1
1 E (ax, - aj.j). This metric evaluates how the introduction of new tasks can positively impact the performance of previous tasks. These three metrics collectively provide a comprehensive evaluation of our method in not only preventing catastrophic forgetting but also enhancing generalizability across tasks.
Baseline Methods. We benchmark our approach against Joint training approach, Finetune training approach, and six models across both computer vision (EWC (Kirkpatrick et al. (016), and LwF (Li & Hoiem (016)) and graph-structured data domains (TWP (Liu et al. (021)) and ER-GNN (Zhou & Cao. 2021). OTGNet (Feng et al. 2023)). Joint training, which trains the model simultaneously on data from all tasks, does not follow the OTGL setting. This approach provides a useful upper bound of AP. Finetune training provides lower bound of forgetting. Details about the baseline and experiment setting can be found in Appendix E.2.
5.2 PERFORMANCE COMPARISON
Performance Comparison with Baselines. Table summarizes the overall performance from ten runs. Our method closely matches the AP performance of joint training and significantly outper- forms other baselines. Specifically, N-ForGOT achieves minimum of 3.07% and up to 18.30% higher AP compared to other baselines. Furthermore, the results for metrics AF and BWT demon- strate that N-ForGOT minimizes forgetting and improves performance on previous tasks during the continual learning of task sequences, thereby demonstrating effective generalization. Key observa- tions include:
Managing Catastrophic Forgetting: Our method has demonstrated exceptional AF performance across all task sequences, both long and short, achieving up to 99%73% and at least 14.34% reductions in forgetting rates compared to the closest baseline. The largest gap in AF results between N- ForGOT and the baselines occurs in the Reddit datasets. In the experiments conducted on the Reddit dataset, methods that preserve knowledge by replaying data from previous tasks in the current task, such as ER- RN-GNN (Zhou & Cao 2021 and OTGNet (Feng et al., (023), demonstrate poorer ability in managing catastrophic forgetting compared methods based on parameter regulation. This may be due to the varied data distributions, which make challenging for these methods to sample important structures that represent temporal topological information effectively for replay.
8

---

Published as a conference paper at ICLR 2025
Generalizability: Higher BWT values represent positive impacts the performance of previous tasks as new tasks are learned. Our method consistently demonstrates higher BWT values, partic- ularly in datasets with long task sequences. This positive impact suggests our method can swiftly adapt to unseen data distributions introduced by incoming tasks. The detailed analysis of the BWT results across continually learned task sequences is introduced in Sec 5.3.
Module Ablation. N-ForGOT with only the TICR module (refers to N-ForGOT w/o LTDO in Ta- ble 2) achieves the lowest AF, indicating its ability to manage forgetting issues. N-ForGOT equipped solely with the LTDO module (refers to N-ForGOT w/o TICR in Table 2) shows better performance compared to when using both modules, but incurs greater forgetting. This phenomenon is likely due to the LTDO module performing the higher performance peak of target tasks during sequential continual learning, as AF is measured by max performance. The module ablation study demonstrates that the TICR module effectively addresses forgetting, and the LTDO module enhances average per- formance across all tasks. Our method combines the two modules for dynamic adaptation to new tasks while preserving previous learning. An extensive analysis of two plug-in modules is conducted through a hyperparameter ablation study, which assesses the roles of hyperparameters a and / as outlined in Equation 9, with details provided in Appendix E.5. The results demonstrate the cru- cial role of both plug-in in modules in balancing the retention of previous data distributions with the adaptation to incoming data.
Table 2: Comparisons (%) analysis and module ablation study of N-ForGOT with baselines.
Method
Yelp
Reddit
TaoBao
Amazon
(1)
AF())
BWT (1)
AP(1)
(1) BWT(1) (P
AF BT AP
(F)) BWT
Juint
70.855
76.455
87,910
83.753
(#1.03)
(#3.11)
(#183
(20.95)
Finetune
61.505
-27.32
-27.322
46.506
-13.050
-12,929
56,050
-44.158
-44.158
69.853
-3.5332
-34.532
(#1.97)
(#3.07)
(#11))
(#5.27)
(#2.62)
(#2.11)
(#1.67)
(#1.31
EWC
52.425
-16.887
-16.445
45.983
220.794
-19.620
70,068
-23,104
-23.103
51.983
-30,362
-30.362
(+2.74)
(+-4.01)
(+4.799
(+-448)
(+1.18)
(+3.33)
(+1148)
(22)
LwF
$4,482
-23.462
-23.461
51 51,626
-18.047
-18047
68.108
-22.294
-22.294
61.875
-25 820
-25.820
(+1.45)
(+3.677
(+3.200)
(+:3.08)
(+2.330)
(+0.77)
(+0.95)
(+3.85)
TWP
60.537
-15.616
-15.185
52.985
-9.973
-9.834
80,895
-19.114
-19.114
68.785
-25.835
-25.651
(115001)
(1:2.27)
(11.24)
(14.18)
(12.09)
(.12.69)
(133005)
(11.80)
ER-GNN
63.713
-14.958
-14.724
43.756
-30.252
-30,252
72.159
-14.104
-14.104
73.941
-27.071
-27.070
(1.3224)
(1.3.55)
(144.45)
(1.2.9) 96)
(11199)
(..2.29)
(11.95)
(.1.40)
OTGNee
61,947
-4,912
-3.156
44,930
-35,540
-35,540
77.853
-28.926
-28.926
77.535
-18.055
-17.623
(.22.5)
(12..77)
(14.98)
(1.520)
(111.07)
(.11.11)
(12.01)
(10077)
N-ForGOT
69.356
-2.829
-2,050
62.681
-1.024
-0.889
83,376
-6,529
-6,529
80.300
-15.500
-12.965
(#1.07)
(=1.72)
(d:3.85
( 23)
(#:1.55
(=1.36)
(#1 98)
[=1.54])
(wlo LTDO)
65.342
-1.932
-1.932
59.710
-1.021
-1.021
76,850
-5.364
-5.364
76.924
-12.318
-12.318
(#2.63)
(=0.90)
(=100))
( 77)
(#2.68)
(=00.17)
(#2 03))
[ 62)
(wr TICR)
70.268
-6.765
-3.975
64.721
-6.014
-1,980
78.074
-11.093
-11.093
89. 39.621
-21.524
-21.524
(20.88)
(=0.45)
(#3.01)
(1:2 60)
(#1165)
(=1.73)
(=11008
(=2.20)
5.3 PERFORMANCE ANALYSIS ACROSS TASKS
We further analyze the effectiveness of our model in mitigating catastrophic forgetting and enhanc- ing generalization ability with the increased tasks.
Performance Trends Across Tasks.
N-ForGoT
Fineture
EWC
LuF
ERGIN
TWP + OTGNNE
We plot the trends of AP values in Fig-
70
ure 3. illustrating the performance tra-
65
jectory throughout the continual learn-
60
ing process. The performance curve of our method, represented by the red line, demonstrates increased perfor- mance compared to those of other base- lines. This indicates that N-ForGOT not only addresses the forgetting issue but also improves task performance by enhancing generalizability across data distributions through task discrepancy optimization. Specifically, the increased performance observed upon the introduction of Task and Task 4 suggests that our method effectively adapts to newly arriving tasks.
55
50
Task learning sequence
Figure 3: Performance trends throughout the continual learning process.
BWT Trends Across Tasks. We analyze performance changes for the target task with new tasks continually introduced, as shown in Figure 5. The x-axis represents the 'Target Task', and the y- axis shows the new tasks introduced continually. This heatmap effectively highlights how each new
9

---

Published as a conference paper at ICLR 2025
(Continual Trained)
N-ForGOT
Finetune
EWC
LwF
TWP
ERGNN
OTGNet
5.0
Task Index
En Ph @ tJ =
-10.0
-20.0
-30.0
Task Index (Target Task)
Figure 4: BWT trends across tasks. Blue cells indicate performance improvements in the target task during the continual learning of task sequences. Conversely, cells in red denote performance drop.
task impacts the performance of previously learned tasks. Each cell in the heatmap indicates the impact on the performance of the target task following the introduction of new tasks, as measured by the BWT metrics. Cells colored blue signify improved performance, indicating that new tasks have positively influenced the target task's performance. Conversely, cells in red denote significant forgetting. Notably, for Task O, our method demonstrates significant performance improvement after successively learning the next five tasks (Task 1-5). Our method not only minimizes forgetting but also occasionally shows positive backward transfer, where the introduction of new tasks enhances the performance of earlier tasks.
5.4 DISCREPANCY MEASUREMENT STRATEGY ANALYSIS
We conduct further analysis of our proposed discrepancy measurement strategy, the LTDO module, focusing on both its efficiency and effectiveness.
Efficiency Analysis. As shown in Fig- ure 5 (left), we compare the efficiency of our discrepancy measurement method (LTDO module) with the classical MMD measurement. For fair comparison, we use single Gaussian kernel as the ker- nel mapping function for both methods due to the classical MMD's inability to support multiple kernels within our device's memory limitations. Further details on kernel design in the main experiment are available in Appendix E.2. We observe that the LTDO module becomes significantly more efficient than classical MMD as the size of the graph increases. This comparison demonstrates the efficiency gains achieved through our proposed discrepancy measurement method in graph data scenarios.
100
time
100
10
10-1
Nodes
Velp
Reddit
Taobao
Figure 5: Discrepancy measurement strategy analysis:
efficiency analysis (left), performance analysis (right),
Performance Analysis. We analyze the effectiveness of our temporal WL subtree and explore the performance differences between classical and linear MMD approaches, as illustrated in Figure (right). The term 'w/o Subtree' denotes that our method measures the discrepancy between tasks directly using node features, without involving subtree structures, Excluding temporal WL sub- tree patterns significantly reduces performance, demonstrating their effectiveness in capturing both temporal and structure distribution shifts and mitigating accuracy loss due to batch data isolation. Additionally, the term 'Classical MMD' refers to our experiment using classical MMD as opposed to linear MMD for discrepancy measurement, Despite theoretical analyses suggesting that classi- cal MMD offers greater stability (as shown in Sec.4.2), our experimental results indicate that our method does not experience a performance drop when enhancing efficiency. We also conduct sensitivity analysis of hyperparameters which can be found in Appendix E.4.
6 CONCLUSION
This study introduces N-ForGOT, a novel approach to Open Temporal Graph Learning (OTGL) that tackles both forgetting and distribution shifts. To our knowledge, this is the first work to ex- tend forgetting mitigation in OTGL by incorporating a theoretical analysis. Our method integrates two plug-in modules for Temporal Graph Neural Networks (TGNNs): one that preserves previ- ously learned knowledge through class decision boundaries and inter-class connections, and another that ensures broad generalization across both historical and emerging data distributions, utilizing an effective linear Maximum Mean Discrepancy approximation with a novel temporal Weisfeiler- Lehman subtree representation for measuring distribution shifts. Experiments on four real-world datasets demonstrate that N-ForGOT not only minimizes forgetting but also enhances performance on previous tasks, showcasing its effectiveness and generalizability. Future work will focus ex- tending our framework to handle unseen graph data from diverse domains and developing a model capable of generating varied graph structures.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
Lei Chen's work is partially supported by National Key Research and Development Program of China Grant No. 2023YFF0725100, National Science Foundation of China (NSFC) under Grant No. U22B2060, Guangdong- Kong Technology Innovation Joint Funding Scheme Project No. 2024A0505040012, the Hong Kong RGC GRF Project 16213620, RIF Project R6020-19, AOE Project AoF/E-603/18, Theme-based project TRS -60-2R0/ CRF Project C2004-21G, Guang- dong Province Science and Technology Plan Project A05050000) Guangzhou municipal- ity big data intelligence key lab, 2023A03J0012, Hong Kong ITC ITF grants MHX/078/21 and PRP/004/22FX, Zhujiang scholar program 202 IJCO2X Microsoft Research Asia Collaborative Research Grant, HKUST- joint research lab and 2023 HKUST Shenzhen-Howe Kong Col- laborative Innovation Institute Green Sustainability Special Fund, from Shui On Xintiandi and the InnoSpace GBA.
REFERENCES
Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. The pushshift reddit dataset, 2020.
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wort- man Vaughan. A theory of learning from different domains. Machine learning, 79(1-)):5111775. 2010.
Espen Bernton, Pierre E. Jacob, Mathieu Gerber, and Christian P. Robert. Approximate bayesian computation with the wasserstein distance. Journal of the Royal Statistical Society Series B: Statistical Methodology, 31(2):355-66) February 2019. ISSN 1467-9868. doi: 10.1111/rssb. 12312.
Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi. Do we really need complicated model architectures for temporal networks? In The Eleventh International Conference on Learning Representations, ICLR 2023, 2023.
Zhengxiao Du, Xiaowei Wang. Hongxia Yang, Jingren Zhou, and Jie Tang. Sequential scenario- specific meta learner for online recommendation. In Proceedings of the 25th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 2895-2904, 2019.
Kaituo Feng, Changsheng Li, Xiaolu Zhang, and Jun Zhou. Towards open temporal graph neural networks. In The Eleventh International Conference Learning Representations, ICLR, 2023,
Arthur Gretton, Karsten M Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs
Advances in neural information processing systems, 30, 2017.
Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in neural network. CORR, 2015.
Wassily Hoeffding. Probability inequalities for sums bounded random variables. Journal of the American Statistical Association, 58(301):13-33) 1963.
Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. Bridging language and items for retrieval and recommendation, 2024.
James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, An- drei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic for- getting in neural networks. CoRR, 2016.
Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in tem- poral interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, KDD, pp. 1269-1278. ACM, 2019a.

---

Published as a conference paper at ICLR 2025
Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in tem- poral interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, pp. 1269-1278, 2019b.
Andrei Leman and Boris Weisfeiler. A reduction of graph to canonical form and algebra arising during this reduction. Neuchno-Technicheskay Informatsiya, 2(9): 1968.
Bo Li, Yezhen Wang, Shanghang Zhang, Dongsheng Li, Kurt Keutzer, Trevor Darrell, and Han Zhao. Learning invariant representations and risks for semi-supervised domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 1104-1113, Computer Vision Foundation IEEE, 2021.
Haoyang Li and Lei Chen, EARLY: efficient and reliable graph neural network for dynamic graphs. 2023. doi: 10. 1455358993888
Proc. ACM Manag. Data,
Xiao Li and Jeff Bilmes. A bayesian divergence prior for classiffier adaptation. In Artificial Intelli- gence and Statistics, pp. 275-282. PMLR, 2007.
Yiming Li, Yanyan Shen, Lei Chen, and Mingxuan Yuan. Orca: Scalable temporal graph neural network training with theoretical guarantees. Pruc. ACM Manag, Data, 1(1):52:1-52:27, 2023. doi: 14/35487788
Zhizhong Li and Derek Hoiem. Learning without forgetting, In Computer Vision ECCV, volume 9908 of Lecture Notes in Computer Science, pp. 614 629, 2016.
Huihui Liu, Yiding Yang, and Xinchao Wang. Overcoming catastrophic forgetting in graph neural networks. In Thirty-Fifth Fifth AAAI Conference on Artificial Intelligence, pp. 8653-8661. AAAI Press, 2021.
Yilun Liu, Ruihong Qiu, and Zi Huang. Cat: Balanced continual graph learning with graph conden- sation. In 2023 IEEE International Conference on Data Mining (ICDM), pp. 1157-1162. IEEE, 2023.
Bin Lu, Xiaoying Gan, Lina Yang. Weinan Zhang, Luoyi Fu, and Xinbing Wang. Geometer: Graph few-shot class-incremental learning via prototype representation. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pp. 1152- 1161, 2022.
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2018.
Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe Weisfeiler and leman go neural: Higher-order graph neural networks. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019. The Thirty-First In- novative Applications of Artificial Intelligence Conference, IAAI, pp. 4602-4609. AAAI Press, 2019.
Chaoxi Niu, Guansong Pang, and Ling Chen. Graph continual learning with debiased lossless memory replay. arXiv preprint arXiv:2404.1000 10984, 2024.
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard. Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs. arXiv preprint N0f.:22006.06637, 2020.
Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. Dysat: Deep neural repre- sentation learning on dynamic graphs via self-attentio networks. In WSDM, pp. 519-527. ACM, 2020.
Chuan Shi, Binbin Hu, Wayne Xin Zhao, and S Yu Philip. Heterogeneous information network embedding for recommendation. IEEE Transactions on Knowledge and Data Engineering, 31 (2):357-370, 2018
Amauri H. Souza, Diego Mesquita, Samuel Kaski, and Vikas K. Garg. Provably expressive temporal graph networks. In Advances in Neural Information Processing Systems 35: Annual Conference Newral Information Processing Systems 2022, NeurIPS, 2022.
12

---

Published as a conference paper at ICLR 2025
Junwei Su, Difan Zou, and Chuan Wu. PRES: Toward scalable memory- based dynamic graph neural networks. In The Twelfth International Conference on Learning Representations, ICLR, 2024.
Rakshit Trivedi, Mehrdad Farajtahar, Prasenjeet Biswal, and Hongyuan Zha. Dyrep: Learning rep- resentations over dynamic graphs. In 7th International Conference on Learning Representations, ICLR, 2019.
Junshan Wang, Wenhao Zhu, Guojie Song, and Liang Wang. Streaming graph neural networks with generative replay. In Aidong Zhang and Huzefa Rangwala (eds.), KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1878-1888 ACM, 2022a.
Liping Wang, Jiawei Li, Lifan Zhao, Zhizhuo Kou, Xiaohan Wang, Xinyi Zhu, Hao Wang, Yanyan Shen, and Lei Chen. Methods for acquiring and incorporating knowledge into stock price predic- tion: A survey. arXiv preprint arXiv:2308.cm 04947, 2023.
Liyun Wang, Xingxing Zhang, Qian Li, Jun Zhu, and Yi Zhong. Coscl: Cooperation of small continual learners is stronger than a big one. In Computer Vision ECCV, volume 13686 of Lecture Notes in Computer Science, pp. 254-271, 2022b.
Liyuan Wang. Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. IEEE Trans. Pattern Anal. Mach. Intell., 46(8):5362- 5383, 2024.
Yanbang Wang, Yun-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks. In 9th International Conference on Learning Representations, ICLR. OpenReview.net, 2021.
Da Xu, Chuanwei Ruan, Evren Körpeoglu, Sushant Kumar, and Kannan Achan. Inductive represen- tation learning on temporal graphs. In 8th International Conference on Learning Representations, ICLR, 2020.
Hongliang Yan, Yukang Ding, Peihua Li, Qilong Wang. Yong Xu, and Wangmeng Zuo. Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 945-954 IEEE Computer Society, 2017.
Yuning You, Tianlong Chen, Zhangyang Wang, and Yang Shen. Graph domain adaptation via theory-grounded spectral regularization. In The Eleventh International Conference on Learning Representations, ICLR, 2023.
Qiao Yuan, She Guan, Pin Ni, Tianlun Luo, Ka Lok Man, Prudence Wong, and Victor Chang. Continual graph learning: A survey. arXiv preprint arXiv:2301. 12230, 2023.
Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger-Plate Central moment discrepancy (CMD) for domain-invariant representation learn- ing. In 5th International Conference on Learning Representations, ICLR, 2017.
Xikun Zhang, Dongjin Song, and Dacheng Tao. Cglb: Benchmark tasks for continual graph learn- ing. Advances in Neural Information Processing Systems, 35:13006-1302, 2022a.
Zeyang Zhang, Xin Wang, Ziwei Zhang, Haoyang Li, Zhou Qin, and Wenwu Zhu. Dynamic graph neural networks under apatio-temmoot distribution shift. Advances in neural information pro- cessing systems, 35:6074-6089, 2022b.
Xin Zheng, Dongjin Song, Qingsong Wen, Bo Du, and Shirui Pan. Online gnn evaluation under test-time graph distribution shifts. arXiv preprint arXiv:4403.09953, 2024.
Fan Zhou and Chengtai Cao. Overcoming catastrophic forgetting in graph neural networks with experience replay. In Thirty-Fifth AAAI Conference Artificial Intelligence, AAAI 2021, Thirty- Third Conference on Innovative Applications of Artificial Intelligence, IAAI, pp. 4722 2021.
Qi Zhu, Natalia Ponomareva, Jiawei Han, and Bryan Perozzi. Shift-robust gnns: Overcoming the limitations of localized graph training data. In Advances in Neural Information Process- ing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS, pp. 27965-27977, 2021.
13

---

Published as a conference paper at ICLR 2025
A PROOF OF THE THEOREM 1
Based on the preliminary definition provided in Sec 3.1, we introduce the detailed definition of the population risk in this case.
Definition 4 For hypothesis function f, the probability according to the data distribution that hypothesis disagrees with the labeling function f. is defined as:
Rc(,,)) -Eg=ell(=)) (
use the shorthund Re( = Rs(s, to refer to the risk of a hypothexis
Our method aims to minimize the population risk of the current task, Rx(O) and reduce the difference between the current and previous task risks, (Rx(f) R1:x-1 Theorem is derived by first assessing (fx(f) R1*x-(f)]- We have:
RR((f) R1:k-1(f)
(10)
= - - -111-1 ( - R fw)l
Based on the triangle inequality (Def A.1 in (Mohri et al. 2018 for classification error which implies that for any labeling functions f1, f2 and f3. we have R(f1 f2) I R(f2, fa) + R(f2.fs) Thus, we can reorganize Equation 11 as follows:
R&(J) S R(x-1( ( (f,f{)) 1
+R((fffa Rx-1(f.
S R(s-1(f) 1 E,t- - (z)]]
(R fa)) R1 fe)l (Definition
S RIN- 1 (f) + E 11f - (=)]]
diow (G1 G+) (Lemma
$ RIck-1 (f) + divn (1 Gx) + 11a--1lfffi----- (x(
2d
em
log (1)
log
d
+
|
2m
(Lemma 2)
We could alternatively add R1(f. fk) instead of Rk(f. f1) in the first line. This would yield the same bound but with the expectation taken over G1 instead of Gk. The smaller bound holds, leading to:
(x( -Ra-1 1f) + di (GI:--1, G&)
2d
em
+
log(
+
log
+
d
2m
where& Ç = min [EA [lffi-1(232 fu (x)l] E1 (lfi - fx(x
Lemma For the definition of hypothesis space H and the symmetric difference hypothesis space HAH (Ben-David et al. 2010).
RR( fa) Ra-1(fifell
Exllf(( f(x))]l Er f(x))
< sup [x(f(x)) - [f()=
< 1/2 divn (G12-11.))
'Here, Re, (f) is equivalent to Re, as defined in Section 3. *For simplicity, we R2 represent Re, in the following pronf.
14

---

Published as a conference paper at ICLR 2025
Lemma 2 According the Theorem 11.8 in Mohri al. 2018 Let F be a famity of real-valued functions and the family of loss functions associated with F is non-negative. Assume Pdim(F] d. Then 48 O, with probability at least 6 over the choice of sample size and natural exponential e, the following inequality holds:
R() R(J)
2d
log(
em
+ V
log
m
2m
B THEORTICAL FOUNDATION FOR THE DEFINITION
The theoretical foundation for Definition is established by the principles of the WL algorithm (Le- man & Weisfeiler 1968) and the nature of TGNNs, as applied to the isomorphism problem in tem- poral graphs. Here is the proposed Proposition and its proof:
Theorem Let Gg and Gk-1 be any two non non-isomorphic temporal graphs. If a neighborhood- based massage passing TGNNs obtains different multisets of node embedding for Gk and Gg-1- Then, the temporal WL subtree algorithm determines that G and Gk-1 are not isomorphic.
Proof. Based on the foundational theory (Moris et al., 2019), we consider any two nodes from a temporal graph G EV) If the temporal WL returns Cdd (u) (u) for nodes u and e, implies that their corresponding embeddings from the TGNN are identical, i.e., g (1) = (v). This assertion be proved through induction.
= 0: Initially, the temporal WL algorithm uses the initial node features as colors, similar to how TGNNs use these features embeddings. Thus, if (u) (v), then gu g0(v) holds trivially.
0: Assume the proposition holds for depth d. The color of node v, Cd. output from the WL algorithm at iteration d depends on the multiset of colors at iteration d of its neighbors. Applying the induction hypothesis, for any two nodes u, v, such that ( Cd+ (v), it follows:
(c),),, el. (m, E)))
(11)
= (C)
This equivalence translates to the TGNN outputs:
(g"(w), (i), = (g), { Hll (z),), (v, E {})))
(12)
From this, we conclude that if the multisets of temporal WL subtree patterns (1C (w))) then {gg (w)) heev (v) )eev. Hence, neighborhood-basee based massage passing TGNN with d - layer will output the same result for nodes with the same depth in the temporal WL subtree pattern.
C GENERAL MMD
Definition 5 Maximum Mean Discrepancy. Given x² independent random variables with distri- bution Gy, and 2 independent random variables with distribution Gk-1- An unbiased empirical estimate of MMD (Gretton et al. 2012) can be defined as,
MMD G GA]
(x
k-1)
m(m
1) i=
jæi
2
m
(a
nn 1)
mn
15

---

Published as a conference paper at ICLR 2025
D TRAINING PROCEDURE
We provide the pseudo-code for our training procedure in Algorithm 1.
Algorithm Continual Learning Algorithm
1: Input: Continual tasks K. = {K1,... Kx).
2: Output: Optimized model f f(0)). ). Predicted label Y
3: Initialize 0 randomly;
4: for Gk. Yi E КK do
5: Split the training data into batches B₃ in chronological orders
6: for B% in BN do
7:
Ve 0k +- B.)
B²)
8:
LDB Fh- 9k
Eq.
9:
P&. Pk t 9k-1(0k-1, BB), (0 13,)
Eq.
10:
LIC < Pk.Pk
b Eq.
11:
Construct temporal WL subtree set S B
Prop.
12:
Obtain features of subtree patters C (vl (b)Bi)
Eq. 8,9
13:
DDSV = div B - v
> Eq. 12
14:
Lelasssfication fication f- CrossEntropy ())0)
+ arg min Cclasss ication a a(CDB Lic) + BLDIv)
16:
end for
P& 4- (gk (0%, Gk), Pk-1)
B...qq.
18:
end for
E EXPERIMENT
E. DETAILS OF DATASETS
Specifically, in the Yelp dataset, businesses represented nodes. A temporal edge is established between two businesses user comments on both within a one-week interval. We organize the dataset chronologically, treating data from each year as distinct task. To simulate the introduction of new classes in OTG, we selectively sample two large sets of business categories that have not appeared in previous tasks, treating these new classes. In the Reddit dataset, posts treated as nodes and the community topic of the post is the node label. Connections between posts are established the same manner as in the Yelp dataset. For the Taobao and Amazon datasets, products are represented nodes, and product-to-pront graph is constructed using the same criteria as for Yelp.
Table 3: Yelp.
Task 0
Task
Task
Task
Task 4
Task 5
Nodes
1905
1621
1488
2617
1088
12273
# Edges
15702
7352
2995
4383
2351
201676
Table 4: Reddit.
Task
Task
Task 2
Task 3
Task
Task 5
# Nodes
1706
1749
1853
1677
1691
2338
# Edges
12558
25610
23573
24950
61702
68004
E.2 EXPERIMENT SETTING
We assess our proposed method, in a scenario where the model must differentiate among all classes from the current and previous tasks without task indicators. For example, if the model is sequentially
16

---

Published as a conference paper at ICLR 2025
Table 5: Taobao.
Task 0
Task
Task
# Nodes
15679
8318
91415
# Edges
59523
25865
370274
Table 6; Amazon.
Task
Task
Task 2
Task
Task 4
Task 5
Task
Nodes
9097
12225
28022
39906
58930
79743
134042
# Edges
31907
63854
86814
132214
330134
1809135
1718641
trained on two tasks with class pairs ((0,))(2.3)) the model is expected to classify any of the four class labels (,,1, collectively after training the second task. The classifications not confined to separate task-specific pairs {(0,1)} or ((2,3)). This setup can demonstrate the model's adaptability across an expanding class
The model framework consists of three parts: a feature extractor (made up of TGNN layers), a discriminator (made up of MLP layers), and our proposed plug-in modules. For the implementation detail, we utilize multi-layer TGNN, where each layer contains specified number of neurons in the hidden layer. We can apply any neighborhood-basee message passing TGNNs as the feature extractor. In this experiment, we use the widely used TGNNs, TGAT (Xu et al. 2020) for the feature extractor. We sample neighbor nodes (from the current task) for each node to aggregate the neighborhood information. The input features for all datasets are processed in 300 dimensions. The network is trained using the Adam optimizer for each dataset with learning rates set at r = 0.0001 until convergence is reached. The classification is handled by two-layer MLP: the first layer uses ReLU activation and Dropout, with a hidden size of 100, to transform node features into hidden representations. The second layer consists of multiple independent linear classification heads, each designed for a specific task. The entire network is optimized using cross-entropy loss to ensure effective classification performance across tasks. After training, assess the model's performance on the current task as well as on all previous tasks. Our experiments are conducted on GeForce RTX 3090 GPU.
For the MMD measurement, we apply the widely used Gaussian kemnl as the kernel mapping func- tion, defined as (x e -1 where the bandwidth 7 is set the median of pairwise distances among the data points. To capture the characteristics of the data at different scales, we utilize a family of Gaussian kernels (vu.)= varying Yu based on the distance of input data and the number of kernels. We opt for five kernels to ensure comprehensive analysis.
E.3 BASELINES
For computer vision continual learning methods, we adapt three well-known algorithms (EWC (Kirkpatrick et al., 2016), and LwF (Li & Hoiem 2016)) a TGNN backbone. These methods primarily prevent CF through regularization and knowledge distillation. In the graph data domain, we compare our method with two ofte-of-theear graph continual learning approaches: TWP (Liu et al. 202T) and ER-GNN (Zhou & Cao 2021), which focus on weight preservation and dynamic regularization respectively. To ensure a fair comparison, we utilize the same TGNN as the backbone for these approaches, adapting them from their original implementations with static GNNs. Additionally, we include OTGNet (Feng et al. 2023). a replay-based class-incred incremennl learning method specifically designed for OTGL. For fairness in comparisons, standardized the size the candidate set and the replay buffer for the replay- based method to match the training durations of our method.
17

---

Published as a conference paper at ICLR 2025
E.4 SENSITIVITY ANALYSIS OF HYPERPARAMETERS
We analyze the effects of batch size |B] and temporal WL subtree depth d. As shown in Definition use the backbone model's number of layers represent subtree depth. Figure 6 shows that our method is not sensitive to batch size. Figure indicates that a single layer sufficient for cap- turing the necessary information, demonstrating that our temporal WL subtree effectively captures sufficient details by 0-th and 1-th patterns.
70
AP (P
AF (1)
BWT (1)
50
ACC %
30
20
Batch Size
Figure 6: Sensitivity Analysis of the Batch Size
70
AP (T)
AF (f) BWT (1)
AF (1)
BWT (1)
ACC %
ACC %
40
30
20
-10
# Layer
Layer
(a) Yelp
(b) Taobao
Figure 7: Sensitivity Analysis the Number of Layers
E.5 HYPERPARAMETERSS STUDY
To provide clear insights into the impact these hyperparameters, we conducted an ablation study, with the results detailed in Table 7. The findings indicate that both plug-in modules significantly
18

---

Published as a conference paper at ICLR 2025
enhance performance. The model achieves greater robustness when assign relatively similar weights to each plug-in in module in the calculation of the final loss function.
Table Hyperparameters Ablation Study
=
0.8
B 0.4
AP
AF
BWT
AP
AF
BWT
AP
AF
BWT
AP
AF
BWT -1.932
69.356
-2.829
-2.050
67.807
-6.441
-5.979
60.848
-12.971
-8.056
65.342
-1.932
0.8
AP
AF
BWT
AP
AF
BWT
AP
AF
BWT
AP
AF
BWT -3,975
69.356
-2.829
-2.550
66.890
-2.721
-21188
65,382
-9.236
-8.100
70.268
-6.765
F TASK-TO-TASK DISTRIBUTION SHIFTS
We calculate and present the task-to-task distributional shifts, computed by MMD, in the following Table. The data presented in the following table and the heatmap (Figure 1(b)) clearly illustrate the inherent challenges posed by structural and temporal shifts in open temporal graph learning. These visualizations show how these shifts manifest across tasks, with MMD values highlighting the extent of distributional changes shown in Table 2, we found that the L.DO module significantly enhances the model's effectiveness and robustness across four datasets, particularly in the context of the distribution shifts noted.
new tasks are introduced. In reference to the ablation study
Table 8: Task-to-task distributional shifts (measured by MMD)
Tasks
0-1
1-2
2-3
3-4
4-5
5-6
Taobao Dataset
4.245
2.228
Yelp Dataset
1.015
0.681
0.802
1.068
1.348
Reddit Dataset
1.739
1.456
2.113
1.921
1.763
Amazon Dataset
2.148
2.005
1.833
1.785
1.698
2.790
19


Reviews:
[{"review_id": "MLVm84QBGu", "reviewer": "Reviewer_TkEf", "summary": "This paper addresses the challenge of open temporal graph learning (OTGL), where temporal graph neural networks must adapt to continuously emerging new classes while preserving knowledge of existing classes. The method aims to balance preventing catastrophic forgetting while enabling generalization to new classes. The authors provide a theoretical analysis of the generalization error bounds.", "questions": "The computational complexity reduction is impressive - are there any trade-offs in terms of model accuracy when using the more efficient LTDO approach compared to traditional MMD-based methods?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: The paper presents a novel perspective by jointly addressing forgetting and generalizability in temporal graphs, rather than treating them as separate problems. The LTDO module's use of temporal WL subtree patterns is particularly innovative.\n\nQuality: The theoretical analysis is thorough, grounding the approach in PAC-Bayesian theory and providing a clear justification for the chosen architecture. The reduction in computational complexity from O(n²) to O(n) is significant.\n\nClarity: The paper is well-structured and clearly written. The problem motivation using real-world examples (like evolving cell phone characteristics) helps readers understand the practical implications.\n\nSignificance: The work addresses an important real-world challenge in temporal graph learning and provides a practical solution that could benefit various applications like social networks and e-commerce systems.", "weaknesses": "1. While the TICR module's preservation of decision boundaries is well-explained, there could be more discussion about potential trade-offs between maintaining old boundaries and adapting to new class characteristics.\n\n2. The paper could better explain how the temporal WL subtree patterns are specifically designed to capture temporal information, as this seems crucial to the method's success.\n\n3. The relationship between batch size and performance of the LTDO module deserves more analysis, as this could affect practical deployments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pCP6dhBPQW", "reviewer": "Reviewer_h3bq", "summary": "N-ForGOT proposes a novel solution to Open Temporal Graph Learning (OTGL). Specifically, N-ForGOT consists of 2 plug-in modules: Temporal Interclass Connectivity Regularization Module (TICR), which mitigates catastrophic forgetting when new classes emerge in future timestamps, and Localized Temporal Graph Discrepancy Optimization (LTDO) that learns task-invariant representations, enhancing the generalizability for models across different tasks.", "questions": "Please see Weakness.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. N-ForGOT achieves competitive empirical results and conducts a comprehensive empirical evaluation setting that consists of diverse real-world datasets and robust evaluation metrics that assess the generalizability and model’s robustness against forgetting.\n\n2. Overall comprehensive presentation and reproducibility details, as the paper presents a clear diagram describing the method, pseudo-code for training realization, and detailed report of experimental settings, including hyperparameters and computational resources.", "weaknesses": "1. The mathematical notations are hard to follow in Section 4.1, as some notations are not associated with a formal definition or description. \nIn line 238:\n>  … types for the class $i$ are denoted by $\\mathbf{p}^{(i)} \\sum_{j \\in \\mathcal{S}_i} g(x_j)$.\nBefore line 238, or after the definition of prototypes for a class, the definition of $\\mathcal{S}_i$ is not found. Does $\\mathcal{S}_i$ have the same definition as $\\mathbf{S}_i$ defined in line 259? If so, I think authors should declare this and move the definition to the right behind (or before) the definition of prototypes for the class $i$ in line 238.\n\nIn line 241, what is the definition of $\\mathbf{p}^{i}_j$?\n\nIn line 242, for the definition of $I(i, j)$: \n> … is an indicator function used to represent whether a category $n$ appears in a task $j$\nWhy does $I(i, j)$’s definition depend on $n, j$?\n\nIn lines 252-253: \n> … $\\mathbf{p}\\_{k - 1}$ is obtained from the class prototype matrix …\nWhat is the definition of $\\mathbf{p}{k - 1}$? Does it have the same meaning with the class prototype matrix $\\mathbf{P}{k}$ (defined in lines 239-240 but with a different index ($k - 1$ instead of $k$)? If so, authors should revise $\\mathbf{p}\\_{k - 1}$ to $\\mathbf{P}\\_{k - 1}$.\n\n2. What is the role of the bounding risk (presented in Theorem 1)? Does the bounding inspire any component of N-ForGOT? Or do any implications from the bound motivate the proposed method? If so, the authors should state this in the paper.\n\n3. As authors claim that TLDO helps cultivate task-invariant representations that are robust to structural and temporal shifts, I suggest authors present a plot or illustration that shows how the distributional shifts (computed by MMD) change from task-to-task. \n\n4. Are there any constraints for $\\alpha$ and $\\beta$ (defined in Eq. 9)? For better reproducibility, I suggest authors report these hyperparameters used for training N-ForGOT.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HzYyXTFF44", "reviewer": "Reviewer_Jw9d", "summary": "The paper proposes N-ForGOT, a framework for Temporal Graph Neural Networks (TGNNs) that addresses the open-set problem in dynamic graphs by preserving previously learned classes while adapting to new ones. This is achieved with two main modules: the Temporal Inter-Class Connectivity Regularization (TICR) for mitigating forgetting of prior knowledge, and the Localized Temporal Graph Discrepancy Optimization (LTDO) to manage distribution shifts across tasks. Empirical results show that N-ForGOT outperforms existing methods in prediction accuracy, minimizing forgetting, and enhancing generalizability across tasks.", "questions": "I do not have extensive expertise in this research area, but I have some general questions:\n\nQ1: My understanding of the open-set setting is that it allows predictions on completely unseen classes. However, in the proposed method, it seems the model gradually learns across tasks by incorporating new classes over time. If the dataset consists of six tasks, does this mean the prediction space is limited to these six tasks and wouldn’t cover unknown classes beyond them? In the test set, are predictions also evaluated solely on the six classes seen during training? In this case, can we take it as the 'open-set'?\n\nQ2: The paper mentions dynamic graph distribution shifts, but does this refer only to shifts in class labels over time? Are other types of distribution shifts, such as changes in features or interactions over time, also considered or defined in this work? And how these distribution shift issues are modeled and integrated into the proposed method?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1: Clarify: The overall demonstration of open-set learning in dynamic graphs is very clear, in terms of two core challenges of distribution shifts and knowledge forgetting, also with clear motivation for the proposed continual learning mechanism.\n\nS2: Method: Two modules, TICR module and LTDO module, help the model retain essential inter-class information, reduce forgetting and preserve learned knowledge over time, and manage distribution shifts across evolving tasks, sounds rational to me.\n\nS3: Empirical Experiments & Good Structure: The method demonstrates effective performance on multiple benchmarks, with good logic and organized well.", "weaknesses": "See questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HnieYIqL1m", "reviewer": "Reviewer_sieS", "summary": "This paper focuses on temporal graph learning and proposes a method, named N-ForGOT, to address evolution in node classes and dynamic data distributions triggered by the continuous emergence of new class labels. Two specific modules are designed to address the corresponding problem. Experimental results on four public datasets demonstrate the effectiveness of the proposed method.", "questions": "1.\tSome recent works on graph continual learning are not included in the paper, such as [1][3].\n2.\tThe authors are encouraged to discuss the differences between the studied problem and graph continual learning more clearly as the distribution shift and forgetting problem also exist in graph continual learning [2].\n3.\tMore recent graph continual learning baselines are encouraged to be included for comparison,\n4.\tWhen learning a new task, is the data of previous tasks available and included for training? In graph continual learning, there is no access to data from previous tasks and classes.\n5.\tIn line 243, should the category “n” be “i”?\n6.\tIn line 349, there are some overlapping.\n7.\tPlease revise line 420 to make it more clear.\n8.\tIn 484, the figure should be Figure 4 and there should be margins between the caption of Figure 4 and the main text.\n9.\tWhy are there learning rates for the model optimization in line 892?\n\n[1] Niu C, Pang G, Chen L. Graph Continual Learning with Debiased Lossless Memory Replay[J]. arXiv preprint arXiv:2404.10984, 2024.\n[2] Zhang, X., Song, D., & Tao, D. (2022). Cglb: Benchmark tasks for continual graph learning. Advances in Neural Information Processing Systems, 35, 13006-13021.\n[3] Liu, Y., Qiu, R., & Huang, Z. (2023, December). Cat: Balanced continual graph learning with graph condensation. In 2023 IEEE International Conference on Data Mining (ICDM) (pp. 1157-1162). IEEE.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The studied problem is practical and interesting.\n\nThe authors provide a theoretical analysis of the generalization error bound.\n\nThe experimental results are promising.", "weaknesses": "The related works are not comprehensive.\n\nSome symbols and statements are not clear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)