Universal Graph Continual Learning
Authors: Thanh Duc Hoang
Do Viet Tung
Duy-Hung Nguyen
Bao-Sinh Nguyen
Huy Hoang Nguyen
Hung Le
Venue: Accepted by TMLR ()

Abstract: We address catastrophic forgetting issues in graph learning as the arrival of new data from diverse task distributions often leads graph models to prioritize the current task, causing them to forget valuable insights from previous tasks. Whereas prior studies primarily tackle one setting of graph continual learning such as incremental node classification, we focus on a universal approach wherein each data point in a task can be a node or a graph, and the task varies from node to graph classification. We refer to this setting as Universal Graph Continual Learning (UGCL), which includes node-unit node classification (NUNC), graph-unit node classification (GUNC), and graph-unit graph classification (GUGC). Our novel method maintains a replay memory of nodes and neighbours to remind the model of past graph structures through distillation. Emphasizing the importance of preserving distinctive graph structures across tasks, we enforce that coarse-to-grain graph representations stay close to previous ones by minimizing our proposed global and local structure losses. We benchmark our method against various continual learning baselines in 8 real-world graph datasets and achieve significant improvement in average performance and forgetting across tasks.

Published in Transactions on Machine Learning Research (11/2023)
Universal Graph Continual Learning
Thanh Duc Hoang', Vict-Tung Do", Duy-Hung Nguyen, Bao-Sinh Nguyen flris, ace, hector, simonn/Ginnamon..
Cinnamon AI, Vietnam
Huy Hoang Nguyen University of Oula, Finland
bag.ngrayenBoull.ff
Hung Le
thai, @deakin. eda.oo
Deakin University, Austmalia
Reviewed OpenReview: https: openreview. net/ forum? id= waRE5kIn
Abstract
We address catastrophic forgetting issues in graph learning as the arrival of new data from diverse task distributions often leads graph models to prioritize the current task, causing them to forget valuable insights from previous tasks. Whereas prior studies primarily tackle one setting of graph continual learning such as incremental node classification, we focus on universal approach wherein each data point in a task can be node or a graph, and the task varies from node to graph classification. We refer to this setting as Universal Graph Coulinual Learning (UGCL), which includes pode-unit node classification (NUNC), graph-unit node classification (GUNC), and graph-unit graph classification (GUGC). Our novel method maintains replay memory of nodes and neighbours to remind the model of past graph structures through distillation. Emphasizing the importance of preserving distinctive graph structures across tasks, we enforce that coarse-to-grail graph representations stay close to previous ones by minimizing our proposed global and local structure losses. We benchmark our method against various continual learning baselines in real-world graph datasets and achieve significant improvement average performance and forgetting across tasks.
1 Introduction
The recent literature has shown the high polential of graph neural networks (GNNs) in individual tasks from various domains, such as social and molecule networks (McCallum et al., 2000; Hamilton et al., 2017). However, when graph data come sequence, GNNs suffer from catastrophic forgetting (French, 1999). In particular, we treat subgraph a data point, a stream of data arriving from different task distributions may cause graph models to merely learn the current task using current data but forget the older ones. Thus, data from previously trained tasks. The forgetting occur locally (node-unit)
GNNs underperform globally (graph-unit), depending on the task, which can be node or graph classification ( (Galke et al., 2020; Carta et al., 2021; Wang et al., 2022).
There exist many continual learning algorithms to cope with catastrophic forgetting. Unfortunately, they are mainly designed for primitive neural networks such as multiple layer perceptron (MLP) and convolutional neural network (CNN), where the task is image classification (Kirkpatrick et al., 2017; Rusu et al., 2016). Prior works have developed methods for GNN, primarily for node incrementation (Wang et al., 2022) developed method to cast vertex classification as graph classification task by transforming each node of the graph into feature graph, which might not consider relations in the original graph. Furthermore, it is unclear whether the approach can be extended to other settings involving graph incrementation. Recently, Zhang et al. (2022) have introduced benchmark on continual learning for both node and graph classification, partly covering
Equual contributions

---

Published in Transactions on Machine Learning Research (11/2023)
incremental graph learning. However, the benchmark lacks the scenario where each data point is graph, and the task is node classification, which can be seen in document (Park et al., 2019) and scene graphs (Tang et al., 2020).
In this work, we address continual learning in a more general context where the incoming data point can be either a node or graph and the task can be either node or graph classification. Hereinafter, we name the setting Universal Graph Continual Learning (UGCL). First, we provide taxonomy of scenarios in UGCL consisting of node-umit node classification (NUNC), graph-unit node classification (GUNC), and graph-unit graph classification (GUGC). These scenarios have real-world significance. For instance, in NUNC, consider a social network analysis system that must continually adapt to changing user behaviors, identifying emerging trends or influencers. In GUNC, think of a document analysis system classifying text segments within evolving document graphs, essential for applications like document categorization. Lastly, in GUGC, we can have a computer vision system analyzing complex scene graphs to understand object context and relationships, crucial for autonomous driving and robotics.
We propose new method to alleviate catastrophic forgetting in all three scenarios. Our approach adapts rehearsal and distillation techniques to graph-based models to enhance knowledge retention. We maintain a replay memory of nodes coupled with their neighbors and continuously draw samples from the memory to remind the model of previous graphs through distillation mechanisms. We argue that, persevering local and global structure representations are critical for the effectiveness of graph modeling in UGCL. Hence, we enforce these representations produced by the model in a task to be close to those of the previous tasks. As such, we store models corresponding to all tasks and minimize the difference between the representations.
We demonstrate the performance of our method and other continual lcarning baselines in the three scenarios. For NUNC and GUGC, we employ 5 established datasets to verify our method. For GUNC, we create a new benchmark consisting of 3 public document image understanding datasets, in which a document is represented as a graph of text. boxes, and the task is to extract the content and the type of these text boxes (i.e. node classification). In any case, our proposed approach outperforms the other continual learning methods significantly and shows competitive performance against the theoretical upper-bound baselines. The contributions of our study are as follows: (1) we formalize a comprehensive taxonomy of graph continual learning and propose a new method to handle all cases in the taxonomy. Our method leverages rehcarsal to preserve graph data's local and global representations; (2) we create new continual learning benchmark for graph-unit node classification, following the Class- Incremenall setting with the task as node classificatin;; (3) we conduct extensive experiments for node and graph classification to validate our method using a total of 8 datasets (see Table 6).
2 Related Work
Recent studies on catastrophic forgetting (CF) mainly consist of experience replay, regularization, and parameter isolation. Experience replay methods store old data via experience replay buffers and replay them while learning new task (Iscle & Cosgun, 2018). Regularization-basse methods leverage additional loss terms (Febrinanto et al., 2022) to consolidate knowledge in the learning process for new tasks and keep previous knowledge. For instance, EWC (Kirkpatrick et al., 2017) keeps the network parameters close to the optimal parameters of previous tasks using L2 regularization while parameter Isolation methods (Mallya & Lazebnik, 2017; Le & Venkatesh, 2022) keep an independent model for each task to prevent any possible CF among tasks. As another example, progressive networks (Rusu et al., 2016) grow new branches for new tasks while freezing previous task parameters to avoid modification of prior knowledge. These methods are designed for simple neural networks such as MLP or CNN and are validated only in image classification. It is non-trivial task to apply them to graph data, which are commonly found in real-world applications in a streaming fashion, such as social networks, road routes, and citation graphs.
Feature graph networks (FGNs) (Wang et al., 2022) is an early approach that addresses CF in GNNs. FGN takes the features as nodes and turns nodes into graphs so that new nodes in the regular graph turn into individual training samples. As result, the node classification task becomes a graph classification task. Research in rehearsal methods is also applied to graph data such as ER-GNN (Zhou & Cao, 2021). This work introduced three novel approaches to selecting prior nodes to be stored in the experience buffer that.
2

---

Published in Transactions on Machine Learning Research (11/2023)
aims to prevent the forgetting phenomenon in continual node classification. ContinualGNN (Wang et al., 2020) proposed to combine both rehearsal-based methods and regnlarization-based methods to tackle the CF issue when the graph evolves, and new patterns appear over time. During the new pattern detection process, they store a small amount of high-influence nodes of existing patterns. Then, a regularization-bant based method similar to EWC is employed to keep the current model parameters do not go too far from previous model parameters so that both prior knowledge and current knowledge are maintained.
However, all the aforementioned approaches focus one evolving graph over time. As far as our knowledge, the problem with multiple separate graphs from task has not been studied. The closest to this task is ContinualGNN (Wang et al., 2020), formulating nodes and neighbors belonging to old patterns as one single graph. In the prior study, each data point is a single graph, but these graphs can have common nodes, whereas we aim to deal with multi-separated graphs. It is crucial to derive a method that can work in any setting, including a single big graph and multi-separated graphs. Our method can be cast as a hybrid of regularization- based and rehcarsal- based methods, as attempts to preserve knowledge inferred in previous lasks to overcome rophii forgetting through representation constraints and replaying. Unlike previous replay methods, however, we do not focus which samples to replay; instead, we integrate task-agnostic local and global information from graph structures into continual learning.
3 Problem Formulation
In the following, we first summarize all scenarios in graph learning and their characteristics regarding continual learning. Then, we detail the problem of graph-unit continual learning, especially GUNC, which is currently underexplored in the literature.
3.1 Graph Learning Taxonomy
A graph is defined as G (V, E) where V is the set of nodes and E C V2 is the set of edges. Each node v E V is coupled with feature vector I E X C. R.. Each edge e E E' is associated with a weight vector w E R.W
Task
Ngraphs
Unit
Classification
NUNC
Single
Node
Node
GUGC
Multiple
Graph
Graph
In Table 1, we categorize regular graph learning into 3 distinct tasks based on the number of graphs Ng and the definition of sample (or unit): node-unit node classification (NUNC), graph-umit graph classification (GUGC), and graph-unit node- classification (GUNC). First,, in NUNC and GUNC, we learn predictor f to assign a label E Y for every node u V, given a graph G with its node features X and edge weights W. Unlike the node-unit version working on a single big graph, our task usually involves multiple individual graphs. Big graph datasots like CORA (McCallum et al., 2000) often have a big node feature vector of size 1000 constructed from long text paragraph which sometimes makes it less dependent extracted from datasets like CORD (Park et al., 2019) have only smaller feature vector of size smaller than 100 made from a few words and numbers. Consequently, graph-unit learning is rather topology-dependen this case. In a single big graph with a million nodes, we can use message-passngg to propagate information from far away nodes. On the other hand, multiple graphs in graph-unit learning often have less number of nodes (about 200 nodes), thus a shallower topology. Hence, with those limitations, graph learning and continual learning with GUNC are prone be harder than their node-unit counterparts. NUNC has been formulated and studied extensively under continual learning context (Zhou & Cao, 2021; Wang et al., 2022). The following section will focus more on GUNC and GUGC.
GUNC
Multiple
Graph
Node
Table 1: The distinction of graph-unit node classification (GUNC) tacked in this work compared to node-unit node classification (NUNC) and graph-unit graph classification (GUGC).
the graph topology. Meanwhile, graph-unit data
in
3.2
Graph- Unit Continual Learning
In the continual learning setting for GUNC and GUGC, we have the same objective as regular graph-unit learning, but the data comes in the form of a sequence Gt (uf, et, at, w/, vt) v where & is the task index
3

---

Published in Transactions on Machine Learning Research (11/2023)
and M & M is the number of samples in the sequence. Assume that all items in G belong to a distribution pl or (uf: et, ar,, uf) ~ Pt, our task is to maximize the likelihood of a label y' ~ P'llee e, e, The test sample might belong to one of the past tasks or the current task t. Therefore, our graph-umit continua learning objective is not only to perform current task training but also not to forget past tasks t' to overcome calastrophic lorgetting (Li & Hoiem, 2017; Kirkpatrick et al., 2017; Rusu et al., 2016).
In particular, we focus our experiments on a specific continual learning setting called dlass-incremental learning (CIL). In CIL, each task contains an exclusive subset of classes Y C Y. According to Hsu et al. (2018), CIL is the most practical yet most difficult scenario where have disjoint output spaces Y iinn y = 0 between tasks which naturally leads to disjoint label distribution Pi(Y) n p (Y) = 0 and thus Pt(G) & p (G). We note that it differs from task-incremental learning (TIL), where the latter has a big relaxation with the known task identity or task descriptor. Therefore, one could build a multi-head architecture to treat each task separately, which totally eliminates the risk of cross-task prediction.
4 Method
In this work, we aim to develop system to universally overcome catastrophic forgetting for NUNC, GUGC, and GUNC. The workflow of the proposed system is shown in Figure 1. Given an imput graph and embedded feature of nodes, our experience replay (ER) backbone solects replay samples and stores them in a Replay Buffer (Rolnick el al., 2018). Then, we utilize Local Structure Distillation and Global Structure Distillation to remind the model of the global-local structure of graph samples in previous tasks. The backbone of our method is based on the rehearsal approach, which often dominates in continual learning because the replayed samples provide direct information about past tasks, giving huge advantages against regularization-base approaches. Whereas most prior replay mechanisms focus on which samples to replay, we investigate another aspect. As such, argue that aside from task-related information, additional knowledge can be derived from replayed samples in the form of a task-agnostic distillation loss. Although we discuss only one replay mechanism as the backbone, our distillation mechanism is not limited to any specific one.
4.1 Graph Experience Replay
Experience Replay (ER) is well-known in the continual image classification setting with its simple imple- mentation yet effectiveness. Regarding graph learning, the ER adaptation requires more or fewer changes depending on the specific learning task. Similar to the image domain, ER for graphs (Zhou & Cao, 2021) stores a portion of data belonging to classes from past tasks in the replay buffer Gg. Then, in each training step of the current task, we query k samples to optimize together with the original training data to prevent catastrophic forgetting. For GUGC, we store graphs belonging to classes from past tasks with all of their nodes in the replay buffer. For GUNC, although we only need nodes belonging to classes from past tasks, we also store their correspouding graphs to use the connectivity information for later computation. Hence one item (either node or graph) in our replay buffer always comes with an associated graph. For NUNC, we apply similar procedure except that all nodes are from a single graph. Therefore, unlike NUNC having all nodes connected to each other, GUNC has only some connections from nodes in the same graph, thus making it harder to learn (see more in Sec 3). In the following, we describe the implementation of the replay bulfer.
First, our replay buffer G, is implemented as fixed-size list whose each element is a graph/ node with its label y E Y To expand G.. we collect all training data in G after finishing the training on G At this point, our replay buffer G, might become bigger than the allowed size d, which triggers replacement. procedure.
The replacement strategy iterates between steps: (1) finding the class that has the biggest number of samples in the replay buffer, and (2) removing one sample belonging to that class to ensure class balance (Prabhu el al., 2020). To choose which node to remove, we can use random selection procedure or ranking selection like Mcan Feature and Coverage Maximization (Zhou & Cao, 2021). Finally, with the replay buffer Gb, we can optimize our model continually, minimizing the total loss CTotal which is convex combination of the current downstream task loss L and the contimual learning loss CCL focusing on previous tasks as follows,
cTotal_ =aL + (1 ca)
(1)
4

---

Published in Transactions on Machine Learning Research (11/2023)
Task Loss Minimization
Update
Local Structure Distillation
Update
GNN
Global Structure Distillation
Update
Task
GNN
Graph metching
Replay Buffor
Sampling
Figure 1: Overview of the proposed method. For simplicity, we show N tasks, where the model will be continuously learned and updated. The left side gives the building and updating of the Replay Buffer (RB) across tasks. While the right side describes the replay and knowledge distillation process. Modes//rappp sampled from RB, along with those in the current task, will be used gain structured knowledge from the previous task model by using Local Structure Distillation and Global Structure Distillation processes. The RB will be expanded with current task graphs by replacement strategies.
Traditionally, L L(J(U, x,,, s.l. (V,e,,,w,)) G¹, and L LER [(f eb, Tb, wp), Mo) s.t. (Va, (6,T6, Wb, yb) Go, where LER is known experience replay loss, f is the predictor, and L is the conventional cross-entropy. We use uniform sampling strategy to sample examples from the replay buffer Gs for simplicity. Here, a is the coefficient balancing the learning of the current and past tasks, which can be tuned during training. However, to reduce the hyperparameter tuning cost, we compute it as a = [Gsl/([ss) + |G'I). the next sections, we propose to enhance the continual learning loss LCL with our new local and global structure losses.
4.2
Knowledge Distillation
As graph learning is mainly about iterative feature aggregation or message passing across local neighbors, transferring learned graph structure representations is critical to retain a graph model performance. Since the experiment replay strategy reminds the model of previous task samples within the task objective, it may not help explicitly transfer graph topology structure representation learned by trained models. Inspired by distillation approaches (Hinton et al., 2014; Yim al., 2017), we propose a way to effectively transfer the acquired structure knowledge from models trained in old tasks. Prior distillation strategies extract the output (Hinton et al., 2014), intermediate activation (Zagoruyko & Komodakis, 2016; Huang & Wang, 2017), or label (Hou et al., 2019; Douillard et al., 2020) from CNN models with image inputs. Unlike them, our method focuses on graph neural networks, which handle a more general input from mixed domains (visual, lext, molecules). While there are some studies (Deng & Zhang, 2021; Yang et al., 2020) try to distill knowledge from previous graph models to a new model, they merely focus on learning the structure of the input graph or only learning local structure. Our work aims transfer global and local charactoristies of graph structure representations embedded in trained models a model in the new task, thereby preventing the model from forgetting key characteristics of past tasks.
5

---

Published in Transactions on Machine Learning Research (11/2023)
4.2.1 Local Structure Distillation
Aggregating neighbor information plays an important role in graph learning, which depends much on typical local context to make the prediction. Therefore keeping the model from forgetting previously learned local context or structure between a node and its neighbors is necessary. We hypothesize that the model trained on previous lasks has already encoded the local structure representations of past lasks. Hence, using past samples from the replay buffer can reconstruct the representations in the past models. These representations will be used to enforce the current model to output similar local structure representations as the past models!.
Let 9 denole the graph associated with the item sampled from the replay buller. We use this graph to extract the neighbors of the considering node to build the local structure representations. We construct a representation of the local structure through steps: (1) extracting node representations and (2) forming local structure representations.
Given a set of node features X = (2.,,,, from the graph 9 containing N nodes and an adjacency matrix A that represents the edges among nodes. The hidden reprosentation of node v; at the I-th layer, denoted as z;
(1) is formulated as ( = o(jek, A,z2 " where K(i) denotes the neighbors of node oi
o() is an activation function, and W is the transformation matrix of the (-th layer. (0) represents the input feature of node t; W is weight matrix that defines the aggregation strategy from neighbors. Inspired by Mepherson et. al. (2001), we represent the local structure representation by measuring the difference between the features of a node and the features of neighbor nodes. More specifically, the local structure difference would be formulated as follows
N
(0.0) = z
Z
(2)
N;
==1
M,
S(g.5)
2
Zm
(3)
M,
m=11
Eqs. (2) and (3) build the local structure embedding vectors as the difference between a node feature and its neighbor mean feature, where z and z are node features from the current model and the previous model, respectively. z and zn are neighbor node fcatures from the current and old models, respectively. N, and Mc are the neighbor size where neighbors were selected based on their connection with the node e; which is defined in the adjacency matrix A. We then try to minimize the distance between two embeddings by minimizing the following Local Structure Loss
Kg
LLS
1
1 cos (S S
(4)
Kg
Kn
where cos is the cosine similarity, Kg is the number of associated graphs sampled from the replay buffer, and K2 is the number of nodes sampled from those graphs. This mechanism prevents the current model from forgetting previously learned local structure representation in previous tasks while not interfering directly with the learning of the current task since the regularization occurs only in feature space, not in the final prediction layer, as in Experience Replay. The choice of K,, and its impact will be discussed the Experiment section.
4.2.2 Global Structure Distillation
The latent interactions among non-neighbor nodes also play an essential role in learning tasks that require global interactions, such as graph classification. Preserving the global structure of how the previous model embeds would keep the current model from forgetting such information learned previous tasks. We explicitly train the current model to preserve the global structure distil ropresentational knowledge from the previous model better. To achieve this, we introduce the Global Structure Loss to maximize similarities among all node features.

---

Published in Transactions on Machine Learning Research (11/2023)
Specifically, distil the representations learned from the previous task model by computing similarities between graph embeddings from the current and the previous task's 8 model as follows
K2 [
CGS
1
E j=1
cos 19f1
(5)
Kg
where ff and JJ are the graph embeddings from the previous and the current task's model, respectively, where they are computed by applying weighted sum and max pooling to the node features generated by the models followed by linear layer. Kg is the number of graphs retrieved from the replay buffer, similar to that in the local structure distillation. Here, replaying graph samples of previous tasks prevents the current model from forgetting learned global structure information in previous lasks, which assists the represent ation learning the entire graph distribution.
Finally, the continual learning loss L becomes a linear combination of the local, global structure and the experience replay loss:
C = CER + BCLS + CCSS
(6)
where B and y are hyperparameters to be tuned while training.
5
Experiments
5.1
Implementation Details
Model Architecture Both node classification (GUNC, NUNC) and graph classification (GUGC) share the same graph model architecture simple graph convolutional network (GCN) with 3 GCN layers (Kipf & Welling, 2016). Before being fed to the first GCN layer, the input features go through a linear layer to learn a more compact representation. Except for the first layer, each GCN layer has a residual connection from its input to its output. For GUGC beuchmarks, we add a WeightedSum AndMaaax AndMax pooling layer to calculate the graph embedding. Finally, we pass the final GCN output through a linear layer with a softmax activation to output the multiclass probability for each node/graph.
(Zhang el al., 2022)
Training Protocol We follow the traditional setting of Class Incremental learning. Specifically, first train the GNN with either node classification or graph classification objective the first task consisting of N1 Velasses classes. Next, from the second task to the last task, we keep expanding the model output layer with additional Netasses units. Strictly, we only provide the current. lask labels, even though our model has past task output units. As result, the model performance vastly degrades if no mechanism is implemented to prevent catastrophic forgetting. Additionally, we keep all the model checkpoints after every task to run the evaluation later. Besides, keeping past checkpoints is necessary for our distillation strategics.
Evaluation Protocol After training model on each task, we evaluate its performance on the testing data of the current task and all previous tasks. To be in line with Chaudhry et al. (2018); Lopez-Paz & Ranzato (2017); Zhang et. al. (2022), we also utilize the metries Average Performance (AP) and Average Forgetting (AF).
Baselines We extensively compare our method with non-continual learning baselines and continual learning baselines. Non-CL basclines:
Finetune (Girshick et al., 2013): This baseline follows the above training and evaluating protocols without any further mechanisms to prevent catastrophic forgetting. This baseline a basic starting point for further integration of advanced continual learning techniques.
Feature extraction (Donahue et al., 2013; Razavian et al., 2014): The baseline trains the model on the first task dataset and then freezes all previous layers while adding new output units to the classification layer (see Training Protocol). With this baseline, we want validate whether model can still learn a new task without forgetting past tasks by keeping the same representation.
7

---

Published in Transactions on Machine Learning Research (11/2023)
Independent (Paz & Ranzato, 2017): Instead of expanding the model, train whole new GNN with Nelnsses/ Nuastes output units for every encountering task. As a result, the Forgetting Measure for this baseline is always zero. At the same time, the Average Accuracy is calculated by averaging the current task accuracies but not the final ones like the other basclincs:
Niasoe
A Niasss
Σ ajs
Ntasks
Often, the negative impact of forgetting is bigger than the positive impact of the forward transfer. Thus, the result of this baseline is considered an upper bound for other methods.
Joint training (Caruana, 1998): This baseline is very similar to Finetune, but we keep the past datasets alongside the current task dataset to train the model. Therefore, the result of this baseline is also another upper bound for other modh
CL baselines. By adding different mechanisms prevent catastrophic forgetting over the Finetune baseline, obtain several advanced baselines: LwF (Li & Hoiem, 2017), DK (Hinton et al., 2015), EWC (Kirkpatrick et al., 2017), MAS (Aljundi et al., 2018), TWP (Liu et al., 2021), GEM (Topez-Paz & Ranzato, 2017), and ER (Rolnick et al., 2018). Since some of the methods are originally for non-graph data, we slightly adapt them to our tasks as follows:
In LwF, we strictly follow the original implementation with the distillation loss calculated based on the output of the current model and the model in the previous task for an input graph in our current training data. Specifically, given an input graph, we calculate the distillation loss for all node features.
DK is implemented similarly, except for the loss function changes to KL. Divergence Loss (Joyce, 2011).
For EWC, we also follow the original implementation that employs the Fisher information matrix to regularize the loss function so that network parameters are close to the Iearned parameters of previous lasks.
MAS can be implemented similarly EWC, except we use the gradient of the output with respect to a parameter to evaluate its importance.
On the other hand, TWP is designed for graphs with loss to preserve the topological information of the graphs between tasks.
For GEM, we store training samples (graphs/ nodes) in the replay buffer to then modify the current task gradients with the gradients calculated with the replay buffer data.
ER-GNN is implemented in (Zhou & Cao, 2021). This method the direct adaptation of the well-known Experience Replay mechanism in classical continual learning to the new domain- graphs.
ER is the base method for our work that already described in Sec 4.1 which is highly inspired by ER-GNN but with the additional replacement procedure. For the replacement strategy, we use random selection. We note that since GUGC and NUNC are common settings in the literature (Zhang et al., 2022), we can implement or reuse baselines on those settings than on the less well-known setting GUNC.
Experience Replay with Structure Distillation (Ours). Our method integrates task-agnostic knowledge into ER. Like other ER-related methods, the replay buffer size acts one hyperparameter. Moreover, to apply Structure Distillation to NUNC, GUNC, and GUGC, we loop through set of T graph samples from both the replay bufler and input graphs. In our experiments, we sampled graphs from the replay bulfer equal to the batch size of the input graph for distilling structure knowledge. Similar to other baselines like LwF and DK, the distillation loss in our method transfers naturally from node classification to graph classification. In total, we examine 3 variants, which are ER with global structure distillation (ER-GS), ER with local structure distillation (ER-LS), and ER with both global-local structure distillation (ER-GS-LS).
8

---

Published in Transactions on Machine Learning Research (11/2023)
CORD
SROIE
Wild Receipt
Method
AP (%) 1
AF (%) 1
AP (%) 1
AF (%)
AP (%) 1
AF (%)
Finetune
15.3 4 0.3
76.7 4 0.2
12.4 0.4
80.1 4.1.1
23.6 0.1
-78.8 0.1
Feature Extraction
19.6 1.3
1.7
22
0.8
231
-10.5 0.6
Independent
1..9
819
816
Joint
9.8
80.9 144
0.3
EWC
14.5 2.0
-33.0 1.6
12.6 1 0.2
335.4 0.1
19.3 0.4
-34.1 1 13
LwF
15.3 1.0
0.9
13.8 1.0
-3299 1.7
0.5
8.7
DK
1.39
22
11.1
56
22.6
0.6
ER (Ours)
79.4.10..4
-2310.
81.0 1 1.4
-15
81.6.00 0.3
1244
ER-GS (Ours)
52.7+111
0.5
8..0
-12#02
021
0.0
ER-LS (Ouns)
80.4 1.1
0.3
0.2
0.3
82.3
-2.
ERL EII-GS-LE LS (Ours)
84.0 1.0.1.00
0.4
0.3
84.4
0.3
-1.2.10.22
83.0 0.1
-1.2
0.6
Table 2: Performance comparisons on the task of Document Image Undorstanding. the best/second best-perforning CL technique for each column. We note that non-CL baselines (first rows) serve as theoretical bounds for reference purposes. denotes ignored AF calculations.
denote
5.2 GUNC: Document Image Understanding
Datasets We consider document image understanding datasets: SROIE (Huang et al., 2019), CORD (Park et al., 2019), and WILDRECEIPT (Sun et al., 2021) (Appendix Table 6). We can set up graph-unit node classification document dalasets with few heuristic steps (refer to Task Description). As mentioned, document image understanding datasets often have less informative feature vectors and shallow topology, which causes difficulty for graph-unit node classification. For example, in CORD, the maximum number of nodes is 84, and the input feature vector size 51. For SROIE and WILDRECEIPT, the quantities are (240, 43) and (217,81) respectively. Thus, doing continual learning for GUNC for those document datasets is rather challenging.
In the graph-umit node classification task, given the input nodes with associated features and also the graph, the task is to generate the embedded feature for each node such that nodes with different classes can be separated. We adopt CORD, SROIE, and WILDRECEIPT datasets that contain, 30, 4, and 12 classes, respectively. For CORD, we keep the original train-validatioon- split while we further split the training data of SROIE and WILDRECEIPT with a ratio 80-20 for the training and validation set.
Task Description For document image understanding using graph-unit node classification (GUNC), we first build graph where each node is equivalent to a text box. Subsequently, the label for each node the calegory of the corresponding text line. For example, in CORD (Park et al., 2019), we have categories like adopt a heuristic
_hore_naee, price. Next, to form the edge information and build an adjacency matrix, rule based on the horizontal/ viriiontal/vertcaall vertical alignments between nodes as well as their distances (Qian et al., 2018). Particularly, since different documents have different sizes, we use a relative distance compared with the document image size to justily an edge.
After that, with the set of distinct classes as C, we construct Nissk disjoint same-length subsets (G*) as Neasks continual learning tasks. In each subset, we ignore nodes with labels y & Ct in the training loss for that set. A detailed task setting is shown in Appendix Table 6 for GUNC, including SROTE, CORD, and WILDRECEIPT. Often, in GUNC, we leverage non-target nodes negative samples to boost the overall accuracy. Therefore, in GUNC, tasks can have only 1 class like in the case of SROIE without losing generality.
Hyperparameter Tuning We use grid-scarch to tune common hyperparameters. Particularly, the number of epochs is sampled in the range of [50, 100] with a discretization step of 10, and the learning rate of Adam optimizer sampled from the set {10 2 10 10 10 53. For EWC, LwF, and DK, the old-new task weight sampled in the range of 00.10.9) For ER, we set the replay buffer size to 1000 nodes from all classos. For our methods, we optimize the loss weights and y in the range of [0.1, 0.9) using a faster hyperparameter search: Bayesian Optimization (Wu et al., 2019) and realize that balance weights B 7 0.5 work best. For
9

---

Published in Transactions on Machine Learning Research (11/2023)
Method
ENZYMES
Aromaticity
AP (
AP (%) T
AF (X)
Fincture
23.0
+..5
Joint
93.2162
77.2 4 1.3
172100
11
7.5116
2.5
MAS
21,6116
2.5
7.8
80.5 2.1
GEM
2..0++00
9.3 2.4
75.6 1.1
2..5.10..
-66.5
6.5
LwF
2100111
5.2
3.7
-16.1 4.7
ER-GNN
21.3 1.3
4.5
18.2 1.6
ER (Ours)
2.95
-3.0
37.2
110011 1.3
ER-GS (Ours]
21.
7.2
35.4
6.7
ER-LS (Ours)
23.3 1.7
2..00
10.0
11.5
ER-GSSTS (Ours)
25.5
20.0
45.4 1.5
1..
Table 3: Performance comparisons under task Graph Classification different datasets. Bold/unded /underlied denote the best best/second best-performing CL technique for each column. We note that non-CL baselines (first rows) serve as theoretical bounds for reference purposes. - indicates ignored AF calculation.
ER-LS, and ER-GS-LS, we set K₂, equal to batch size which is graphs in the document not large and dense compared to other datasets.
The results on the 3 document image understanding datasets CORD, SROTE, and WILDRECEIPT are shown in Table 2. Through reusing past samples from provions tasks, ER can largely improve the performance of the fine-tuning stralegy and is significantly better than baselines, including EWC, LwF, DK, and Feature Extraction. Particularly, the model can remember a past task by utilizing a set of past representative samples in the replay buffer. This direct strategy turns out to be more effective with graph data since graph-unit continual learning often contains distinct tasks, which is unlike image classification, where different tasks share similar structures like digits (MNIST (LeCun et al., 1998)). For instance, a task in CORD can contain address information while another task can contain price. As a result, regularization-baese basedd methods like EWC, LwF, and DK tend to perform poorly on our GUNC benchmark. Our method further improves the vanilla ER with the ability to transfer the global and local structure information, resulting the best performance among all the comparison methods.
Results
Regarding evaluation metrics, the average improvements of our method over Finetune CORD, SROIE, and WIL.DECEEPP are 68.7%, 72.0%, and 59.4%. As stated in the baseline subsection, joint training can be considered an ER-based method whose replay buffer contains all past data. Thus, it is reasonable that our proposed method, in general, has lower performance than this upper bound baseline. Surprisingly, SROIE benefits most from our framework, which performs even better than joint training. However, without the local-structure (I.S) and global-structure (GS) distillation, ER baseline alone only has similar performance as the joint training on SROIE. That indicates the importance of introducing additional topology information in the form of LS and GS. Besides, our proposed method requires fewer samples than the Joint baseline, which results in a faster training speed. Formally, a joint training uses Neasta task data while our method uses only Nusks task data. Thus,
2
method is (Neaket1) times the speed of the joint training bascline. For example, in CORD, Ntaks = 5, we have a 3 times faster training speed. Note that we ignore the cost for the replay buffer because its fixed size does not scale with the number of tasks.
The improvements mainly come from incorporating information from both the data and graph learned structure representation, which provides additional supervision that can fully utilize the graph structures and limited samples. Specifically, our methods are remarkably less forgetting compared to other baselines with AF of 1.9%, 11.2, and 1.22% in CORD, SROIE, and Wild Receipt, respectively. The results are also nearly approach results of Independent and Joint approaches that require all training samples across tasks and more computing resources.
5.3 GUGC: Molecule Classification
Datasets We consider molocule graph classification datasets: ENZYMES, and Aromaticity (Xiong et al., 2019) (see details in Appendix Table 6). FNZYMMS is dataset of protein tertiary structures which contains
10

---

Published in Transactions on Machine Learning Research (11/2023)
600 enzymes. The task is to classify each enzyme into one of the 6 Enzyme Commission numbers (EC number). A Aromaticity (Xiong et al., 2019) is a molecule dataset with the task of predicting the number of aromatic atoms in molecules. The node features and adjacency matrices are available in the dataset. We split the data into iaining/validation, /esst sets with a ratio of 80/10/10.
Task Description For graph classification using graph-umit graph classification (GUGC), we are given the processed graphs with all the necessary information. Besides, unlike GUNC, we do not have to filter out non-target nodes of each task since the labels of each graph group each subset. As there is no None/otha /other class in graph labels, we have to ensure at least 2 classes are presented in each task, so we just group 2 classes into one task incrementally, resulting in 3 and 15 binary classification tasks for ENZYMES and Aromaticity, respectively.
Hyperparameter Tuning To tune the hyperparameters, we follow exactly the procedure in (Zhang et al., 2022) for EWC, MAS, GEM, TWP, and LwF. For ER, we use 10 graphs as the buffer size for each class in small datasets like ENZYMES, while big datasets like Aromaticity use 100 graphs. For our methods, based our experience with the previous task, set the loss weights 3 and y 0.5 and 0.5, respectively. In ER-LS and ER-GS-LS, we set Kn 15 as the graph size in this task is much bigger compared to the graph in the GUNC task.
Results We report the results on 2 datasets ENZYMES and Aromaticity in Table 3. We strictly follow the class-incremental setting, training procedure, and data processing introduced in a GUGC benchmark (Zhang et al., 2022). Overall, the performance of Finetune method is among the worst both in terms of AP and AF. While our ER variants still demonstrate outstanding performance. Specifically, on the ENZYMES dataset, we observe a big gap of 27.5% AF and a slight improvement of 2.5% AP compared to Finetune. On Aromaticity, big improvements happen in AP (40.0%) and AF (83.9%). Furthermore, the positive AF shows that our method can overcome not only catastrophic forgetting but also bring positive forward transfer. Finally, ER-GS-LS achieves the best performance compared to other ER-based methods. This proves the usefulness of our Local-Global Knowledge Distillation, even in the graph-level classification task.
5.4 NUNC: Social Graph Learning
Datasets We consider 3 public datasets: CoraFull (McCallum et al., 2000), Reddit (Hamilton et al., 2017), and Arxiv We strictly follow the class-incremennall setting, training procedure, and data processing introduced in NUNC benchmark (Zhang et al., 2022).
Hyperparameter Tuning To tune the hyperparameters, we follow strictly the procedure in Zhang et al. (2022) for EWC, MAS, GEM, TWP, and LwF. For ER, we use 10 nodes as the buffer size for cach class small datasets like CoraFull, while big datasets like Reddit use 100 nodes. For our methods, based on our experience with the previous tasks, we set the loss weights 3 and y as 0.5 and 0.5, respectively. In LS related method, we set K2 = 15 for CollFull and Arxiv dataset, while on Reddit we set K, = 5. We provide details of performance for some choices of K, in Table 5.
Results As shown in Table we compare mothods with other basclines in the benchmark. Overall, both Arxiv and Reddit datasets, the performance of our proposed methods including ER-GS, ER-LS, and ER-GS-LS surpasses other advanced baselines like EWC, MAS, or ER by large margin. On Corafull, our ER-GS-LS ranks highest with a substantial gap among CL basclines followed by TWP, we create a significant gap of 36.6% AP and 39.6% AF to the Finetune baseline. On top of that, our method also gain the highest scores on the rest two datasets, Arxiv and Reddit. Specifically, on Arxiv, the gap is 29.9% AP and 63.1% 1% AF to the Finetune baseline, and our method can come even close to the upper bound baseline Joint with a gap of 5.9% AP. For Reddit, our Local Structure Distillation (ER-LS) is suboptimal due to the densely connected nodes in a graph, which makes global information to be more favored in this case. This has been proven to be truly effective in the Global Structure Distillation setting. Then combining local structure information can
11

---

Published in Transactions on Machine Learning Research (11/2023)
Meshod
CoraFull
Arxiv
Reddit
AP (%)
AF (X1T
AP (16)
AF (S))
AP (P)
AF (F
Finetune
2.0
-91
1.:000
-87.0
5.1
-915
Joint
1.3
EWC
3.1 0.6
-93.0 1.2
4.9 100:000 D.D
-8.9 0.3
1...3.1..
1.2
MAS
2.2
2.1
-228
9.7
2.2
GEM
6.4+1.5
-90.0
4.9 1100-10 1.2
86.4 2.4
28.4
-71.9+4.22
TWP
13.1 #2.1
23
4.8 0.9
18.3
9.3 1.2
-1.3
LwF
-93.7
87.0
5.2
81.0
ER-GNN
2.1
1.7
26.8 1.4
2.14
7.2
22.6
5.2
ER (Ouns)
3.1118
90.9
29.3 1.6
49.6.120
71.7
28.4 1.8
ER-GS (Ours)
2.9
90.80 1.1
29.9 1.8
46.5.1..
79.7 4.7
19.9
5.0
ER-LS (Ours)
33.4
22.33
ER-GS-LS (Ours)
30.5+3.11
-55.1
31.8
0.9
1.1
75.7 1.
-212
Table 4: Performance comparisons on the task of Node-Uni Node Classification. Bold/undorlle /undddrlliine denote the best best/second best wrettpprrfrming CL technique for each column. We note that non-CL baselines (first rows) serve as theoretical bounds for reference purposes. indicates ignored AF calculation.
CoraFull
Aromaticlty
Ke
AP
[66] F
AF (%)
AP 18)
AF 12)7
AP [X]
AP (X]
AP [X]
AF [X]
35.1
3.6 1: 1.2
-92.8
31.5 1.2
-37.6 1:1:1:1 1.3
2.
33
315
10.31:0.9
321
1.2
22611
374++4
13.5+088
5
6.2
-90.5 1.4
3399
1.5
75.7 4.4
-24.2151
128112
14.7..99
26.6
2.
67.88 4.23
340 1000-00
-33.4+000 0.9
72.1 4.6
-279++41
41.
14.++1.
55.1
23.
73.7
263
20
38.2
56.28
2.8
312=12
-20.35
72.1 8.8
-27.12:4
418
0.9
Table 5: Performance comparisons on the selection of Kn values for our ER-GS-LS. Bold denote the best-performing CT. technique for each column.
be redundant or even harm global information. Therefore, the ER-GS-LS baseline performed worse than the Global Structure Distillation (ER-GS). Nevertheless, our method still has a gap of 70.6% AP and 70.3% AF with the Finetune baseline. With this result, we can confirm the usefulness of the Local-Global Knowledge Distillation in the node-level node classification task.
5.5 Model Analysis
In this section, we study the impact of the replay buffer on our distillation strategy. The hyperparameter Kn determines the contribution of the samples from the replay buffer to the local and global distillation process. K2 0 is equivalent to ER-GS variant of our method. We examine different values of K,, and report the performance of our method in Table 5. As the results demonstrate, sampling more nodes, i.e. larger Kn, in the graphs from the replay buffer brings significant improvement in both AP and AF, especially in CoraFull, we have a substantial gap of 35% in AP with K, = 15. This result confirms the effectiveness of our method when the model was given samples in previous tasks to gain the learned local and global structure representation. In general, with a higher Kn value, our model can be consistently improved; however, K, 20, the performance stops improving, which might indicate we have reasonable local information distil and increasing to higher value might. not help.
Finally, examine method with more advanced backbones besides GCN. The results consistently demonstrate that our proposed method is also compatible with GAT (Velivkovic et al., 2017) and GIN (Xu et al., 2018) backbones, improving them in continual learning tasks (see Appendix Table 7).
6 Discussion
In this paper, we systematically formulated Universal Graph Continual Learning (UGCL) and proposed a novel approach to prevent catastrophic forgetting issues under both node classification and graph classification settings. We extensively evaluated several common continual learning approaches on UGCL. The experiments show that standard regularization-base methods cannot effectively maintain prior knowledge in graphs, while rehearsal-based and inrrameter-isolatin--beeee methods can perform better. The results also highlight.
12

---

Published in Transactions on Machine Learning Research (11/2023)
that our proposed solution using memory replay and subgraph representation constraints often achieves the best results. The method almost reaches the upper-bound result. on both average accuracy and forgetting measure metrics. Our work is the first step toward the UGCL setting. In the following work, we will explore our method for largo-scale graph problems such as scene graphs and transport networks.
References
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European conference on computer vision (ECCV), pp. 139-154, 2018.
Antonio Carta, Andrea Cossu, Federico Errica, and Davide Bacciu. Catastrophic forgetting in deep graph networks: an introductory benchmark for graph classification. ArXiv, abs abs/21103.11755 2021.
Rich Caruana. Multitask learning. In Sebastian Thrun and Lorien Y. Pratt (eds.), Learning to Learn, pp. 95 133. Springer, 1998. doi: 11.1007/97--1-6115-3522-- 5. URL https://dor httts://abi.org/11..0000 org/10.1007/
978-1 44-446615599---5
Arslan Chaudhry, Puncet Kumar Dokania, Thalaiyasingam Ajanthan, and Philip H. S. Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. CoRR, abs/1801.10113 2018.
URL http:://axxat http:://axxvvv.rrge orggabs/ 10112.
Xiang Deng and Zhonglei Zhang. Graph-free knowledge distillation for graph neural networks. In IJCAT, 2021.
Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. CoRR, abs/1310.1531, 2013. URL http://arxiv www.aaawww.a..iaaaaeeaeeeeooeeee
org/abs/ abaaaba/aaaa 1//11.11..111 1531.
Arthur Douillard, Matthicu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. PODNet: Pooled Oulpuls Distillation for Small- Tasks Incremental Learning, pp. 86 102. 11 2020. ISBN 978-3-030-58564-.
Falih Febrinanto, Feng Xia, Kristen Moore, Chandra Thapa, and Charu C. Aggarwal. Graph lifelong learning: A survey. IEEE Computational Intelligence Magazine, 18:32-51, 2022.
Robert M French. Catastrophic forgetting in conncctionist networks. Trends in cognitive sciences, 3(4): 128-135, 1999.
Lukas Galke, Benodikt Franke, Tobias Zielko, and Ansgar Scherp. Lifelong learning of graph neural networks for open-world node classification. 2021 International Joinl Conference on Neural Networks (IJCNN), pp. 1 1-8, 2020.
Ross B. Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. CoRR, abs/1311.2522 2013. URL http://arxiv.ot org/abs// / 1521.2524.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Geoffrey Hinton, Jeff Dean, and Oriol Vinyals. Distilling the knowledge in a neural network. pp. 1 9, 03 2014.
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge in neural network. arXiv preprint arXiv:1503.08 02531, 2(7), 2015.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In 2019 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 831 839, 2019. doi: 10.1109 20t.///1....2000000000000
Yen Chang Hsu, Yen Cheng Liu, and Zsolt Kira. Re-evaluating continual learning scenarios: categorization and case for strong baselines. CoRR, abs/1810 12488, 2018. URL http:://are: httt::://arxx..og//bb//////////e 12488.
13

---

Published in Transactions on Machine Learning Research (11/2023)
Zehao Huang and Naiyan Wang. Like what you like: Knowledge distill via neuron selectivity transfer. 07 2017.
Zheng Huang, Kai Chen, Jianhua He, Xiang Bai, Dimosthenis Karatzas, Shijian Lu, and C. V. Jawahar. Icdar2019 competition on scanned receipt oer and information extraction. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1516-1520, 2019. doi: D..100//CDDAR.219..
David Isele and Akansel Cosgun. Selective experience replay for lifelong learning, In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018
James M Joyce. Kullback-leibler divergence. International encyclopedia of statistical science, pp. 720-722 Springer, 2011.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. CoRR, abs/1609.02907, 2016. URL http://arxiv.. org/abs/ /tt.///////////// 1609. 02907.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Bariinkk,, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521 -3266, 2017.
Hung Te and Svetha Venkatesh. Neurocoder: General-purpose computation using stored neural programs. In International Conference on Machine Learning, pp. 12204-12221. PMLR, 2022.
Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-233) 1998.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-2900 -947, 2017.
Huihui Liu, Yiding Yang, and Xinchao Wang. Overcoming catastrophic forgetting in graph neural networks.
In Proceedings of the AAAT conference on artificial intelligence, volume 35, pp. 8653-8661, 2021.
David Lopez-Paz and Aureloo Ranzato. Gradient episodic memory for continual learning. In NIPS, 2017.
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. 2018 ZEEE/CVP Conference on Computer Vision and Pattern Recognition, pp. 7765-7773, 2017.
Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3(2):127-16)) 163, 2000.
Miller Mcpherson, Lynn Smith-Lovin, and James Cook. Birds of a feather: Homophily in social networks. Annual Review of Sociology. 27:415-, 01 2001. doi: 3334100E7223356224799555444
Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jaeheung Surh, Minjoon Seo, and Hwalsuk Lee. [CORD): A consolidated receipt dataset for post-foer) parsing. In Workshop Document Intelligence at NewrlPS 2019, 2019, URL https: htaps:://operreviiie htt.:rteeeeooormro.e//11112/6
David Lopez Paz and Mare Aurelio Ranzato. Gradient episodic memory for continual learning. Tn Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Newral Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 6467 6476, 2017.
Ameya Prabhu, Philip HS Torr, and Puneet K. Dokania. Gdumb: A simple approach that questions our progress in continual learning. In European conference on computer vision, pp. 524-540. Springer, 2020.
Yujie Qian, Enrico Santus, Zhijing Jin, Jiang Guo, and Regina Barzilay. Graphic: graph-based framework for information extraction. ArXiu, 11101810.11 13083, 2018.
14

---

Published in Transactions on Machine Learning Research (11/2023)
Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. CNN features off- off-the-shelf: astounding baseline for recognition. CoRR, (hss/1403.6389 2014. URL http:://axxii.iooote http://arxii..oorabb////ss wwt.:/org//b///..//.// 6382.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy P. Lillicrap, and Greg Wayne. Experience replay for continual learning. In Newral Information Processing Systems, 2018.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv: 146160667, 2016.
Hongbin Sun, Zhanghui Kuang, Xiaoyu Yue, Chenhao Lin, and Wayne Zhang. Spatial dual-modality graph reasoning for key information extraction. CoRR, 11s//103.11470,, 2021. URL /ttp:p////xxvvoorr///// /bss/ $103.144700
Kaihua Tang, Yulei Niu, Jianqiang Huang, Jiaxin Shi, and Hanwang Zhang. Unbiased scene graph generation from biased training. In Proceedings of the LEEE/CVF conference on computer vision and pattern recognition, pp. 3716 3725, 2020.
Petar Velivkovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pictro Lio, and Yoshua Bengio. Graph allention networks. arXiv preprint arXiv: 11177111110000 2017.
Chen Wang, Yuheng Qiu, Dasong Gao, and Sebastian Scherer. Lifelong graph learning. In Proceedings of the LEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 13719-13722 2022.
Junshan Wang, Guojie Song, Yi Wu, and Liang Wang. Streaming graph neural networks via continual learning, In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp-
1515 1524, 2020.
Jia Wu, Yun Chen, Hao Zhang, Li-Dong Xiong, Hang Lei, and Si-Hao Deng. Hyperparameter optimization for machine learning models based on bayesian optimization. Journal of Electronic Science and Technology, 17(1):26-41) 40, 2019.
Zhaoping Xiong, Dingyan Wang, Xiaohong Liu, Feisheng Zhong, Xiaozhe Wan, Xutong Li, Zhaojun Li, Xiaomin Luo, Kaixian Chen, Hualiang Jiang, et al. Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. Journal of medicinal chemistry, 636) 8760, 2019.
Keyulu Xu, Weihua Hu, Jure Leskovee, and Stefanic Jegelka. How powerful are graph neural networks? 2018.
arXiv preprial ar
Yiding Yang, Jiayan Qiu, Mingli Song, Dachong Tao, and Xinchao Wang. Distilling knowledge from graph convolutional workorks 2020 IEEE/CV Conference on Compuler Vision and Pallern Recognition (CVPR), pp. 7072-7081, 2020.
Junho Yim, Donggyu Joo, Hoon Bae, and Junmo Kim. A gift from knowledge distillation: Fast optimization, network minimization and transfer lcarning. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7130 7138, 2017.
Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. 12 2016.
Xikun Zhang, Dongjin Song, and Dacheng Tao. CGLB: Benchmark tasks for continual graph learning. In Thirty-siath Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022. iaterrumidd-xxNiiiyiDDi.
URL
Fan Zhou and Chengtai Cao. Overcoming catastrophic forgetting in graph neural networks with experience replay. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 4714 4722, 2021.
15

---

Published in Transactions on Machine Learning Research (11/2023)
Appendix
Dataset
N Ngaahs
Nelusses
Nessia
Netasses/ /N Nasks
Graph-unit node classification
SROIE
973
4
4
1'
CORD
1000
30
5
WILDRECEIPT
1739
12
Craph-unit graph classification
ENZYMES
600
6
Aromaticity
3868
30
15
Node-unit node classification
CoraFull Reddit
70
35
40
Arxiv
40
20
Table 6: Description of benchmark datasets. *SROIE has #classes task 1 but is still valid because we provide an additional class named None/other in negative sampling.
Method
GCN
GAT
GIN
AP (%)
(%)
(%)
(%)
(X)
(%)) 1
Finctune
4.9:1.000
47.0 1.5
4.8 0.2
90.9 4 0.9
0.1
88.2 0.8
Joint
40.7 2.3
43.4 014+08
40.5 0.5
EWC
0.0
0.3
0.1
32.2 1.2
0.1
90.3 0.6
MAS
4.1
1.4
6.5 0.5
47.9 1.3
4.7 0.1
84.6 1.88
GEM
4.2
26
4.9003
-8.6
4.9
1.0
TWP
1.5
LwF
1.1
2.3
1.9=0.0 0.6
2.
5.0
2..
ER-GNN
26.8+ 1.4
2.14422.1
26.7
1.6
45.5+ 1.5
30.6 0.3
-52.5
ER (Ouzs)
29.3
29.6
23.3 1.2
63.8 1.8
31. 1.2
-555
ER-GS (Ows)
29.9
1.9
-653
278
-.19
ER-LS (Ouns)
33.4
1.0
36.0 1.3
24.7+ 0.8
30.9
2.8
29.41.1
40.1
ER- EIR-GS-LS (Ouns)
34.8 1. 0.9
23.9
1.1
213 23
31.5
32.612 1.2
44.5
1..1
Table 7: Performance comparisons the task of Nodo Unit Node Classification with different backboncs Arxiv-CL dataset. Bold/ /underlinedd denote the best/secone best-performing CL technique for each column. We note that non-CL baselines (first rows) serve as theoretical bounds for reference purposes. indicates ignored AF calculation.
Number replay modes per
CoraFull
Arxiv
Reddit
AP (30)
AF 130
(P)
(F
(5)
[3]
10
2.1 + 1.1
91.6
26.8
524+ 2.1
36.2
66.0+26
25.8
2..6
1000
20+12 1.2
-2.0
5.8 1.2
87.5+
23.2 1.2
-79.5+22.5
Table 8: Performance comparisons of ERGNN (Zhou & Cao, 2021) on the task of Node-Unit. Node Classification with 3 different replay buffer sizcs. We report the number of replay nodes per class because datascts have different numbers of nodes and thus need to scale differently. Bold/underlines denote the best/second best- perso-eefoomin CL technique for each column. We note that non-CL baselines (first rows) serve as theoretical bounds for reference purposcs. indicates ignored AF calculation.
16


Reviews:
[{"review_id": "2GkgFSnBDz", "reviewer": "Reviewer_STKY", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- I think that the paper is nicely written, and gives a very good explanation of the problem to be solved, existing solution, and the proposed method itself is very clear. Also, I like it that the authors discuss each method compared with in the experimental section.\n\n-The results seem promising and offer a universal approach, as suggested by the authors.\n\nWeaknesses/Questions:\n- How would your performance change if you use another GNN backbone instead of GCN?\n- How did you choose to use 3 layers? What is the performance when using more or less layers?\n- In some cases your method does very well, for example in CORD dataset, but less so in other datasets. Can you please explain why?\n- I understand that your loss contains several hyper parameters, which is fair. However I think that it would be nice if you can provide some intuition about their weighting and an ablation study of their influence on the results.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a new method for graph continual learning, that is based on mixing several techniques for this task. The method composed of three main components: Experience Replay (ER),Global Knowledge distillation (GS), and  Local Knowledge Distillation (LS).\n\nThe authors provide many experiments to show the performance of their method, that often times improves existing methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see questions in my review.", "broader_impact_concerns": "none", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LN1wXfX60h", "reviewer": "Reviewer_8eiQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n>1 The topic the paper focuses on is meaningful, and the authors try to address the deep problems in the field.\n\n>2 The paper is well structured and easy to understand.\n\n>3 The authors conducted experiments to support their ideas.\n\nWeakness\n\n>1 The abstract section does not provide sufficient background and further additions are recommended.\n\n>2 In the loss function, there are three additional constraint terms in addition to the main task loss. Although these constraint terms are tuned by parameters, I still hold the doubt whether all of these constraint terms really play a decisive role.\n\n>3 The font size of Figure 1 is small and the information conveyed is limited; it is recommended that the figure be further improved to fully demonstrate the modeling structure of the paper.\n\n>4 I hold a confusion, according to the author, in the past, the task class data information to be stored in the buffer, so if the amount of data accumulated more and more, whether the memory occupation will be a problem?\n\n>5 Experimentally, did the authors use a smaller dataset? The methods compared are more classical algorithms, but are there emerging methods that have not been compared?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper is about Universal Graph Continual Learning, which is an interesting topic. The authors propose a novel method that enables graph neural networks to excel in this universal setting. The paper is well written and well organized. However, there are several concerns in the current version of the paper that addressing them will increase the quality of this paper.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1 Improved summary writing.\n2 Adjust Figure 1.\n3 Add more experiments.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "XMDJQ0f00H", "reviewer": "Reviewer_sgKy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThis work introduces a comprehensive taxonomy for graph continual learning and presents a novel method designed to address all categories within this taxonomy. Additionally, it establishes a new benchmark for continual learning in graph-based node classification, following a class-incremental setting.\n\nWeaknesses:\n1. To underscore the practical importance of the three problems (GUGC, GNUC, NUNC), authors should include real-world examples illustrating the significance of the study\n2. How to address the disparity in node and graph sizes requires further explanation regarding how this difference impacts the buffer. \n3. It is crucial to compare the proposed approach with existing Graph Continual Learning methods. This is particularly important as most current baselines in the paper are not specifically tailored for Graph Neural Networks (GNNs).\"\n4. The notation should be more friendly. For instance, θ in Equations 2 and 3 is typically used to denote a parameter. Additionally, the alphas in Equation 1 and Equation 6 should be guaranteed to be consistent.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces the concept of Universal Graph Continual Learning (UGCL) and presents a novel approach to mitigating catastrophic forgetting in both node and graph classification scenarios. Proposed methods are evaluated against several conventional continual learning approaches applied to UGCL. Results consistently demonstrate that the solution, incorporating memory replay and subgraph representation constraints, frequently outperforms other methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. To underscore the practical importance of the three problems (GUGC, GNUC, NUNC), authors should include real-world examples illustrating the significance of the study\n2. How to address the disparity in node and graph sizes requires further explanation regarding how this difference impacts the buffer. \n3. It is crucial to compare the proposed approach with existing Graph Continual Learning methods. This is particularly important as most current baselines in the paper are not specifically tailored for Graph Neural Networks (GNNs).\"\n4. The notation should be more friendly. For instance, θ in Equations 2 and 3 is typically used to denote a parameter. Additionally, the alphas in Equation 1 and Equation 6 should be guaranteed to be consistent.\n5. Are there plans to make the Benchmark publicly accessible?", "broader_impact_concerns": "Nothing.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Overall the reviewers agree that the topic is relevant and meaningful to the TMLR community and that the contribution is technically correct. The discussion period allowed the authors to improve the paper by incorporating feedback from the reviewers.
Decision: Accept (poster)