Meta Continual Learning on Graphs with Experience Replay
Authors: Altay Unal
Abdullah Akgül
Melih Kandemir
Gozde Unal
Venue: Accepted by TMLR ()

Abstract: Continual learning is a machine learning approach where the challenge is that a constructed learning model executes incoming tasks while maintaining its performance over the earlier tasks. In order to address this issue, we devise a technique that combines two uniquely important concepts in machine learning, namely "replay buffer" and "meta learning", aiming to exploit the best of two worlds. In this method, the model weights are initially computed by using the current task dataset. Next, the dataset of the current task is merged with the stored samples from the earlier tasks and the model weights are updated using the combined dataset. This aids in preventing the model weights converging to the optimal parameters of the current task and enables the preservation of information from earlier tasks. We choose to adapt our technique to graph data structure and the task of node classification on graphs. We introduce MetaCLGraph, which outperforms the baseline methods over various graph datasets including Citeseer, Corafull, Arxiv, and Reddit. This method illustrates the potential of combining replay buffer and meta learning in the field of continual learning on graphs.

Published in Transactions on Machine Learning Research (11/2023)
Meta Continual Learning on Graphs with Experience Replay
Altay Unal
wnal21 @ilu.edu..tr
Deportment of Computer Engineering Istanbul Technical University
Abdullah Akgill
akgul sdu dk
Department Mathematics ond Computer Science University of Southern Denmark
Melih Kandemir
kandemir@imada..dd .kk
Department of Mathematics and Computer Science University of Southern Denmark
Gozde Unal
gozde. wwalditu....mmmmm
Department of Computer Engineering Istanbul Technical University
Reviewed on OpenReview: https: // ottp:///epervvee....m net/ formm? id= 8tmrh56P5W
Abstract
Continual learning a machine learning approach where the challenge is that constructed learning model exccutes incoming tasks while maintaining its performance over the carlier lasks. In order address this issue, devise technique that combines two uniquely important concepts in machine learning, namely 'replay buffer" and "meta learning', aiming to exploit the best of two worlds. In this method, the model weights are initially computed by using the current task dataset. Next, the dataset of the current task is merged with the stored samples from the earlier tasks and the model weights are updated using the combined dataset. This aids in preventing the model weights converging to the optimal parameters of the current task and enables the preservation of information from carlier tasks. We choose to adapt our technique to graph data structure and the lask of node classification graphs. We introduce MetaCLGraph, which outperforms the baseline methods graph datasets including Citeseer, Corafull, Arxiv, and Reddit, This method illustrates the potential of combining replay buffer and meta learning in the field of continual learning on graphs.
various
1 Introduction
Deep learning models have proven to perform successfully at numerous machine learning tasks including classification and regression. Despite their celebrated performances, deep learning models tend to provide poor results when they are expected to learn from a sequence of data or tasks (L.i Hoiem 2017). This process is called continual learning, which aims to train a deep learning model such that manages Icarning different lasks in order while avoiding forgetting the tasks that it has learned previously. In continual learning, model is trained in way such that it can be retrained for future tasks while preserving information from earlier tasks. The main challenge of continual learning is catastrophic forgetting (McCloskey & Cohen Goodfellow et al. 2013). which causes the model to lose obtained information from earlier tasks, leading to a performance drop in the earlier tasks.
1989
The studies addressing the catastrophic forgetting problem in continual lcarning can be divided into three groups. The first group comprises the memory based studies which either use examples (Rebuffi et al. 2017 Lopez-Paz & Ranzato, 2017 generate pseudo samples using the stored information Lavdaa et al. 2018

---

Published in Transactions on Machine Learning Research (11/2023)
Atkinson et al. 2018). The second group focuses on parameter isolation from earlier tasks such that the model can preserve its performance on earlier tasks (Mallya et al. 2018 Serra et al.
2018), while the third group is the regularization-batt based methods which propose an extra regularization term to preserve information from carlier tasks while learning on a new task (Kirkpatrick et al.
2017 Li & Hoiem 2017). While the main
problem of continual learning is capastrophic forgetting, continual learning is also challenging due to the differences in the problem setup.
Generally, continual learning has two different setups: class incremental and task incremental (De Lange et al. 2021 In class incremental learning, the individual classes are presented sequentially (Masana et al, 2022 while only the tasks are presented in task incremental learning (De Lange et al.) 2021). Class incre- mental setup requires a deep learning model to classify across the observed and current classes whereas the task incremental setup requires an indicator to separate tasks since the model predicts classes within each task in the task incremental setup. Class incremental setup is more challenging than task incremental sctup since the model has no indicator of which task tested (Zhang et al. 2022). In addition, class incremental learning becomes more challenging as the number of classes increases and the data concerning the earlier tasks is not provided.
Continual learning aims to adjust to newly incoming tasks without forgetting the older ones. Meta learning has the potential to provide benefit. to the continual learning problem as the aim of the meta learning is to provide a model with the capability to generalize itself to new tasks. Meta learning is used for scarce data regimes (Finn et al. 2017) and typically refers to processes in which a model learns how to learn. For instance, meta learning becomes an important method for few-shot learning in which deep learning models have very few samples to train with (Nguyen et al., 2019 Chen et al.) 2023). The meta learning paradigm guides the neural network model weights so that the model weights do not fit the optimal parameters solely for given task. As parameters that are optimal for each task cannot be attained, the model tends to find the set of parameters that can collectively adapt to all observed tasks. The model parameters are calculated for the current task without any update. After that, the loss which is called the meta loss, is calculated with the earlier calculated weights, and the model is updated with the meta loss. This miligates catast rophic forgetting as well as quick adaptation to new tasks. The meta learning method can also be applied graph structured data (Zhou et al. 2019 Tan et al. 2022).
Graph structured data is lype of data that consists of vertices and edges. In addition, graph structured data is non-Euclidean as it does not have a specific hierarchy and order (Asif et al.] 2021). Hence, graph structured data does not have any certain geometry. In certain real world applications such as citation networks or online social networks, data that are represented by graphs dynamically change and, hence tend to expand continuously. Due to the expansion of the data, two problems may arise: (i) new classes may emerge, and the deep learning model cannot handle the newly arrived classes since the model is not trained for those classes; (ii) on the contrary, if the model is trained with the new classes, they may lose the carlier information. Therefore, continual learning on graph structured data becomes necessary.
Continual learning on graphs has been studied recently (Liu et al. 2021 Zhou & Cao 2021 Zhang et al. 2022) and it has become an emerging field. While the studies addressing this field has similar approaches such as memory-based (Zhou & Cao 2021) or regularization- based (Liu et al.] 2021),, they also consider the properties of the graph structured data as their methods can be affected by different factors of the graph structure (Xu et al.] (020) such as topology and irregularity of the graph. In addition, newly arriving lasks alter the dynamics of the graph, interfering with the learning of the model. Therefore, continual learning on graphs diverges and it is handled concerning both the continual learning paradigm and the characteristics of the graph structured data.
In our research, we fused the best of two worlds on contimual learning for the graph data type, merging meta learning and replay buffer in a single method with learnable learning rates for the first time, to the best of our knowledge. The comparison of our method to the algorithmic familics that build on replay buffer, and meta learning is given in Figure T. Our developed method fuses the experience replay & selection method in the ER-GNN (Zhou & Cao 2021) with the meta learning & learnable learning rates from the LA-MAML (Gupta et al.) 2020) to advance on graph continual learning. In the graph continual learning problem, we focus on the task of node classification for the class incremental setup, which requires the model to 'remebber' earlier
2

---

Published in Transactions on Machine Learning Research (11/2023)
Algorithm 1 ER-GNN
Algorithm 2 LA-MAML
Algorithm 3 MetaCLGraph
for t= 1 to M do
for t= 1 M do Calculate weights for T. Meta loss with calculated weights
for t t= to M do
Calculate loss with Tr for t'=1 tot - do
Gour +- B BUT,
Calculate weights with T₃ Meta loss with calculated
Get T3r from B
Calculate loss with Te. end for
Update learning rates end for
weights on Jaue
Update learning rates
Sum Losses from T. & Extend B with T. samples end for
Extend B with T, samples end for
Figure The algorithms for ER-GNN, LA-MAML, and MetaCIGrrph (our approach). B represents the initially emply replay buffer while M is the number of tasks and Te is the current task. ER-GNN uses the replay buffer to store examples from tasks while LA-MAMT. uses meta learning for continual learning. GrapCh fuses both by using meta learning in the training phase while storing examples on the replay buffer.
classes so that the model can avoid catastrophic forgetting. Our approach has outperformed the benchmark methods various graph datasets.
2 Background
Graph Neural Networks (GNN) are constructed to process the graph structured data and their inference problems Kipp & Welling, 2016 Given a graph g = (V,E), where V = { denotes the set of nodes and = (es (igi denotes the edges of the graph, GNNs operate two phases compute the output embeddings h First, the emboddings are obtained by multiplying the weight matrices of cach layer W(I) with the embeddings calculated from earlier layers h !!-1), Next, the embeddings of the neighboring nodes to node are aggregated by using an aggregation function, and the resulting neighboring embedding is further aggregated with the embedding of the node v from the carlier layer. Scparating the embedding of node v from the neighboring embeddings allows GNNs to preserve the information from node v.
GNNs contribute to understand the interactions within adjacency matrix and reflect these interactions onto the features so that lasks related to graph structured data can be completed. GNNs are used for node and graph classification while they are also useful for the link prediction between nodes (Asif et al. 2021 Wu ct al. 2022). As the graph structured data in real world often evolve, GNNs require continual learning so that the incoming new knowledge can be learned while preserving the information from earlier tasks.
One of the common solutions to the continual learning problem is to use a replay buffer that stores examples from carlier tasks (Zhou & Cao 2021), ensuring the model's adaptability for incoming tasks without losing earlier information. Using the replay buffer, the model 'remembers" the past tasks.
Meta learning is method that aims to train model on variety of tasks Fiin et al. 2017 Given the task set T = (Ti, Ty...., T T....., the task i's set of parameters which is denoted as Φ, This process is called the inner update that is used to find the set of parameters for task i. After calculating di, the model weights 0 are updated by calculating the derivative of o, with respect to 0. This allows altering the model weights in the direction of the gradient of the loss calculated accordingly to the current task. The objective function for is given below where D and D.s represent the training and test set for task i, respectively.
TM} where M denotes the number of tasks, the model initially calculates
min #
Σ c aV (B, D.
(1)
task
3

---

Published in Transactions on Machine Learning Research (11/2023)
Meta learning ensures that the model does not converge to the optimal parameters of single task and preserves the adaptability of the model to incoming tasks. Unlike catastrophic lorgetting, meta learning aims to develop a "collective intelligence' that allows the model to perform over all observed tasks.
3 Related Work
Continual Learning mainly deals with the catastrophic forgetting problem. In order to overcome the latter, three approaches are generally used aiming to maintain the adaptability of the model to incoming tasks without any performance loss. The first approach is to isolate the parameters so that the previously obtained information would not be lost. Important parameters for the current task are determined, and those parameters are isolated such that they would not change with the future tasks, and the model can still perform for that specifie task (Serra et al., 2018 Mallya et al. 2018). The second approach is to use a regularization term such that the weight importance can be considered, and the changes in the weights that are important for the past tasks are penalized (Kirkpatrick et al.) 2017 Aljundi et al., 2018 The regularization prevents the model weights from fitting the optimal parameters of the current lask, and hence from forgetting the old tasks. The third approach is to use an additional memory that maintains some examples from the previous tasks. The training process for future tasks uses certain examples from earlier tasks, hence the model would not lose previously obtained information. The usage of additional memory allows the model to observe examples from earlier tasks while it concurrently observes the current. task. As a result, the weights would not distance themselves from those of the carlier tasks. The additional memory is used to store information about the earlier lasks to generale similar samples, or it can store examples from the tasks, and those examples can be observed during future tasks. The idea of additional memory is shown to be effective against catastrophic forgetting (Lopez-Paz. & Ranzato 2017 Zhou & Cao. 2021] Capacity saturation is also another challenge in continual learning (Sodhani et al. 2020). The architecl.re of the model defines the capacity as the components of the architecture affect learning dramatically (Mirzadeh et al. 2022: Shahawy et al. 2022). A model with fixed capacity saturates as it is kept training more tasks since the model loses its a ability to adapt to incoming tasks. This is called the stability stability-plasticity dilemma (Mermillod et al. 2013). The same dilemma exists in graph continual learning where for GNN architectures, the learning may become limited as a large number of new classes arrives. Addressing this dilemma requires a specialized focus, and while numerous studies addressed the effects of architecture on continual learning (Huan et al. 2021 Feillet et al. 2023), a similar study on GNNs has yet to be conducted.
Graph Neural Networks recently have been used in continual learning on graphs (Wang al. 2020 Since GNNs consider both the adjacency matrix and the features as mentioned earlier, they are capable of discovering the relationship between the nodes and the knowledge held by the graph. Graph continual learning is a very recent topic (Zhou & Cao 2021) where GNNs are employed for continual learning of grapl-structured data. General continual learning approaches, namely regularization based and replay based methods, are adapted to graph continual learning. Inspired by the work in the continual learning field, replay based methods are developed for graph continual learning, storing samples from earlier tasks [Zhou & Cao 2021) or generating samples by using the stored representations from carlier tasks (Wang et al.) 2022). On the other hand, regularization based met use an additional term in the loss function to adjust the updates in the model weights. In graph continual learning, regularization based methods (Liu et al. 2021
Chen et al. 2021: Cai et al. 2022 Sun et al., 2023) use a regularization term to handle the issues caused by the properties of the graph structured data such as topology and irregularity. Since graph structured data obtains objects which are connected to each other, the tasks obtained by dividing the graph provide several connections among different tasks. Hence, learning difffrent tasks can improve the performance of the GNN for earlier tasks, as different connections can be discovered with every incoming task, and this can be used remedy against catastrophic forgetting. In real-world scenarios, graphs tend to be dynamic and the boundaries between the tasks do not usually exist (Tang & Matteson, 2020). Due to the nature of the real- world scenarios, dynamic graph learning Rossi el al. 2020 Kazemi el al. 2020) can be applied to capture the dynamics of the real-world graphs. Whereas dynamic graph learning primarily concentrates on capturing the up-to-date graph reprosentations (Huang et al., (022) and does not focus on the forgetting problem because the observed data can still be accessed by the model. However, in graph continual learning, the observed data is not accessible once after the model is trained on that task, requiring the model to develop
4

---

Published in Transactions on Machine Learning Research (11/2023)
mechanisms to deal with forgetting and use the obtained knowledge. Due to the nature of the real-world graphs, the GNN model becomes more powerful with continual learning, as the same GNN model can be used for different tasks. This allows us to avoid training separate models for each task, and obtain a single model that can perform all tasks which is in line with the ultimate goal of lifelong learning.
Memory Mechanism. The implementation of memory mechanisms proves to be really effective [Knoblanch et al., 2020) as memory mechanisms allow models to revisit the earlier examples during the current task and maintain the performances from earlier tasks. Various types of memory units are explored for continual learning problem. In one approach, the implemented memory mechanism can store examples from earlier lasks such that when the model is learning fulure tasks, is also trained on the examples on the memory (Zhou& & Cao 2021 Lopez-Paz & Ranzato, 2017). This allows the model not to differentiate from the Iearned parameters of the carlier tasks. Another approach stores some representations from cach task such that those representations can be used to generate examples belonging to earlier tasks (Rebulli et al., 2017). The generated early-task examples are also used during the training of the current task. Hence, the model can preserve its performance while learning the current task with this approach.
Meta Learning is used in reinforcement learning and few-show learning (Finn et al, 2017 to perform multi (ask learning on a large set of lasks. Since the aim of continual learning is to adjust the model to newly coming tasks while maintaining its ability to perform in older ones, meta learning becomes useful in continual learning. Meta learning prevents overfitting for a specific task by using the gradicnts of optimal parameters for each task (Gupta et al., 2020). This approach is effective in mitigating catastrophic learning as it prevents the model from overfitting a specific task and the learning process of the model would not be stopped.
Our proposed approach, namely Meta Continual Learning for Graphs with Experience Replay (MetaCL- Graph), fuses meta learning with a memory mechanism to increase the efficiency in learning incoming tasks. Focused on the node classification problem in the class incremental learning setting, MetaCLGraph uses a meta learning mechanism to improve the model's ability to store already-secn information while learning the current lask and incorporates a replay buller as the memory mechanism that allows the model to observe samples from carlier tasks while learning new one. In addition, learnable parameters are introduced to the model weights as learning rates that the updates of each parameter can be different.
4 Method
4.1
Problem Setup
The goal is to prodict the classes of the nodes for collection of tasks T (T1, T2....,Taa'' }, where M is the number of lasks. The inntlnual graph problem here entails the model to learn those series of tasks in T. For each task Tü, we have a training node set D and test node set Dtst, Node classification aims to predict the right class for each node, i.e., to classify each node in the test node set Dtat into the correct class by learning the tasks using D In our graph continual learning setup, we aim to classify incoming nodes based on early observed classes which is also known as class incremental learning (Masana al. 2022).
4.2 Our Method: MetaCLGraph
MetaCLGraph is our proposed solution for graph continual learning which fuses meta learning and episodic memory through experience replay mechanism. The algorithm for our method is given in Algorithm 4 Although Zhou & Caol (021) and Gupta et all (2020) both inspire the MetaCLGraph, the transition to the latter is not straightforward, as the adaptation of the LA-MAML to graph continual lcarning while using an episodic memory includes overcoming the following obstacles such as lack of graph nodes batches and utilizing the replay buffer efficiently. The model weights are initially calculated for the current task and later updated using gradients calculated while using both stored examples and current task data. This update is called the meta update and it allows instead of fitting for just the current task T, to be able to learn the previous tasks along with current. task T, The experience replay is obtained when the model is trained
5

---

Published in Transactions on Machine Learning Research (11/2023)
Algorithm 4 MeiaCL.rraph-eetallee Algorithm
The model weights 0, learning rates a, Graph of current task 9, Buffer B, number of tasks M, learning rate for a: 7
1: for t = to M do
2:
Join buffer samples with the current task dataset: G' = = U for epoch to E do Inner update with the Ge: fast 00 a² Vo L (0.gr)
Learning rate update: a= = a' -1. Va L (8, g')
6: Meta update with the g¹: 0 = ffst m0 a²) Vojas Lu (0,9)
7: end for
8: Save samples to buffer B with coverage maximization
end for
a task and some nodes are selected by using the coverage maximization function (4.2.1) The replay buffer can also be seen as an episodic memory since the buffer stores the nodes from the current task the end of the training. The replay buffer is constructed in this way that the finished tasks can be revisited by the model using the buffer. The main reason for using the meta learning with buffer is to prevent the model weights from positioning close to the optimal parameters for certain task. Following a continual lcarning paradigm, MetaCLGraph adjusts the model weights to the incoming tasks. Revisiting the stored sample nodes allows the model to remember the earlier tasks. With this added reminder process, the model avoids adapting to only one task. is able distance parameters from the optimal parameters of the current task and also stays tuned to the carlier tasks.
4.2.1 Experience Replay
At the end of the training for each task, the additional memory is updated by selecting some examples from that particular task. The additional memory allows our model to 'remember' previons tasks as the model weights are trained for the next task. As the model also observes the selected examples, the experience replay allows our model to preserve knowledge from earlier tasks. However, the selection process for the experience replay is important as the representation of given task relies on the stored nodes.
During the sample selection, we use the coverage maximization method (Zouu & Cao 2021] determine the nodes which can represent their tasks accordingly. Coverage maximization method is used as the selection function to maximize the coverage of the embedding space by finding the distances of nodes to the other class. According to the distances, the nodes are ranked and lower ranked nodes are selected. The equation for the coverage maximization given in the following.
N (vx) = {uy dist (u, - v3) < <.)) + (u,))
(2)
where ((x)) is the set of nodes coming from different classes with distance d to node wa V(e)) and M(y,) are the labels of the nodes U; and Fy, respectively. The distance inside the embedding space minimized so that the classes can be represented inside the buffer. This reinforces the model in remembering the previous classes the selected examples in this manner cover their embedding spaces.
The experience replay allows us to construct a merged datasot as illustrated in Figure 2 When the training process for a lask is finished, several nodes are selected according to the coverage maximization function and stored inside the experience replay. After that, when a new task arrives, the nodes stored inside the experience replay are merged with the data of the current task, and mergod graph is stored. The merged graph allows us to train our model accurately since the model can keep the information learned earlier while learning the current task.

---

Published in Transactions on Machine Learning Research (11/2023)
class 1
class 3
class class T3
T1
T2
class 2
class 4
The dataset merging process is illustrated in this figure. The arrows represent the transition between the tasks while coloured nodes represent the active nodes during that task. In our approach, multiple nodes are selected the end of the task and stored inside the replay buffer. When a new task is arrived, stored nodes are merged with the current task and form merged datasel. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Figure 2:
4.2.2
Meta Learning
The meta learning is adapted MetaCLGraph in two steps as follows. When learning task T;, the model weights for the current task Tr, which are called fast weights fast are calculated using the current task data, shown in Algorithm 4 line 4 This step is called the inner update and allows the model to benefit from the direction of the gradients for the current task. After the inner update, f is introduced to the model, and the meta loss is calculated when the model is fed with the merged dataset which contains the current task data and the buffer samples from previous tasks. This is called the meta update and the model weights 0 are updated with the gradients with respect to the meta loss using fast this step, shown in Algorithm 4 line B At the end of the meta learning, the model is updated considering the gradients with respect to the current task data and also the samples from previous tasks.
4.2.3 Learning Rate Update
In addition to meta learning and experience replay setups, the MetaCL.Grapp assigns learning rales to every weight, and the assigned learning rates are also updated during the training. Every weight having a different importance for each task during the training as the model learns task helps in the preservation of the learned knowledge (Gupta et al. 2020
Using the meta learning loss calculated for the merged dataset, the learning rates for cach parameter are also updated by using the gradients. With the updates on the learning rates, the updates on our model weights become more foresecable. If the model weights require more updates in order to learn the task better, then the learning rate update would be limited. However, if some of the model weights have reached an optimal region, then their learning rate decrcases more dramatically than the learning rate of the other weights since the model would lend lo preserve that information. This update difference among the model weights allows our model to adjust the parameters according to the tasks, hence the already-acquired information is not lost. For a further detailed reading on how the computation of the gradients with respect to lcarning rates are calculated, we encourage the readers to refer LA-MAML Gupta et al. 2020
5 Experiments
The continual learning experimental setups for the GNNs are investigated, and the performance of our setup is compared with the other continual learning setups. The pipeline provided by (Zhang et al. 2022 used

---

Published in Transactions on Machine Learning Research (11/2023)
Table 1: Details of the datasets used in the experiments.
Dataset
# nodes
# edges
# features
# classes
# tasks
CiteSeer- CL
3327
9228
3703
6
3
Corafull-CL
19793
130622
8710
70
35
Arxiv-CL
169343
1166243
128
40
20
Reddit-CL 227853
114615892
602
40
20
for the experiments. The datasets, baselines, experimental settings, and results are reported and described next. We provide a reference implementation of the proposed model and the experimental pipeline
5.1 Datasets
For evaluating Meta MrraChh,, four benchmark datasets were employed: Corafull Bojchevski & Günnemann, 2017), Arxiv (Hu et al., 2021j, Reddit. (Hamilton et al. 2017 and Citeseer (Sen et al. 2008). Arxiv, Corafull, and Reddit are used in (Zhang et al. 2022) to construct a benchmark for continual graph learning, and Citeseer is additionally included for its significance in graph structured data. In this problem, we divide the datasets into tasks containing two classes for each task. This resulted in 35 tasks from Corafull, 3 from Citeseer, and 20 each from Arxiv and Reddit. For the Reddit dataset, the 41st and the last class is dropped since it only has one example. Table provides the detailed task information, and graph structures for each dataset.
To manage device memory requirements, larger graphs like Reddit are divided into batches, while the rest are processed as single batches.
5.2 Baselines
The following baselines are used in order to evaluate the MetaCLGraph.
Base model is the graph neural network without using any continual learning components. The data is simply passed to the GNN architecture and the model is trained with the provided data. There are further improvements on the GNN architecture.
Elastic weight consolidation (EWC) (Kirkpatrick et al., 2017) focuses preserving crucial model weights for each task, allowing updates only less significant weights during the learning of the new tasks to preserve carlier information.
regularization based method that eval- uates the importance of the parameters according to the sensitivity of the predictions the parameters. When new data arrives, the model weights are updated according to the activations of the network so that the important model weights for the newly acquired data can be updated.
Memory aware synapses (MAS) (Aljundi et al., 2018)
Topology-aware weight preserving (TWP) (Liu et al., 2021) is method introduced for graph continual learning which integrates weight preservation considering the graph topology. Graph structure is laken into consideration in order to prevent catastrophic forgelting. After the loss is calculated, it is regularized by calculating the importance score of the model weights according to the topological structure of the provided graph. It benefits from the properties of the graph.
ER-GNN ZZhou & Cao, 2021 uses experience replay for this problem. After learning a task Ti, sample nodes are saved to the buller with a selection function and when T2+1 is being learned, separate graphs for each learned task from k= to i} are constructed, and the GNN is trained with those graphs. The overall
8

---

Published in Transactions on Machine Learning Research (11/2023)
loss is calculated with the loss for the current task regularized by the loss calculated from the constructed separate graphs using the experience replay.
MetpCh (Ours) uses an episodic memory buller, and combines meta learning with experience replay.
Joint is the method where all the learned tasks are accessed during training. Therefore, serves as the upper bound for our setup.
EWC and MAS are the techniques focusing parameter isolation, while ER-GNN uses the additional mem-- ory as replay bulfer to store examples from earlier lasks, however, lacking any mela-learning component. TWP is regularization method that considers the topology of the graph while updating weights. Our model, the MetaCLGraph is compared with those bascline techniques for performance assossment.
5.3 Experiment Settings and Performance Scores
The experiments are conducted with a learning rate of 0.005, and each task is trained for 200 epochs. The batch size is selected as 2000 for the batched datasets. Adam optimizer (Kingma & Ba 2014 is selected as the optimizer. All methods use the graph convolutional network as the backbone GNN architecture (Kipf & Welling 2016). The selection algorithm relies on coverage maximization. The hyperparameters concerning the compared methods are obtained from the benchmark paper (Zhang et al., 2022) and its repository, the results in the benchmark are reproducible. The buffer budget is selected methods, namely the ER-GNN and the MetaCI.Graph. Given in the benchmark paper (Zhang et al. 2022 the buffer budget sets the number of samples selected for each class in the task, and the entire class or task can be stored when the bulfer budget is set. high. Setting the bulfer budget. high does not serve the purpose of continual learning since it would become likely that an entire class or task can be observed in future tasks. To avoid storing an entire class or task that would defcat the purpose of continual learning, the buffer budget is determined as 10. This is based the observation that all of the datasets contain more than 10 samples for each class. Hence, the buffer budget. becomes strict where storing entire class task is avoided
10 for the replay based
The two evaluation measures are the Average Performance (AP) and the Average Forgetting (AF) (Lpez-- Paz & Raazato 2017). AP focuses on the model's accuracy in classification of each task. It is obtained by calculating the mean accuracies of the observed tasks so far. AF considers whether or not the model preserves its performance on the observed tasks. The AF calculated as follows:
aij ak.jo Vj<k
(3)
where ojg' and ajj represent the performance scores obtained from the accuracy matrix. j and k represent the numbers of tasks where k represents the current task and j represents a certain old task when it was trained as the current task. In summary, forgetting is determined by finding the performance difference between the last performance and the performance obtained as the current task for each task. All experiments are repeated 5 times on one Nvidia RTX A4000 GPU.
5.4 Results
The experimental results are reported in Table 2 The results indicate that the proposed MetaCL outperforms the compared baselines, and achieves the best performance across all datasets terms the AP measure, and the best performance in terms of the AF measure excluding only the Reddit dataset. These results provide evidence for the fact that the proposed method allows the model to learn the tasks and stores information about the early-obeerved tasks. The MetaCLGraph outperforms the regularization based methods such as MAS, TWP, and EWC by employing its replay buffer. As mentioned carlier, regularization based methods alter the loss such that the model can preserve the obtained information. However, our proposed method uses the stored examples during the learning of the future tasks as those examples are observed during their training. Therefore, the utilization of the replay buffer allows the MctaCI.Graph outperform regularization based methods. Although MAS has better AF score for the Reddit dataset, this score is obtained since the AP score is low. Therefore, it can be deduced that the MAS method cannot learn
9

---

Published in Transactions on Machine Learning Research (11/2023)
Table 2: The AP and AF measure results across four datasets for various baselines and the MetaChGraph. The results are showing mean + standard deviation across 5 repetitions. Best performances are highlighted in bold.
Citrseer
CoraFall
Roddit
Method
AP
AF
AP
AP
AF
AP
AF
Base Method
31-19=0.15
-77.4820.22
2.21=0220
-9182200.3
4.86+0.01
-87.17=1.97
5.49=11055
-93.39#1.25
EWC
31.42:0.15
-77.4620032
15.0242.15
-81.6742.22
4.8620.01
-88. 33..5520.45
8.8821.61
-91.56i:11.5
TWP
31.37+0.08
-88.43+0.26
16.21+1.76
-77.1++1.57
4.80 02
-88. -89.89=00133
11.75+11.0
-91.59+11.81
MAS
31.58+0..07
-77.47+0.32
6.85+1.89
-88.5242.14
4.86--0.06
-86.22-0.62
12.58-44.67
236.24=8.55
ER-GNN
46.81+0.33
-51.10+0045
2.26++.32
-05.12+0.40
27 27.83+0.22
-54.67+0.17
37.64+00.64
-64.55++0.64
MetaCI.Graph (Ours
51.4811.66
-46.79:22.78
64.73:10.34
-16.86::0.38
31.76:10.74
-51.03:10.82
67.66:1:3.64
-33.01 13.86
Joint (Upper bound)
76.40+00.20
81.56-40.14
40.35-+0.85
96.33-+0.26
the tasks for the Reddit dataset, and AF score does not mean a better performance as the MAS method has not learned enough information about the observed tasks for storage. The MetaCLGraph outperforms ER-GNN which also utilizes replay buffer. Although they use the same selection function for the examples to be stored inside the replay buffer, the MetaCT.Graph exploits meta learning during the training process. During training, the optimal model parameters are determined for the current task, however, the loss is not calculated until the model observes the stored examples from the earlier lasks. Therefore, the model does not exhibit over- fietingg to the current task. On the contrary, it finds the set of parameters that can perform for the carlier tasks while learning the current task. The calculation of the meta loss allows the model adjust the parameters considering the earlier tasks, hence leads MetaCLGraph model outperforming the ER-GNN method.
In Figure 3 the performance matrices of the benchmark methods and MetaCLGraph are visualized. The performance scores are presented for all of the tasks after the observation. It can be observed for each dataset that the changes in the performance scores for MetaCI.Graph are less pronounced than those for other methods, thanks to a relatively better preservation the learned information in the former.
5.5 Computational Costs
The computational costs are also investigated and the costs are reported in Table 3 The experiments are repeated 5 limes on the Corafull dataset. The results show that the computational costs of our proposed model surpass the baseline models. Although the computational costs of our model are the highest among the baselines, its computational costs are justifiable since its performance beyond other methods and the computational costs are tested the dataset with the greatest number of tasks among the tested datasets. Corafull dataset contains the greatest number of tasks among the benchmark datasets, showing that the application of our model to other datasets would not be limited since the computational cost difference among the basclines would not be greater.
Table 3: The computational cost table for various baselines and MetaCLGraph. Method
Time per experiment (s)
Base Method
159.77
EWC
381.46
TWP
418.71
MAS
177.50
ER-GNN
234.87
MetaCLGraph
616.28
5.6 Ablation
This section focuses on the impact of meta lcarning and experience replay on the performance of MctaCL. Graph in class incremental learning. MetaCLGraph variants that either exclude meta learning or experience replay examined to understand their contributions to the model performance. For testing the effects of
10

---

Published in Transactions on Machine Learning Research (11/2023)
0
D
0-
Citeseer
100
1-
2
CoraFull
17-
17
17-
17
34
34
34
34
34-
34
17
34
34
17
17
34
17
34
Arxiv
10-
10
10-
10-
10-
19
19
10
0
-20
Reddit
10
10
0
14
19
39
19
10
10
10
19
10
10
10
Bare
EWC
MAS
TWP
ER-GNN
MetaCLGraph
Figure 3: The visualization of performance matrices for MetaCT.Graph and benchmark methods for the trained datasets. Each entry in the matrices represents the performance of the method on the column task while learning the lask on the row. The determined performance metric on the visualization is accuracy where lighter colours represent the higher score while darker colours represent the lower scores. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
experience replay, the buffer budget for the experience replay is changed. When the buffer budget is 0, this would be our approach without experience replay. On the other hand, our method without meta learning would be equivalent to ER-GNN whose results were given in Section 5.4
The results for the ablation study are given in Figure 4 Tt. can be observed that the model only using meta learning gives similar results those of the base model. Since this version of our proposed model does not store any information from carlier tasks, and only relies meta learning, it only learns the current task. Therefore, its AP and AF scores would be similar to the base model since it adjusts its parameters on only the current task and not the other tasks. It is also observed that the number of stored samples increases our model's performance significantly. The nunber of samples for each class is determined as 10 in order to avoid storing an entire class or task inside the buffer. The results show that when the capacity of the buffer is increased, the performance is improved. However, increasing the capacity too much may cause storing fasks, and storing the tasks would not serve the purpose of continual learning. Although some examples were stored inside the buffer, continual learning setup requires the continuous data flow from earlier tasks to be stopped once the training for task is finished since the finished tasks become unreachable after the training. However, when the tasks start to be stored in the replay buffer, the model will be retrained on the already visited tasks. Therefore, the continual learning setuu is no longer valid, and the model would be trained in a traditional supervised learning setup. This is observed in average forgetting as well, Since all examples from a class may have been stored with a high buffer budget, the model is trained on a task more than once. Therefore, the average forgetting rate bocomes greater than zero due to the fact that it has been trained more than specified epochs and the model's performance on that task is improved by meta learning
11

---

Published in Transactions on Machine Learning Research (11/2023)
25
50
75
100
Average Performance
Average Forgetting
80
-20
-40
40
20
0
-100
10 15
20
25
35
5
10
20
25
35
Number tasks
Number tasks
Figure 4: The effect of buffer budget on MetaCI.Graph is shown for AP (left) and AF (right). As the buffer budget increases, both the model performance and forgetfulness are improved due to the increase in the stored number of samples per class. (For interpretation of the references to color this figure legend, the reader is referred the web version of this article.)
and additional epochs. Therefore, a lititt on the number stored examples the buller imposed so that the integrity of continual learning can be maintained.
6 Conclusion
Summary. We present the MetaCLGraph for graph continual learning which merges the meta learning paradigm with the use of replay buffer. MetaCLGraph is compared with the baseline continual learning methods and graph continual learning methods. The experiments provide evidence to our hypothesis that the meta learning paradigm improves the efficiency of the replay buffer and mitigates the catastrophic forgetting problem by conserving the obtained information from carlier tasks. is observed that the MetaCLGraph oulperforms the corresponding baselines in terms of average performance and average forgetting measures.
Broad Impact. GrapC framework can be used in applications for expanding networks where new members and connections are emerging and their characteristics discovered. The replay buffer of our model can be altered with different. lype of memory that store the class representations instead of class examples in order to address data security concerns.
Future Work. Our work has shown that the utilization meta learning improves the efficiency of using memory mechanism such as a replay buffer. Expanding from MetaCT.Graph, futurr research direction could be changing the memory mochanism from a replay buffer subconscious memory that stores representations for every observed class.
Limitations. Storing examples from earlier tasks creates dilemma for the memory based solutions in terms of data sharing. Our method performs storing limited number of examples, hence, limiting the buller capacity and not storing the entire incoming task or class. By limiting the number of examples stored in the replay buffer, our method serves well for the purposes of continual learning while alleviating data privacy and ethical concerns. Another limitation the increase in the number of tasks may limit the application of the proposed model since the overall complexity of the model is already higher than its counterparts.
12

---

Published in Transactions on Machine Learning Research (11/2023)
References
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European conference on computer vision (ECCV), pp.
139-159--- 154, 2018.
Nurul A. Asif, Yeahia Sarker, Ripon K. Chakrabortty, Michael J. Ryan, Md. Hafiz Ahamed, Dip K. Saha, Faisal R. Badal, Sajal K. Das, Md. Firoz Ali, Sumaya I. Moyeen, Md. Robiul Islam, and Zinat Tasneem. Graph neural network: A comprehensive review on non-euclidean space. IEEE Access, 9:60588-6000 2021. doi: 10.1109 S.t0/ACCESS.2000/ 302.3771777
catastrophic forgetting problem in deep neural networks. arXiv preprint arXiv:1802.0888 2018. Craig Atkinson, Brendan McCane, Lech Szymanski, and Anthony Robins. Pseudo-recursal: Solving the
Aleksandar Bojchevski and Stephan Günnemann. Deep gaussian embedding of graphs: Unsupervised indue- tive learning via ranking. arXiv preprint
2017.
Jie Cai, Xin Wang, Chaoyu Guan, Yateng Tang, Jin Xu, Bin Zhong, and Wenwu Zhu. Multimodal continual graph learning with neural architecture search. In Proceedings of the ACM Web Conference 2022, pp. 2022.
Lisha Chen, Sharu Theresa Jose, Ivana Nikoloska, Sangwoo Park, Tianyi Chen, Osvaldo Simcone, et al. Learning with limited samples: Meta-learning and applications to communication systems. Foundations and Trends in Signal Processing, 17(2):79 208, 2023.
Xu Chen, Junshan Wang, and Kunqing Xie. Traficstream: A streaming traffic flow forecasting framework based on graph neural networks and continual learning. arXiv preprint arXiv:2106.062 2021.
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. 1EEE tronsactions on pattern analysis and machine intelligence, 44(7):3366-338)) 3385, 2021.
Eva Feillct, Grégoire Petit, Adrian Popescu, Marina Reyboz, and Céline Hudclot. Advisil-a class-inrremental learning advisor. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1400-2409 2023.
Chelsca Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126-11355 PMLR, 2017.
Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient- based neural networks. arXiv preprint arXiv:1312.622 6211, 2013.
Gunshi Gupta, Karmesh Yadav, and Liam Paull. Look-ahead meta learning for continual learning. Advances in Neural Information Processing Systems, 33:1 18:1558-11555 11598, 2020.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs, 2021.
Shenyang Huang, Vincent Francois-Tavel, and Guillaume Rabusseau. Understanding capacity saturation incremental learning. In Canadian Conference on AI, 2021.
Xuanwen Huang, Yang Yang, Yang Wang, Chunping Wang, Zhisheng Zhang, Jiarong Xu, Lei Chen, and Michalis Vazirgiannis. Dgraph: large-scale financial datasct for graph anomaly detection. Advances in Newral Information Processing Systems, 35:22765 22777, 2022.
Seyed Mehran Kazemi, Rishab Goel, Kshitij Jain, Ivan Kobyzev, Akshay Sethi, Peter Forsyth, and Pascal Poupart. Representation learning for dynamic graphs: A survey. The Journal of Machine Learning Research, 21 (1):2648 2720, 2020.
13

---

Published in Transactions on Machine Learning Research (11/2023)
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv:1418 12. 12269880 2014.
arXiv preprint
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint ar Xin: 1009. 02907, 2016.
James Kirkpatrick, Razvan Pascamu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, (14(13):32213322) 2017.
Jeremias Knoblauch, Hisham Husain, and Tom Diethe. Optimal continual learning has perfect memory and is up-hard. In International Conference on Machine Learning, pp.
PMLR, 2020.
Frantzeska Lavda, Jason Ramapuram, Magda Gregorova, and Alexandros Kalousis. Continual classification learning using generalive models. arXiv preprinl ar arXie: 1810: 10612, 2018.
Zhizhong and Derek Hoiem. Learning without forgetting. IEEE transactions on pollern analysis and machine intelligence, 40(12):2935 2947, 2017.
Huihui Liu, Yiding Yang, and Xinchao Wang. Overcoming catastrophic forgetting in graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 8653-8661 2021.
David Lopez-Pat Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Arun Mallya, Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting single network to multiple tasks by learning to mask weights. In Proceedings of the European conference on computer vision (ECCV), pp. 67-82, 2018.
Mare Masana, Xialei Liu, Bartlomiej Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost Van De Wei- jer. Class-incremental learning: survey and performance evaluation on image classification. IEFE Trans- actions on Pattern Analysis and Machine Intelligence, 45(5):5513 5533, 2022.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin. The stability dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects, 2013.
Seyed Iman Mirzadch, Arslan Chaudhry, Dong Yin, Timothy Nguyen, Razvan Pascanu, Dilan Gorur, and Mehrdad Farajtabar. Architecture matters in continual learning. arXic preprinl arXiv: 2022.
Binh D Nguyen, Thanh- Mhanh-Toan Do, Binh X Nguyen, Tuong Do, Erman Tjiputra, and Quang Tran. Over- coming data limitation in medical visual question answering. In Medical Image Computing and Computer Assisted Intervention-MIICC.
2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part IV 22, pp. 522 530. Springer, 2019.
Sylvestro-Alvise Rebuffi, Alexander Kolesníkov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2014 2017.
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs. arXiv preprint arXiv:2006. 10637, 2020.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collec- tive classification in network data. Al magazine, 29(3):93 93, 2008.
Joan Scrra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International conference on machine learning, pp. PMLR, 2018.
14

---

Published in Transactions on Machine Learning Research (11/2023)
Mohamed Shahawy, Elhadj Benkhelifa, and David White. A review on plastic artificial neural networks: Exploring the intersection between neural architecture search and continual learning. ar preprinl arXiv:2206.05000 2022.
Shagun Sodhani, Sarath Chandar, and Yoshua Bengio. Toward training recurrent neural networks for lifelong learning. Neural computation, 32(1):1-35, 2020.
Li Sun, Junda Ye, Hao Peng, Feiyang Wang, and S Yu Philip. Self-supervised continual graph learning in adaptive riemannian spaces. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 4633 4642, 2023.
Zhen Tan, Kaize Ding, Ruocheng Guo, and Huan Liu. Graph few-shot dlas--nccemmnnal learning. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Minóng, pp. 987- 996, 2022.
Binh Tang and David S Matteson. Graph-based continual learning, arXiv preprint arXiv:2007.04811 2020.
Junshan Wang, Guojie Song, Yi Wu, and Liang Wang. Streaming graph neural networks via continual learn- ing. In Proceedings of the 29th ACM international conference on information & knowledge management, pp. 1515 1524, 2020.
Junshan Wang, Wenhao Zhu, Guojie Song, and Liang Wang. Streaming graph neural networks with gen- erative replay. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1878 1878-1888, 2022.
Lingfei Wu, Peng Cui, Jian Pei, and Liang Zhao. Graph Neural Networks: Foundations, Prontiers, and Applications. Springer Singapore, Singapore, 2022.
Yishi Xu, Yingxue Zhang, Wei Guo, Huifeng Guo, Ruiming Tang, and Mark Coates. Graphsail: Graph struc- ture aware incremental learning for recommender systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 2861-2868, 2020.
Xikun Zhang, Dongjin Song, and Dacheng Tao. Cglb: Benchmark tasks for continual graph learning. In Thirt Conference on Neural Information Processing Systems Datasels and Benchmarks Track, 2022.
Fan Zhou and Chengtai Cao. Overcoming catastrophic forgetting in graph neural networks with experience replay. In Proceedings of the AAAT Conference Artificial Intelligence, volume 35, pp- 4714-4722, 2021.
Fan Zhou, Chengtai Cao, Kunpeng Zhang, Goce Trajcevski, Ting Zhong, and Ji Geng. Meta-gnn: On few- shot node classification in graph meta-learning. In Proceedings of the 28th ACM International Conference on Information and Knowledge Monagement, pp. 2357-2360, 2019.
15


Reviews:
[{"review_id": "oR2cAs4GUW", "reviewer": "Reviewer_yCNC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. Graph neural networks with the two strategies are shown to have superior performance in controlled experiments. \n\nWeakness: \n\n1. The work is somewhat incremental as it combines two known techniques. \n\n2. The proposed method is general enough for most learning tasks, but I don't know why it is tested on graph learning tasks. If the authors want to claim the value of the proposed new method, then it should be tested over general learning tasks. \n\n3. The learning tasks in the experiment section seem to be all made-up problems. Could you elaborate more on the practical value of solving these problems? I am also wondering whether there are actual continual learning problems in this field. \n\n4. The writing is a little bit redundant. In particular, some content is repeated (e.g. the discussion of meta learning in sections 2 and 3; the discussion of the meta update at the beginning of section 4.2 and in section 4.2.2). The first three sections can greatly be compressed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The submission proposes to combine \"replay buffer\" and \"meta learning\" to address continual learning problems on graph data. In particular, these techniques are applied to the training of graph neural networks in node classification tasks. With \"replay buffer\", nodes from previous classes are stored in the buffer for later training. With \"meta learning\", the model is updated by considering both the current task and previous tasks. The proposed model is applied to multiple graph classification tasks and shows superior performance over several competing methods.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "While it is hard to suggest any changes to increase the novelty of the work, I would like to see a more rigorous study of the proposed method. \n\n1. Can you evaluate the proposed method in general learning tasks. If better performances are observed across multiple learning domains, the work still deserves a publication. If the proposed model only perform better on graph learning tasks, then a thorough explanation is needed. \n\n2. The writing of the paper should be compressed. The text introduction of the proposed algorithm can be clearer. For example, the following text is hard to understand. The following text quoted from the submission contains examples of writing issues. Questions are in brackets. \n\n\"First, the model is trained with Dtri in order to determine (*optimize until convergence?*) the parameters for the current task Ti. The parameters for the current task are determined (*some repetition of the previous sentence*) in order to find the direction of the optimal parameters (*how the \"optimality\" is defined?*) for that task. In order to achieve that (*what?*), the model weights are calculated (*how? Are model weights the same as parameters?*) and stored. After that, the gradients are calculated (*which gradients, and how are they computed?*) and the model weights (*are they from the stored weights? If not, then how are gradients calculated*), which are called as fast weights θfast, are updated, shown in Algorithm 4, line 4.\"", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "56QSzGcdnP", "reviewer": "Reviewer_JeUv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n* The idea of using the replay buffer often used in the continual learning setup over the meta-learning framework is interesting.\n* The proposed MetaCLGraph significantly outperforms relevant baselines.\n* This paper is well-written and easy to follow. \n\n\n### Weaknesses\n* The main weakness is that this work has an incremental contribution against existing works, namely ER-GNN and LA-MAML. In particular, this work directly utilizes the algorithms from both prior works (i.e., using the replay buffer while performing the meta-learning) and does not have clear motivations on why using both of them is necessary. \n* The comparison against the LA-MAML model is missing; meanwhile, this LA-MAML, which is the basis model of the proposed MetaCLGraph, is the direct and important baseline to compare.\n* The computation costs for performing the meta-learning might be very high especially when the number of tasks is large, in contrast to using other continual learning techniques (e.g., reply buffer or regularization), due to multiple optimizations (i.e., inner and meta updates). It may be beneficial to discuss them. \n* The coverage maximization method (Section 4.2.1), which is an important notion when selecting and storing samples in the memory, is not clearly described.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper tackles the challenge of continual learning where new classes and tasks continually emerge over time, with the idea of using the reply buffer that can save and learn with the important samples and the meta-learning that can learn model parameters over the distribution of tasks. The authors evaluate the proposed MetaCLGraph on the node classification task of graph-structured data, showing that it can not only improve the performance but also mitigate the forgetting issue.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address the aforementioned weaknesses in marginal contribution against existing works, comparisons against baselines, computational costs, and clarity.", "broader_impact_concerns": "The authors clearly discuss the broader impact and limitations of the proposed work in Section 6.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JTwNcUsc04", "reviewer": "Reviewer_s6gH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "There are multiple concerns about this paper, including some major ones.\n\n\n1. The so-called meta learning in this work is not real meta learning. It is simply optimizing the models with two steps: 1. optimize with the current task data. 2. update the model again with the extended dataset consisting of both current task data and stored data. While meta-learning is learning to learn and typically find an optimal starting point for the model so that the model can easily adapt to any task. Therefore, the motivation of this 'meta-learning' operation is questionable, why not directly train the model on the extended dataset containing both the current task data and the stored data, so that the model optimization considers both new and old data? \n\n2. Continual learning on graphs has been extensively studied recently, but the related work does not reflect this and only mention few works on this, only Zhou&Cao is mentioned. I would recommend the authors to carefully investigate the recent related works.\n\n3. The writing can be significantly improved. For example, some sentences are very redundant, like the snetences in 4.2.2: ' First, the model is trained\nwith Dtri in order to determine the parameters for the current task Ti. The parameters for the current task are determined in order to find the direction of the optimal parameters for that task. In order to achieve that, the model weights are calculated and stored.' All these three sentences are talking about one same thing.\n\n4. The so-called meta learning in this work is not real meta learning. It is simply optimizing the models with two steps: 1. optimize with the current task data. 2. update the model again with the extended dataset consisting of both current task data and stored data. While meta-learning is learning to learn and typically find an optimal starting point for the model so that the model can easily adapt to any task.\n\n5. In line 5 of Algorithm 4, how is the gradients with respect to the learning rate alpha calculated?\n\n6. Nodes are stored in the memory. However, when replaying them, does GNN only take in single nodes? This is weird because GNNs will aggregate information from a neighborhood containing multiple nodes (message passing). If one single node is stored, its neighbors are not stored and the message passing cannot work.\n\n7. If the learning rates monotonically decreases, the proposed model will be less and less capable to adapt to new tasks.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes to model consisting of experience replay and meta learning for the continual learning problem. Also, the learning rate of each parameter is gradually decreased according to the gradient on the parameter.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "All the concerns mentioned above should be addressed to improve the paper.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CZMQWpNarZ", "reviewer": "Reviewer_gQoq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "First, I will list the strengths of this work: \n\n1. The authors presented MetaCLGraph, a combination of replay buffer and meta learning. It is interesting to combine these two previous ideas and shows a new approach for graph continual learning.\n\n2. MetaCLGraph showed strong empirical performance on the node classification task across four graph datasets in the class-incremental learning setting based on the CGLB benchmark. \n\n3. Extensive ablation studies are conducted to validate the components in this approach as well as effect of number of tasks. \n\nNow I will discuss some weaknesses and rooms for improvement (more details see requested changes):\n\n1. the related work discussion on continual learning is incomplete. Outside of catastrophic forgetting, continual learning also faces the challenge of capacity saturation which was pointed out in various previous work. This discussion should be added. \n\n2. missing discussion on how graph continual learning is related to another research area: dynamic graph or temporal graph learning. The idea of adding more nodes / classes over time is exactly the setting seen in dynamic graphs and some discussions should be added. \n\n3. some parts of the writeup seems confusing and counter-intuitive which needs improvement", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this work, the authors presented MetaCLGraph for graph continual learning which combines meta learning and experience replay techniques. By using experience replay / a memory module, the proposed model is able to mitigate the impact of catastrophic forgetting which is a major challenge in continual learning. With meta learning, fast prototype of model weights are first computed on the new task without any updates and then using a meta loss to learn a joint representation for previous tasks. This work targets the node classification task for the class-incremental learning setting on graphs. The contributions can be summarized as follows: \n\n- combining ideas from replay buffer and meta learning to form a new learning method for graph continual learning \n\n- strong empirical performance on current graph continual learning benchmark datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Following on the weaknesses, here are my requested changes to the paper, [important] means it is critical to my recommendation for acceptance, [minor] means it will improve the paper:\n\n1. [important] the authors briefly mentions the dynamic nature of real world graphs (or dynamic graphs), in Section 3. \"In real-world scenarios, graphs tend to be dynamic and the boundaries between the tasks do not usually exist\". How is class-incremental learning on graphs different from classification on temporal graphs? Would it be more natural to learn continuously on temporal graphs as new classes naturally arises over time rather than the current setup of splitting the classes manually into different tasks? I think some discussions on this point would be helpful. I will also provide some references for temporal graphs[1,2,3].\n\n[1] Huang, Xuanwen, et al. \"Dgraph: A large-scale financial dataset for graph anomaly detection.\" Advances in Neural Information Processing Systems 35 (2022): 22765-22777.\n\n[2] Rossi, Emanuele, et al. \"Temporal graph networks for deep learning on dynamic graphs 2020.\" URL https://arxiv. org/abs (2006).\n\n[3] Kazemi, Seyed Mehran, et al. \"Representation learning for dynamic graphs: A survey.\" The Journal of Machine Learning Research 21.1 (2020): 2648-2720.\n\n2. [important] In the related work, the authors only discussed the challenge of catastrophic forgetting in continual learning. However, this is not the only challenge, another significant challenge is capacity saturation where the architecture of a neural network needs to continue to grow / adapt when more information arrives otherwise it would be limited in learning by the fixed architecture. This is discussed in various previous work, including [1,2,3]. The same challenge exists in graph continual learning where if the GNN architecture is fixed, then it is possible that its learning may become limited as large number of new classes arrives.\n\n[1] Huang, Shenyang, Vincent Francois-Lavet, and Guillaume Rabusseau. \"Understanding Capacity Saturation in Incremental Learning.\" Canadian Conference on AI. 2021.\n\n[2] Shahawy, Mohamed, Elhadj Benkhelifa, and David White. \"A Review on Plastic Artificial Neural Networks: Exploring the Intersection between Neural Architecture Search and Continual Learning.\" arXiv preprint arXiv:2206.05625 (2022).\n\n[3] Mirzadeh, Seyed Iman, et al. \"Architecture matters in continual learning.\" arXiv preprint arXiv:2202.00275 (2022).\n\n3. [minor] \"it allows our model not to fit the optimal parameters\nfor a given task\". It seems counter-intuitive to not want model to find the optimal parameters for a given task. The authors should explain this part better in Section 4.2. The goal of continual learning is to find an optimal parameter set for the joint set of tasks at time t. \n\n4. [minor] the explanation in Section 4.2.2 about the model training on the merged dataset is very confusing. Is theta_fast trained with only data from the new task or the merged dataset? how are the merged dataset utilized to compute loss?", "broader_impact_concerns": "There is no concerns on the ethical implications of the work. The authors also addressed this in the broader impact section of the paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers generally agree that this is a sound submission, with good experimental results, while they have also found the idea of combining replay buffer with meta learning interesting. However, the paper still has some small issues. I am thus recommending accept with a minor revision, and I request that the final version of the manuscript implements any remaining items promised during the discussion period and carefully considers the final reviewers' comments, particularly the items listed below:

- provide the overall computational complexity of the proposed method, compare it to the complexity of the baselines and discuss whether this could limit the applicability of the proposed method to large scale datasets (Reviewer JeUv).

- revise the related work section on continual learning on graph data to better acknowledge work that has appeared in the past few years and also discuss how the present work is different from closely related works (Reviewer s6gH).

- improve the quality of the presentation and clarity throughout the paper (Reviewers yCNC and s6gH).
Decision: Accept (poster)