An Efficient Memory Module for Graph Few-Shot Class-Incremental Learning
Authors: Dong Li
Aijia Zhang
Junqi Gao
Biqing Qi
Venue: NeurIPS 2024 poster (2024)
Keywords: graph few-shot class-incremental learning
graph incremental learning
graph few-shot learning
Primary area: graph_neural_networks

Abstract: Graph incremental learning has gained widespread attention for its ability to mitigate catastrophic forgetting for graph neural networks (GNN). Conventional methods typically require numerous labels for node classification. However, obtaining abundant labels is often challenging in practice, which makes graph few-shot incremental learning necessary. Current approaches rely on large number of samples from meta-learning to construct memories, and heavy fine-tuning of the GNN parameters that lead to the loss of past knowledge. These result in significant memory consumption and loss of past knowledge information, respectively. To tackle these issues, We introduce Mecoin to efficient construct and Preserve memory. For efficient storage and update of class prototypes, Mecoin use Structured Memory Unit (SMU) to cache prototypes of the seen classes and update new class prototypes through interaction between nodes and the cached prototypes by Memory Construction module(MeCo). Besides, to avoid extensive parameter fine-tuning and forgetting, we introduce a Memory Representation Adaptive Module called MRaM to separate the learning of prototypes and class representations and use Graph Knowledge Interchange Module (GKIM) to injects past knowledge information into GNN. We analyze the effectiveness of our paradigm from the perspectives of generalization error, and discuss the impact of different distillation methods on model performance through experiments and VC-dimension. By comparison with other related methods, we validate that Mecoin achieves higher accuracy and lower forgetting rate.

An Efficient Memory Module for Graph Few-Shot Class-Incremental Learning
Dong Li Aijia Zhang", Junqi Gao Biqing Qi
Department of Electronic Engineering, Tsinghua University, Shanghai Artificial Intelligence Laboratory. Institute for Advanced Study in Mathematics, Harbin Institute of Technology,
1 School of Mathematics, Harbin Institute of Technology {arvinlee826, zhangai jia665, gjunqi97, qibiqing7/@mmail.com
Abstract
Incremental graph learning has gained significant attention for its ability address the catastrophic forgetting problem in graph representation learning. However, traditional methods often rely on a large number of labels for node classification, which is impractical in real-world applications. This makes few-shot incremental learning on graphs a pressing need. Current methods typically require extensive training samples from meta-learning to build memory and perform intensive fine- muning of GNN parameters, leading to high memory consumption and potential loss of previously learned knowledge. To tackle these challenges, we introduce Mecoin, an efficient method for building and maintaining memory. Mecoin em- ploys Structured Memory Units to cache prototypes of learned categories, as well as Memory Construction Modules to update these prototypes for new categories through interactions between the nodes and the cached prototypes. Additionally, we have designed Memory Representation Adaptation Module to store prob- abilities associated with each class prototype, reducing the need for parameter men-tuning and lowering the forgetting rate. When a sample matches its corre- sponding class prototype, the relevant probabilities are retrieved from the MRaM. Knowledge is then distilled back into the GNN through a Graph Knowledge Dis- tillation Module, preserving the model's 's memory. We analyze the effectiveness of Mecoin in terms of generalization error and explore the impact of different distil- lation strategies on model performance through experiments and VC-dimension analysis. Compared to other related works, Mecoin shows superior performance in accuracy and forgetting rate. Our code is publicly available on the Mecoin- GFSCIL
Introduction
In the field of graph learning, conventional methods often assume that graphs are static However, in the real world, graphs tend to grow over time, with new nodes and edges gradually emerging. For products
example, in citation networks, new papers are published and cited; in e-commerce, are introduced and updated; and in social networks, new social groups form as users join. In these dynamic contexts, simply updating graph representation learning methods with new data often leads to catastrophic forgetting of previously acquired knowledge.
Despite numerous methods proposed to mitigate the catastrophic forgetting problem in Graph Neu- ral Networks(GNNs 2 3 [. a critical and frequently neglected challenge the scarcity of labels for newly introduced nodes. Most current graph incremental learning methods [5] combat catas- trophic forgetting by retaining a substantial number of nodes from previous graphs to preserve prior knowledge. However, these methods become impractical in graph few-shot class-incremental learn-
"Correspondigg author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

ing(GFSCILL scenarios due to the limited labeled node samples. Some methods 77 I employ reg- ularization to maintain the stability of parameters critical the graph's 's topology. Yet, in GFSCIL, the label scarcity complicates accurate assessment of the relationship between parameter importance and the underlying graph structure, thus increasing the difficulty of designing effective regularization strategies. Consequently, the issue of label scarcity hampers the ability of existing graph continual learning methods to effectively generalize in graph few-shot learning scenarios.
GFSCIL presents two critical challenges: H)How can we empower models to learn effectively from limited samples? How can we efficiently retain prior knowledge with imited samples? While the first challenge has been extensively explored in graph few-shot learning contexts 8 9), this paper focuses on the second. Currently, discussions on the latter issue within GFSCIL are rela- tively scarce. Existing methods IO, IT] primarily focus on enhancing models' ability to learn from few-shot graphs and preserve prior knowledge by learning class prototypes through meta-learning and attention mechanisms. However, to strengthen inductive bias towards graph structures and learn more representative class prototypes, these approaches require caching numerous training samples from the meta-learning process for subsequent GFSCIL tasks. This caching not only consumes significant memory but also imposes substantial computational costs. Furthermore, these meth- ods extensively adjust parameters during prototype updates, risking the loss of previously acquired knowledge [2] [3). These challenges underscore the need for innovative strategies that maintain ef- achieved through minimal memory footprint, high computational efficiency, and limited parameter adjustments.
ficiency without compromising the retention of valuable historical data- data-achieved
To address the aforementioned challenges, we introduce Mecoin, an efficient memory construc- tion and interaction module. Mecoin consists of two core components: the Structured Memory Uni for learning and storing class prototypes, and the Memory Representation Adaptive Module(MRaM) for dynamic memory interactions with the GNN, To effectively leverage graph structural information, we design the Memory Construction module(MeCs) within the SMU. MeCs facilitates interaction between features of the input node and prototype representations stored in the SMU through self-attention mechanisms, thereby updating sample representations. Then, it utilizes the local structural information of imput nodes to extract local graph structural information and com- pute local graph structure information matrix (raapinfGr.....f..... The updated sample representations and GraphInfo are used to calculate class prototypes for the input nodes. These newly obtained proto- types compared with those stored in the SMU using Euclidean distance to determine whether the input samples belong to seen classes. If a sample belongs to a seen class, the corresponding prototype in the SMU remains unchanged. Conversely, if sample belongs to an unseen class, its calculated prototype is added to the SMU.
To address catastrophic forgetting in class prototype learning caused by model parameter updates, we introduce the MRaM mechanism within Mecoin. MRaM stores probability distributions for each class prototype, allowing direct access via indexing once input nodes are processed through McCs and corresponding class prototypes are retrieved from SMU. This mechanism separates class pro- totype learning from node probability distribution learning, effectively mitigating forgetting issues caused by parameter updates. Additionally, to enhance the maintenance and updating of knowledge base, we integrate a Graph Knowledge Interaction Module (GKIM) within MRaM. GKIM transfers information about identified classes from MRaM to GNN and extracts knowledge of new classes from GNN back MRaM, facilitating continuous knowledge updating and maintenance.
Contributions. The main contributions of this paper are as follows: i) We design Mecoin, a novel framework that effectively mitigates catastrophic forgetting in GFSCIL by integrating the SMU and MRaM; ii) We design the SMU, which efficiently learns class prototypes by facilitating interaction between node features and existing class prototypes, while extracting local graph structures of input nodes; iii We propose the MRaM, which reduces the loss of prior knowledge during parameter fine-tuning by decoupling the learning of class prototypes from node probability distributions; iv) We analyze the benefits of separating class prototype learning from node probability distribution learning, considering generalization error bounds and VC dimensions. We also explore how different MRaM-GNN interaction patterns affect model performance; v) Through extensive empirical studies, demonstrate Mecoin's 's significant advantages over current ofate-of-the-art methods.
2

---

GNN Encoder
Classifior
Graph data
(a) Graph neural network
Tirrcctonn with di Knew oigh
Updating losses knowledge
Memury losses koeping memory
(4) GKIM
ectracts local graph Reveuni CORROOOI
(b) MeCs
Protocype Regresenlations
matching
class
repussenattin
Memory Representation pdive Module
(c) Structured M emory Unit
Figure 1: Overview of the Mecoin framework for GFSCIL.. (a)Graph neural network: Consists of a GNN encoder and classifier(MIPP) pre-trained by GNN. In GFSCIL tasks, the encoder parameters are frozen. (brutuuctd Memory Unit: Constructs class prototypes through MeCs and stores them in SMU. Me)rry Representation Adaptive Module: Facilitates adaptive knowledge interaction with the GNN model.
Notation
Let Go = (Vo, En) be the base graph with node set Vo and edge set Eg. We consider T temporal snapshots of Go, each corresponding to a GFSCIL task session. Denote S (So, .t...., Sr . as the set of sessions including the pre-training session Sp. and C (Co, cI...., Cr) as the family of class sets within each session. The graph under session S 1,2,....
denoted as G
1 (V,, &.), with node feature matrix and adjacency matrix represented by X; = (x!,... x T E R R/A/xd and A; E dRd respectively. For session S, let X E IR [C| and Y be the features and corresponding labels the training nodes respectively, where K is the sample size each class in C₁, thus defining C,-way K-shot GFSCIL task. Let 3/ be the label space of session S and we assume that the label spaces of different sessions are disjoint, i.c., 3 ny, = if * j. Our goal to learn model fo across successive sessions that maintains strong performance in the current session and also retains memory of the past sessions.
3 Efficient Memory Construction and Interaction Module
In this section, we present a comprehensive overview of our proposed framework. Mecoin. Unlike previous methods that are hampered by inefficient memory construction, low computational effi- ciency, and extensive parameter tuning-which often lead the loss of prior knowledge-Mecoin enhances the learning of representative class prototypes. It achieves this by facilitating interaction between input nodes and seen class prototypes stored in the SMU, while integrating local graph structure information of the input nodes. Moreover, Mecoin decouples the learning of class proto- types from their corresponding probability distributions, thereby mitigating the loss of prior knowl- edge during both the prototype learning and classification processes. Fig. iillsstrates the architec- ture of Mecoin.
3.1 Structured Memory Unit
To acquire and store representative class prototypes, we develop SMU within Mecoin. Let M be the set of class prototypes {my, my...., m,-1), where ICr-al refers to the total number of classes learned from the past T - sessions, with each m; E IR* E [n]). For current session S7,

---

the training node features X are encoded through a pre-trained GNN model J
Zr = = (AT, Xt),
where Zy [((Cr|K)xx, During the training process, the parameters of the pre-trained GNN, denoted as Φ remain fixed, and the training set used for pre-training is excluded from the sub- sequent GFSCIL tasks. To mitigate the significant memory usage resulting from caching meta- learning samples and incorporate as much graph structural information as possible, we design MeCs within the SMU. MeCs merges graph structure information from the past sessions with that of the current session by interacting the encoded features of node Zr with the class prototypes in M. Specifically, McCs firstly facilitates the interaction between Zr and the SMU-stored prototypes Mo.T-1 (mo, m mm-1) through self-attention mechanism:
ZZ (MoT (MoT-1W Vm
H7 == softmax
(2)
where We E A WK E Rkxm Wv E R are learnalle weight matrices and Hr E R((Cr(K)xh, Subsequently, to reduce memory consumption, we perform dimensionality reduction on Hy through Gaussian random projection, resulting in Hy E [[C]]())rr where 0 < < k < h. MeCs then extracts local graph structure information of GT via the vanilla self-attention X and preserves this information in the GraphInfo GT:
GT ATTENTION(*#).
(3)
where Gr E (R(CC(K)x((-r) and is integrated with Hy by concatenation:
U (GT, Hr).
To determine the class prototypes under session Sr, we perform k-means clustering on U?:
Mf := (-means(Ur))
(5)
where My E Crlxk In addition, to improve the utilization of graph structural information, optimize Ur by the edge loss Ledge defined as follows:
Lndge
| fr (X#) UT (
(6)
For each node feature(i.e. each row) in X,, We identify the nearest class prototype by minimizing the Euclidean distance:
m = argmin EM ||u! m2il2,
where ul is the 1-th row in U,.
3.2 Memory Representation Adaptive Module
In the traditional GFSCIL paradigm, adapting evolving graph structures requires continuous learn- ing and updating of class prototypes based on the current task's 's graph structure, alongside classifier retraining. This process, which involves adjusting parameters during class prototype learning, can lead the classifier to forget information about past categories, exacerbating the model's 's catastrophic forgetting. To address this, we introduce the MRaM within Mecoin. MRaM tackles this challenge by decoupling class prototype learning from class representation learning, caching probability dis- tributions of seen categories. This separation ensures that class prototype updates don't affect the model's 's memory of probability distributions for nodes in seen categories, thus enhancing the stabil- ity of prior knowledge retention [14 15 [6].
To maintain the model's memory of the prior knowledge, we introduce GKIM into MRaM for infor- mation exchange between Mecoin and the model. Specifically, let = (Po, Pi.... Pn- } denote the class representations learned from the GNN and stored in MRaM, where each p. corresponds to its respective class prototype m, For a node feature x, from the seen classes with its class repre- sentation Px,, let px, ML be the category predicted probabilities learned from GNN and MLP, then GKIM transfers the prior knowledge stored in Mecoin the model through distillation that based on the memory loss function:
N,
N,
Ememory
KL(p: Ip
MLP
1
Σ
pa log
P P
(8)
N
1=1

---

where N, is the total number of samples from the seen classes.
Furthermore, update the category representations of classes in MRaM, GKIM updates the newly learned knowledge from the model to Mecoin via distillation with the update loss function in 9 For any node feature x, from the unseen classes, its category representation is randomly initialized as Pk.. and its predicted probability vector is px_ Through distillation, p is updated incorporate the new classes' representations which are thus stored Mecoin.
Lupdate
N. Σ Nu
1
MLP p:) (
1 N
Σ
N.
Px.
MLP
log
Px,
MLP
P
where N2 is the total number of samples from the unseen classes. The overall loss of Mecoin consists of the MLP classification loss Cetss Ledges Lmmeory and Cupdate.
Meccoi
Lilss Ledge + Lmmmory + Lupdate-
(10)
3.3 Theoretical Analysis
In this section, we analyze the advantages of Mecoin and how it improves models generalization from the perspective of generalization error. Besides, we also provide insights from the viewpoint of VC-dimension by comparing GKIM with non -onaaaamettiic methods and MLPs in classifying category representations stored MRaM, and distilling Mecoin as a teacher model with GNN model.
What are the advantages of Mecoin over other models? In few-shot learning, the limited training data and the overall samples in current session ST often have different distributions, which leads to overfitting. However, Mecoin can mitigate overfitting since it has lower bound of generalization error than other corresponding models (Thm. I). Before introducing our theoretical result, we first supplement some notations. For the current session Sy, let Xr and V/r be the sample space and label space of X respectively, and Tr- {(x,, yf)) be the training samples. Let be a hypothesis class, f& and f E F be Mecoin and other corresponding models trained on Tr, and assume that the inputs Xy Xy undergo distributional shifts through any function ge. Then by comparing the generalization error bounds of f with f. we demonstrate in Thm. (proof in Appendix B.1) that Mecoin excels in distributional shifts, indicating its stronger generalization capability. The result is given in 1 which derived from theorem 3.1 in [12].
Theorem 1: For any model f E (fM,
(fM,f) fi trained Tr, denote R its generalization error, then there exists a constant such that for any 8 > O, the following holds with probability at least 1 8:
2 2ln(e/8)
R RSR. RB. f++
+
(II)
N
VT)] -(I(((()x))))))) C.T Bj = N Eyr.m 2ETr, D. (xe(f(xt )y Jxl. Cm. vi. = =yrl + cc/n(22/8)/22 where {Eihi are ii.dd random variables uniformly laking values (-1,1}, and is the loss function Leccomm YT), Cyr yx Xr x Dr ly- yr, ar d(ka (x), m.), Tyr E [N] X E yt YTJ, C. fx XT m argm d(: m))). MM = m [[M]]] [ITT| 2 1} and ka is the MeCs operation.
where R. = EyredT Emeryy IT EE (xx)),
Why use GKIM to interact with GNN models? Unlike traditional knowledge distillation techniques that rely on high-capacity teacher models, GKIM uses probability distributions stored in MRaM to preserve node of seen class distributions. This prevents knowledge loss in GNN during teacher model training. For unseen classes, the classifier learns and stores their probability distributions in MRaM. Notably, updating unseen class distributions and extracting seen class distributions can also use non-parametric methods [1]. Thus, we must examine GKIM's advantages over conven- tional distillation and non-parametric methods. We analyze the VC dimension when category repre- sentations MRaM are distilled into models, comparing scenarios where MRaM, non-parametric methods, and multi-layer perceptrons (MLPs) act as teacher models.
5

---

Theorem 2: If n class representations are selected from GKIM, i.c. n is the input size of GNN, then the VC dimension of GKIM is:
O(
voting classifier
VCD O(P
MLP
(12)
Opp2n²H H²)
GKIM
where H is the number of hidden neurons in MLP and denotes the number of parameters of GNN.
In the above theorem, we take the common voting classifier as an example for non-parametric meth- ods. It averages the category representations stored in MRaM and updates them through backpropa- gation. While this method has lower complexity, it is highly sensitive to the initialization of category representations due to its reliance solely on the average while computing the prediction probabilities. When using MLP to update category representations, its parameters require fine -uninn to accommo- date new categories that will lead to the forgetting of prior knowledge. contrast, GKIM exhibits a higher VC-dimension compared to the other two methods, indicating superior expressive power. Additionally, GKIM selectively updates category representations in MRaM locally, preserving the model's s memory of prior knowledge.
4 Experiments
In this section, we will evaluate Mecoin through experiments and address the following research questions: Q1). Does Mecoin have advantages in the scenarios of graph few-shot continual learning? Q2). How does MeCs improve the representativeness of class prototypes? Q3). What are the advantages of GKIM over other distillation methods?
4.1 Graph Few-Shot Continual Learning (Q1)
Experimental setting. We assess Mecoin's performance on three real-world graph datasets: Cora- Full, CS, and Computers, comprising two citation networks and one product network. Datasets are split into a Base set for GNN pretraining and a Novel set for incremental learning. Tab provides the statisties and partitions of the datasets. In experiments, we freeze the pretrained GNN pa- rameters and utilize them as encoders for subsequent few-shot insremeaat learning on graphs. For CoraFull, the Novel set is divided into 10 sessions, each with two classes, using a 2-way 5-shot GFSCIL setup. CS's Novel set is split into 10 sessions, each with one class, adopting a 1-way 5-shot GFSCIL configuration. Computers' Novel set is segmented into 5 sessions, each with one class, also using a I-way 5-shot GFSCIL setup. During the training process, the training samples for session O are randomly selected from each class of the pre-trained testing set, with 5 samples chosen as the training set and the remaining testing samples used as the test set for training. It is worth noting that for each session, during the testing phase, we construct a new test using all the testing samples of seen classes to evaluate the model's 's memory of all prior knowledge after training on the new graph data. Our GNN and GAT models feature two hidden layers. The GNN has consistent dimension of 128, while GAT varies with 64 for CoraFull and 16 for CS and Computers. Training parameters are set at 2000 epochs and a learning rate of 0.005.
Baselines. 1) Elastic Weight Consolidation (EWC) [17-immpsess quadratic penalty on weights to preserve performance on prior tasks.
Table 1: Information of the expermental datasets.
2) Learning without Forgetting (LwFF) [18]- retains previous knowledge by minimizing the discrepancy between old and new model out- puts. 3) Topology-aware Weight Preserving (TWP) -idientifiss and regularizes parame- ters critical for graph topology to maintain task performance. 4) Gradient Episodic Mem- ory (GEM) [ an episodic memory to adjust gradients during learning, preventing loss increase from prior tasks. 5) Experience
Dataset
CoraFull
CS
Computers
Nodes
19,793
18,333
13.752
Edges
126,842
163,788
491.722
Features
8,710
6805
767
Labels
70
15
10
Training set
50
Novel set
20
10
5
Replay GNN (ER-GNN) 3 incorporates memory replay into GNN by storing key nodes from
prior tasks. 6) Memory Aware Synapses (MAS) [20] evaluates parameter importance through sensi-

---

Table 2: Comparison with SOTA methods on CoraFull dataset for GFSCIL.
Methoo
Average
T
ERGNN
GCN
73.43
77.6 64
29.38
32.14
20.93
21.84
16.13
15.28
15.2
11.77
11.
62.15
29.55
GAT
73.43
77.64
29.38
32.14
20.93
21.84
16.13
15.28
15.20
11.77
11.30
62.13
29.55
Mecoin
GCN
45.92
49.49
12.79
11.91
MAS
GCN
GAT
73.43
09.06
66.64
46.40
44.45
25.20
48.13
18.40
37.08
19.44
13.09
47.04
48.58
11:20
29.16
11.47
44.26
9.86
49.13
8.83
46.39
K.27
65. 65.16
22.67
49.99
22.
1.W
GCN
73.43
45.92
25.75
17.82
14.16
10.95
11.38
9.90
8.08
7.73
65.7
22.26
13:24
EWC
GCN
GAT
69.06
73.43
45,92
48.5
25.58
26.48
25.34
18.52
20.39
20.27
14.59
18.59
11.41
14.86
22.71
11.59
10.04
19.7
21.31
7.79
12.55
7.95
56.51
65.68
27.14
22.53
TWP
GCN
70.54
44.90
26.71
20.65
24.16
23.29
14.56
18.77
14.37
19.37
12.48
14.48
56.06
25.54
HAG-Meta
87.62
82.78
79.01
74.5
67.65
63.38
60.00
57.36
55.5
52.98
51.47
36.15
66.57
Geometer
72.23
61.75
43.34
37.61
36.24
32.79
29.97
22.11
21.01
17.34
16.32
55.91
35.52
OURS
GCN
75.53
82.18
81.56
77.47
71.53
70.43
69.70
68.78
68.07
66.70
62.10
61.36
20.82
70.90
400
Geometer OURS
Average Aocuracy
67-
300
Memory Usagee
Accuraccy
300
200
100-
HAG-Maa Grameter (N)
HAG-Mels Geometer (c)
(b)
Figure 2: The comparative analysis of the mean performance, accuracy curves and memory uti- lization of HAG-Meta, Geometer and Mecoin across 10 sessions CoraFull, conducted under the experimental conditions delineated their respective publications.
tivity to predictions, distinct from regularization-base EWC and TWP. 7) HAG-Meta [10] GFS- CIL approach, preserves model memory of prior knowledge via task-level attention and node class prototypes. 8) Geometer [[]] adjusts attention-basee based prototypes based on geometric criteria, ad- dressing catastrophic forgetting and imbalanced labeling through knowledge distillation and biased sampling in GFSCIL.
Experimental results. We execute 10 tests for each method across three datasets, varying the ran- dom seed, and detailed the test accuracy in Tables 2 and Key findings as follows:1) Mecoin surpasses other benchmarks on CoraFull, CS, and Computers datasets, exemplified by 66.72% improvement over the best baseline on the Computers dataset across sessions. Mecoin maintains a performance edge and exhibits reduced forgetting rates, substantiating its efficacy in mitigating catastrophic forgetting. 3) Geometer and HAG-Meta perform poorly our task setting, likely because, to ensure a fair comparison, the models do not use any training data from the meta- learning or pre-training processes during training. The computation class prototypes in these two methods heavily relies on the graph structure information provided by this data. Additionally, in Fig[]] we compare the results of Geometer and HAG-Meta tested under the experimental conditions given in their papers with the results of Mecoin the CoraFull dataset. The experimental results in- dicate that our method still achieves better performance and forgetting rates than these two methods under low memory conditions, demonstrating the efficiency of our method in terms of memory.
4.2 MeCs for Memory (Q2)
We conduct relevant ablation experiments MeCs across the CoraFull, CS, and Computers datasets:1) Comparing the impact MeCs usage on model performance and memory retention; 2) Analyzing the effect of feature of node interaction with class prototypes stored in SMU on model

---

Table 3: Comparison with SOTA methods on Computers dataset for GFSCIL.
Acc. in each session (%)
Average
Method
PD
ACC f
backbone
0
3
4
5
GCN
100.00
50.00
33.33
25.00
20.00
16.67
83.33
40.83
ERGNN
GAT
100.00
50.00
33.33
25.00
20.00
16.67
83.33
40.83
GCN
100.00
50.00
33.33
25.00
20.00
16.67
83.33
40.83
Mecoin
GAT
100.00
50.00
33.33
25.00
20.00
16.67
83.33
40.83
GCN
100.00
50.00
33.33
25.00
20.00
16.67
83.33
40.83
MAS
GAT
100.00
50.00
33.57
37.39
25.90
21.64
78.36
44.75
GCN
100.00
50.00
33.33
25.00
20.00
16.67
83.33
40.83
LWF
GAT
100.00
50.00
33.33
25.00
20.00
16.84
83.16
40.86
GCN
100.00
50.00
33.33
25.00
20.00
16.67
83.33
40.83
EWC
GAT
100.00
50.00
33.33
25.00
20.00
16.67
83.33
40.83
GCN
100.00
50.00
33.33
25.00
20.00
16.67
83.33
40.83
TWP
GAT
100.00
50.00
33.33
25.00
20.00
16.67
83.33
40.83
HAG-Meta
20.00
16.67
14.28
12.5
11.11
10.00
10.00
14.09
Geometer
59.40
33.00
23.57
18.56
15.40
13.20
46.20
27.19
GCN GAT
89.65
88 .49
69.77
73.32
74.67
68.43
21.22
77.39
OURS
91.44
91.44
54.94
68.73
73.64
67.66
23.78
74.64
Table 4: Comparison with SOTA methods on CS dataset for GFSCIL.
Method
Acc. in each cession (%)
PD
Avenge
hackhone
2
3
7
8
9
10
ACC1
ERGNN
GCN
50.00
33.33
25.00
20.00
16.67
14.29
23.34
12.5
11.11
10.00
14,02
85.99
27.90
Mecoin
GCN
100
50.00
33.33
25.00
20.00
16.67
14.29
12.50
11.11
10.00
12.12
87.88
27.73
GAT
100
50.00
38.71
25.00
20.00
17.03
14.85
13.98
14.17
21.52
18.09
81.91
30,30
MAS
GUN
12.36
GAT
64.36
63.92
LWE
GCN
100
50.00
33.33
25.00
20.00
16.73
14.29
12.
11.11
16.36
15.44
84.55
28.60
GAT
100
50.00
33.33
24.71
36.28
36.40
28.92
28.84
21.23
29.12
32.51
67.49
38,30
GCN
50.00
33.33
25.00
20.00
16.67
14.29
12.5
11.11
10.00
12.12
87.88
27.73
31.65
TWP
GCN
GAT
100
100
50.00
50.35
33.33
33.33
25.00
25.18
38.14
20.00
47.14
16.67
39.34
14.29
12.43
27.67
11.04
30.52
45.11
16.81
$2.02
15.03
84.97
47.98
44.44
28.6
HAG Meta
20.00
16.67
14.29
12.50
11.11
10.00
9.09
8.33
7.69
7.14
6.66
13.34
11.24
Geometer
24.05
19.03
24.87
OURS
GCN
GAT
98.07
97.83
91.09
95.13
90.75
91.01
68.02
73.75
71.79
79.95
77.58
82.21
75.38
74,13
72.17
75.06
72.52
69:18
63.01
65.92
59.66
62.21
35.62
38.41
77.96
77.50
performance and memory retention:3) the impact of using graphInfo on model perfor- mance and memory retention. We conduct a randomized selection of ten classes from the test set, extracting 100 nodes per class form clusters with their corresponding class prototypes within the SMU. Subsequently, we assess the efficacy of MeCs and its constituent elements on model perfor- mance and the rate of forgetting. method to compute class prototypes. In experiment, we use GCN as backbone. The experimental parameters and configurations adhered to the standards established in prior studies.
cases where the MeCs was not utilized, we employ the k-means
Experimental Results. The results of the ablation experiments on CoraFull and CS are shown in Fig and results on other datasets shown in Appendix.A. We deduce several key findings from the 1i The class prototypes learned by MeCs assist the model in learning more representative prototype representations, demonstrating the effectiveness of the MeCs method. 2) The difference in accuracy between scenarios where graphInfo is not used and where there is no interaction with class prototypes is negligible. However, the scenario without graphInfo exhibits a higher forgetting rate, indicating that local graph structural information provides greater assistance to model memory. This may be because local structural information reduces the intra-class distance of nodes, thereby helping the model learn more discriminative prototype representations.
8

---

23
30
-4
40
-6
Leer
Goopltak
-00
-00
-21
-0
.50
<55
The outcomes of GKIM when conducting the few-shot continuous learning task on the CoraFull, Computers and CS datasets. The results are presented sequentially from left to right: GKIM with full capabilities, GKIM where node features not interact with class prototypes in the SMU, GKIM without GraphInfo and GKIM without MeCs The experimental results for CoraFull are shown the above figure, the results for Computers are in the middle and the results for CS are in the figure below.
Figure 3:
4.3 GKIM for Memory (Q3)
In our experimental investigation across three distinct datasets, we scrutinized the influence of vary- ing interaction modalities between MRaM and GNN models on model performance and the propen- sity for forgetting. We delineate three distinct scenarios for analysis: 1) Class representations stored in MRaM are classified using non-parametric methods and used as the teacher model interact with the GNN. 2) Class representations stored in MRaM are classified using MLP and Mecoin is employed as the teacher model to transfer knowledge the GNN model. 3) GNN models are de- ployed in isolation for classification tasks, without any interaction with Mecoin. In this experiment, we use GCN as backbone. Throughout the experimental process, model parameters were held con- stant, and the experimental configurations were aligned with those of preceding studies. Due to space constraints, we include the results on the Computers dataset the Appendix.A.
Experimental Results. It is worth noting that due to the one-to-one matching between MRaM and SMU via non-maramttric indexing, there is no gradient back- back- proppggtopannn between these two components. This implies that updates to MRaM during training do not affect the matching between node features and class prototypes. Our experimental findings are depicted Fig and they yield several key insights: 1) GKIM outperforms all other interaction methods, thereby substantiating its efficacy. 2) The second interaction mode exhibits superior performance compared to the first and third methods. This is attributed to the MLP's higher VC-dimension compared to the parammreettic

---

OURS receroda
OURS
Accureyy
10:4
CoraFull
CS
CoraFull
CS
Figure 4: Left 2 columns: Line charts depict the performance of models across various sessions on the CoraFull and datasets when using different distillation methods; Right 2 columns: His- tograms illustrate the forgetting rates of different distillation methods on these two datasets.
method, which gives it greater expressive power to handle more complex sample representations. However, the use of MLP results in a higher forgetting rate compared to the non non-parametric method. This is because when encountering samples from new classes, the parameters of MLP undergo changes, leading to the loss prior knowledge. For third method, extensive parameter fine- tuning leads to significant forgetting of prior knowledge. Method one performs less effectively than GKIM, primarily because GKIM, with its larger VC-dimension, ensures that the process of updating class representations does not affect representations of other classes stored in MRaM.
Conclusion
Current GFSCIL methods typically require a large number of labeled samples and cache extensive past task data to maintain memory of prior knowledge. Alternatively, they may fine-tune model parameters at the expense of sacrificing the model's adaptability to current tasks. To address these challenges, we propose the Mecoin for building and interacting with memory. Mecoin is made up of two main parts: the Structured Memory Unit (SMU), which learns and keeps class prototypes, and the Memory Representation Adaptive Module (MRaM), which helps the GNN preserve prior knowledge. To leverage the graph structure information for learning representative class prototypes, SMU leverages MeCs to integrate past graph structural information with interactions between sam- ples and the class prototypes stored in SMU. Additionally, Mecoin introduces the MRaM, which separates the learning of class prototypes and category representations avoid excessive parameter fine-tuning during prototype updates, thereby preventing the loss of prior knowledge. Furthermore, MRaM injects knowledge stored Mecoin into the GNN model through GKIM, preventing knowl- edge forgetting. We demonstrate our framework's 's superiority in graph few-shot continual learning with respect to both generalization error and VC dimension, and we empirically show its advantages in accuracy and forgetting rate compared to other graph continual learning methods.
Acknowledgement
This work is supported by the National Science and Technology Major Project (2023ZD0121403) We extend our gratitude the anonymous reviewers for their insightful feedback, which has great- lycontributed to the improvement of this paper.
References
[1] Junshan Wang, Guojie Song, Yi Wu, and Liang Wang. Streaming graph neural networks via continual learning. In Proceedings of the 29th ACM international conference information & knowledge management, pages 1515-1524, 2020.
[2] Hongbo Bo, Ryan McConville, Jun Hong, and Weiru Liu. Ego-graph replay based continual learning for misinformation engagement prediction. 2022 International Joint Conference on Newral Networks (IJCNN), pages 01-08. IEEE, 2022.
10

---

[3] Fan Zhou and Chengtai Cao. Overcoming catastrophic forgetting in graph neural networks with experience replay. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 47 4722, 2021.
[4] Yilun Liu, Ruihong Qiu, and Zi Huang. Cat: Balanced continual graph learning with graph condensation. In 2023 IEEE International Conference on Data Mining (ICDM), pages 1157- 1162. IEFE, 2023.
[5] Xikun Zhang. Dongjin Song, and Dacheng Tao. Ricci curvature- graph sparsification for continual graph representation learning. IEEE Trunsactions on Neural Networks and Learning Systems, 2023.
[6] Xikun Zhang, Dongjin Song, and Dacheng Tao. Hierarchical prototype networks for continual graph representation learning. IEEE Transactions on Pullern Analysis and Machine Intelli- gence, 45(4)::622-4463, 2022.
[7] Huihui Liu, Yiding Yang. and Xinchao Wang. Overcoming catastrophic forgetting in graph neural networks. Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 8653-8661, 2021.
[8] Victor Garcia and Joan Bruna. Few-shot learning with graph neural networks. arXiv preprint 1rXiv:171.004044 2017.
[9] Kaize Ding, Jianling Wang, Jundong Li, Kai Shu, Chenghao Liu, and Huan Liu. Graph proto- typical networks for few-shot learning on attributed networks. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 295-304, 2020.
[10] Zhen Tan, Kaize Ding, Ruocheng Guo, and Huan Liu. Graph few-shot dass-incremental learn- ing. In Proceedings of the fifteenth ACM international conference on web search and data mining, pages 987-996, 2022.
[11] Bin Lu, Xiaoying Gan, Lina Yang, Weinan Zhang, Luoyi Fu, and Xinbing Wang. Geometer: Graph few-shot class- dass-incremental learning via prototype representation. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 1152-1161, 2022.
[12] Frederik Träuble, Anirudh Goyal, Nasim Rahaman, Michael Curtis Mozer, Kenji Kawaguchi, Yoshua Bengio, and Bernhard Schölkopf. Discrete key-value bottleneck. In International Conference on Machine Learning, pages 34431 144453445. PMLR, 2023.
[13] Biqing Qi, Xinquan Chen, Junqi Gao, Dong Li, Jianxing Liu, Ligang Wu, and Bowen Zhou. Interactive continual learning: Fast and slow thinking. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 11882-18892 2024.
[14] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interfer- ence. arXiv preprint arXiv:/810.1 /910, 2018.
[15] Biqing Qi, Junqi Gao, Xingquan Chen, Dong Li, Jianxing Liu, Ligang Wu, and Bowen Zhou. Contrastive augmented graph2graph memory interaction for few shot continual learning. arXiv preprint arXiv:2403.cm 04140, 2024.
[16] Biqing Qi, Junqi Gao, Xinquan Chen, Dong Li, Weinan Zhang and Bowen Zhou. Sr-cis: Self-reflective incremental system with decoupled memory and reasoning. arXiv preprint wwxiiv2408.019700 2024.
[17] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, An- drei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinst Barwinska, al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 1 14(13):3521-322) 2017.
[18) Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-2447 2017.

---

[19] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learn- ing. Advances in neural information processing systems, 30, 2017.
[20] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuyte- laars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European conference on computer vision (ECCV), pages 139-154, 2018.
[21] Marek Karpinski and Angus Macintyre. Polynomial bounds for ve dimension of sigmoidal and general pfaffian neural networks. Journal of Computer and System Sciences, 54(1):169-176 1997.
[22] Stefanie Jegelka. Theory of graph neural networks: Representation and learning. In The International Congress of Mathematicians, 2022.
12

---

A Ablation Experiments
In Section 4.2, presented the relevant ablation experiments the CoraFull and CS dataset. Below are the experimental results of the model on the Computers datasets.
beer, ampplests proteeypes Eopaliat,
AYL
Figure 5: From left right are the results of GKIM, without using GraphInfo, node features not interacting with class prototypes in SMU and without using McCs when performing the graph small-sample continuous learning task in the Computers dataset, four randomly selected categories from sessionl and 400 randomly selected samples from the four categories clustered at the class center of the class prototypes bit class centers obtained from the learning during the training process.
The experimental results in this section similar to those the CoraFull and CS dataset, and relevant experimental analyses can be referred to in the main text. Additionally, we designed re- lated ablation experiments the dimensions of GraphInfo and its integration position with node features. Detailed results are shown in tab 3 6 From these results, we can draw the following conclusions: It is evident from the tables that concatenating GraphInfo the left of node features with a dimension of 1 yields the best performance; 2yThe model's performance decreases as the dimension of GraphInfo increases, while the forgetting rate generally exhibits decreasing trend. This indicates that local graph structural information provides some assistance to the model's mem- ory. When the dimension of class prototypes is fixed, an increase in the dimension of GraphInfo implies less graph structural information extracted from prototypes of seen classes in past sessions. Consequently, MeCs integrates less graph structural information, making it difficult for the model to learn representative class prototypes, thereby limiting the model's performance.
Table 5: The table counts the effect of GraphInfo dimension change on the model's performance on the CoraFull dataset as well as the forgetting rate when concatenating GraphInfo to the left-hand side of the node features.
Method
Acc. in each session (%)
PD
Average ACC1
0
4
5
7
8
9
10
GraphInfao13-11
80.49
71.8
67.94
65.39
71.83
GraphInfo(12-2)
79.66
79.33
76.61
71.96
71.02
69.96
68.62
68.15
67.04
64.19
63.76
15.9
70.94
Grahhnfo(11-3)
80.25
80.19
76.35
71.49
70.43
69.35
67.26
66.5
64.38
63.65
16.6
70.69
GraphInfo(1044)
79.7
79.12
76.14
71.53
70.4
68.77
66.94
67.13
66.01
63.39
62.48
17.22
70.15
GraphInfo(9-5)
79.63
79.04
76.14
72.15
70.97
69.88
68.55
68.1
67.03
63.97
63.44
16.19
70.81
GraphIn(o(8-6)
79.19
78.88
75.48
71.22
70.16
69.15
68.15
67.17
66.44
63.01
62.68
16.51
70.14
Gruphnnfa(7-T)
79.28
78.73
75.54
71.5
70.76
69.87
68.11
67.82
66.62
63.28
62.88
16.4
70.40
Graphafo((-8)
78.45
78.37
70.11
68.93
65.68
15.96
69.72
Gruphfnfu(599)
9)
77.04
76.21
72.67
68.99
67.57
66.59
65.54
65.27
64,
61.55
61.3
15.74
67.90
GraphInfo(4-10)
76.45
75.19
67.8
66.74
64.34
61.35
67.53
GraphInfo(3-11)
77.07
76.46
74.24
69.34
68.58
67.86
66.21
63.65
64.28
61.56
60.23
16.84
68.33
Gmphhnfo(4411)
73.33
72.87
70.34
65.7
65.18
64.73
62.84
62.66
61.83
57.88
57.48
15.85
64.99
Graphnffo(1-33)
74.93
74.59
72.38
67.3
66.13
64,86
63.3
62.65
61.79
59.36
$8.73
16.2
66.00
Furthermore, for Section 4.3, the relevant ablation experiment results on the Computers dataset can be found in Figure 8.
13

---

Table 6: The table counts the effect of GraphInfo dimension change the model's performance the CoraFull dataset as well as the forgetting rate when concatenating GraphInfo to the right-hand side of the node features.
Method
Acc. in each session (5))
PD
Average ACC
8
Gruphlnfo(13-1)
79.33
79.06
75.65
71.5
70.77
69.7
67.98
67.55
66.36
64.15
63.52
15.81
70.52
(raphinfa(12-2)
78.22
77.77
75.36
71.15
70.05
69.1
67.49
67.14
66.18
63.43
62.53
15.69
69.86
Graphtnfo(11-3)
78.65
78
75.09
70.95
09.94
68.79
67.28
66.9
66.04
63.48
63.06
15.59
09.83
GraphInfo(104)
78.95
78.
75.62
71.09
70.18
69.12
67.57
67.2
66.04
63.6
63.14
15.81
70.08
GruphInfo(9-5)
GruphInfo(8-6) ()-)
79.53
79.05
75.02
71.19
70.13
68.96
67.3
66.78
65.92
62.61
62.3
17.23
69.89
Graphlnf((--T)
79.36
78.96
75.19
71.39
70.51
69.39
67.86
67.6
66.69
63.99
62.96
16.4
70.35
Graphnnf(6-8)
78.94
78.27
75.2
71.16
70.06
68.62
67.26
66.82
66.
62.09
62.36
16.58
69.78
GraphIn(o(5-9)
77.84
77.25
74.27
70.05
68.72
68.05
66.49
66.35
65.70
62.51
61.64
16.2
68.99
Graphlafo(4100) 10)
76.28
75.31
72.81
68.77
67.84
66.54
65.14
65.29
64.42
61.36
60.88
15.4
67.69
Gmahhnffa(3-11)
76.78
75.94
73,53
69.02
67.76
67.19
65.44
64.41
63.95
61.23
60.09
16.09
67.81
Gmphhnfa((-12))
71.92
71.51
64.46
60.04
GraphInfo(1-13)
76.16
75.55
72.37
68.92
67.41
66.67
64.8
64.71
63.87
60.7
$9.68
16.48
67.35
76353
OURS
80
no-mecoin
70
non-para mip
60
Accuracy
PD
40
21.01
12.02
12.61
3
OURS
no-mecoin non-para
mlp
Figure 6: HAG-Meta, Geometer, and Mecoin methods' average performance, performance curves, and memory consumption over 10 sessions on the Computers dataset under conditions set forth in their papers.
B Proof of Theorems
B.1 Proof of Theorem 1
Proof. By instituting CYT with Cy. C. with Cx. IYT with IV TXT with Th m, with KG,, and k, with
kg o Zas our proof is similar to the proof of theorem 3.1 in [12] (please refer to [22 for more details).
For f Mf f). through similar procedure in the proof of [12,, we obtain the coarse upper bound
14

---

of generalization eтror R:
1 N
R - Ez,e (a
Σ {((x)))))))
N
N Σ
Ty
E - C
ErEVT melM
(13)
Bel((((xr) C.T)
Σ BET
{(((x)),))))
N
YrEAT mEI
2 ln(e/0)
+ov
N
We then further conduct the tighter generalization bound for f and f separately based on Eq. Consider the case of f for the second term of right hand side of equation (13), according to Lemma 4 of (Pham et al., 2021) and the fact that Eyreve Emery [ = N, we obtain that for any 8 > O, with probability at least 1- 5.
1
Σ
Σ
(T(((((())))T)) CYr)
Σ
(f(xt):
N yreyr mE
T
IEINT
(14)
Rynmllof + MV
In(Drr||M)/0)
T
2 Σ
IVT
Σ
Σ
N
2N
N
ET mEI
yT
mE
where Ryr.m( (1 F) Err.cluupeer jer Σ (xe(f(x) )r}) |x1. Cm. y yrl. and { are
independently and identically distributed random variables that randomly taking values in 13. Moreover, using Cauchy-Schwar inequality, we have that
Σ
IYr N
S VD
(15)
Er mE
Then by In(()/|I)))/) < max(1, In (T In(e/6), we obtain the final tighter generalization upper bound of f:
Eg fl
1 N
Σ e(((x)))))
S
Σ
Iv m
E, [, (es((( -((((x)))))) C
(16)
yrEVr merdt
Rynmlloff + +ol 2ln(e/0) N
[I((e/0)
+ 2
Σ
Σ
IVr
N
2N
yrEVr melxe
For the of fM the second term in the right hand side of equation (13) equals 0. In fact, based on the definition Cyr .IT and I&T, and the matching method (i.e. based on the smallest Euclidean distance) between the input node features xx and the class prototypes in Mecoin, have that
E_ (xT). CYT)
Σ AS x1)
T
TETYT
(17)
E.(((h((rr).vT)le Cyr
((be(pr).yT)
I
(ho (pr())))))))))) ). 0.
where pr is the value that is uniquely corresponding to the class prototype m hio (xT), and ha represents the knowledge interchange process GKIM.
15

---

B.2 Proof of Theorem 2
When we use a simple non-pan paraamettric decoder function which uses average pooling cal- culate the element-wisg wise average of all the fetched memory representation and then applies softmax function to the output, the decoder is the same as a single hidden layer nets with fixed input weights. This network can be formulated as follows
proof:
f(u) c
Σ ( ==1
(18)
where A; is the i-th row of weight matrix, by is the bias, e is the output layer weights. For average pooling, C and is O, and A; is 4. So easy to prove that the VC-dimension according to (21), the VC-dimension of MLP with one hidden layer n² H² Then we consider the case when graph structure information is induced. When we use the distillation technique to inject graph structure information into GKIM via graph neural networks. the upper bound the capacity of memory representation in GKIM is the expressive capacity of the graph neural networks. Thus, according [22), we have that the VC-dimensinon of GKIM with graph structure information p²n² H2. In summary, when graph structure information is introduced, the upper bound of the VC dimension of GKIM increased and expressive power is also enhanced.
1. Then,
C Parameters and Devices
The relevant experimental parameters in this paper were determined through grid search. The pa- rameter settings for each dataset are shown in Table 7.
Table 7: Parameters used in each data
Dalasets
Backbone
Hidlen dim
Epach
Learning rale
weight decay
Dropout
protolype dim
CoraFull
GCN
GAT
128
64
2000
2000
0.0005
0.0005
0.5
14
14
CS
GCN
GAT
128
16
2000
2000
0.0005
0.0005
0.5
14
Computers
GCN
GAT
128
16
2000
2000
0.0005
0.0005
0.5
14
The environment which we run experiments is:
CPU information:24 vCPU AMD EPYC 7642 48-Core Processor
GPU RTffrmmttion:TTX A6000(48GB)
D
Limitations
While our method has shown promising performance in graph few-shot incremental learning, there still some issues that need to be addressed: 1) Our approach assumes that the graph structures across all sessions originate from the same graph. However, in real-world scenarios, data from different graphs may be encountered, and we have not thoroughly explored this issue; 2) Although our method currently maintains model performance with only a small number of samples, further validation is needed to assess whether it can still achieve excellent performance under low-resource conditions where graph structural information scarce.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Contributions are detailed in the last paragraph of the Introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We give the limitations in Appendix D.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the ap- proach. For example, a facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Ys]]
17

---

Justification:
Theoretical results are detailed in Section3.3 and Appendix B.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theo- rems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We report settings and details of reproducible experiments Section and Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or the contribution is a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data also be provided via
often one good way accomplish this, but reproducibility detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of model checkpoint, other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or a way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models. it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
18

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We will make our code public after the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
liaipss.cc/ubllcccuu..o. htto:i/ccguuiees/deeeuubttmt Cat///ooeeSSiiciccccee ssi onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We report all settings and details of experiments in Section and Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [No]
Justification: Error bars are not reported because we repeated each experiment the paper 10 times and reported the mean of the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
19

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors). should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK report I-sigma error bars, but one should state The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Appendix C of the paper reports the computer resources needed to reproduce the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't t make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics hattps://euurpp:: ctc//uuulic/Ehhcsuuuuuuee
Answer: [Yes]
Justification:
The research conducted in the thesis complied with the NeurIPS Code of Ethics in all respects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our approach does not address social impacts and this question is not appli- cable to our approach.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
20

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere to usage guidelines restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The creators original owners of assets used in the paper properly cred- ited and the license and terms of use explicitly are mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, pattustts.paperswiiiae paperswithcooeeeo com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documenta- tion provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
For crowdsourcing experiments and research with human subjects, does the pa- per include the full text instructions given to participants and sereenshots, applicable, well as details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper deseribe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
22

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23

---

FINESNN
helaos at Pi2liN
Mrtl wirdL PD
14-
-00
344
I60
20-
15-
Figure 7: Caption for Image 2
Are HNOR
Nolaar Ang PackAys
Gropiiall, Are
No Mi- www.T.
12:0
-15
-120
Figure 8: Caption for Image 2
The outcomes of GKIM when conducting the few-shot continuous learning task on the CoraFull and CS dataset. MeCs is the new name for MeCo(according to R3's suggestion). The results are presented sequentially from left to right: GKIM with full capabilities, GKIM where node features do not interact with class prototypes in the SMU, GKIM without GraphInfo and GKIM without MeCs The experimental results for CoraFull are shown the above figure, and the results for CS are in the figure below.
Figure 9:
24

---

GNN Encoder
Classifler
Graph data
(a) Graph neural network
Interatiion Lnowledge
Updating for new knowledge
Mensory losses for keeping memory
(d)
extmmets local gmoph
New class
Protety Representations
maiching
class
representation
Memory Representation Adaptive
(c) Structured Memory Unit
Figure 10: Overview of the Mecoin framework for GFSCIL.. (a)Graph neural network: Consists of GNN encoder and classifier(MIIP) pre-trained by GNN. GFSCIL tasks, the encoder parameters are frozen. Structtrdd Memory Unit: Constructs class prototypes through MeCs and stores them SMU. Me)ory Representation Adaptive Module: Facilitates adaptive knowledge interaction with the GNN model.
25


Reviews:
[{"review_id": "wPbAUvmN9t", "reviewer": "Reviewer_f8WD", "summary": "This paper mainly focuses on graph few-shot class-incremental learning. To alleviate the significant memory consumption and catastrophic forgetting of old knowledge, it proposes to store only old class prototypes and update class prototypes by considering the interaction among nodes and prototypes. Furthermore, it gives a theoretical analysis on the generalization error. Experimental results show that the proposed method can achieve better results.", "questions": "1.What are the meta-learning samples? There is no definition or explanation about them. It would be much better if more explanation would be given. \n2.Why using Gaussian random projection to reduce the dimensionality, instead of others?\n3.Why concatenating G_T and H_T^p? Could the authors give more explanation?\n4.What is the difference between class prototype and class representation? It is very hard to tell from the paper.", "limitations": "The authors have discussed the limitations in the paper.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The proposed idea of storing old class prototypes and updating class prototypes by considering their interaction with node features seems reasonable, as it has been proved to be useful in few-shot class-incremental learning. Experiments are relative sufficient to show its efficacy.", "weaknesses": "In general, this paper needs further polish, from my point of view. Just name a few, the abbreviations MRaM and MRAM are not consistent; the adjacency matrix A is incorrect; some sentences need careful attention. In addition, the motivation or the reason why the proposed method uses self-attention and MRAM is not clearly explained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1**：The paper needs further polish.  \n**A1**:Thank you for your thorough review and suggestions. We have carefully reviewed our paper and standardized the symbols and abbreviations. For example, we have unified MRaM and MRAM to MRaM. Additionally, we have conducted a comprehensive review and revision of the paper, focusing on sentence expression and structure to ensure clarity and accuracy.   \n   \n**W2**:self-attention and MRaM is not clearly explained.  \n**A2**:Thank you for your thorough review and suggestions. In Eq.2, self-attention is used to enable the interaction between current task node features and seen class information. Specifically, the SMU stores class prototypes of seen classes, which are representative samples containing information about these classes. In continual learning on graphs, there are implicit relationships between graph structures of different tasks. Capturing these relationships helps in learning more representative class prototypes, thereby preventing forgetting[1]. Self-attention has been widely used to capture interactions between pieces of information. In Eq.3, attention is used to process the current task's node features, effectively capturing the relationships between nodes, which aids in learning more representative class prototypes.\nWe designed MRaM to decouple the learning of class prototypes from the learning of class representations. Traditional graph continual learning methods further learn the probability distribution of classes (class representations) based on learned class prototypes for classification. This often leads to forgetting due to extensive parameter updates during prototype learning. MRaM separates the learning of class prototypes from class representations, avoiding this issue.  \n[1]Geometer:Graph few-shot class-incremental learning via prototype representation.  \n  \n**Q1**: Explanation of meta-learning samples.   \n**A1**: Thank you for your valuable feedback. In the context of graph few-shot incremental learning, 'meta-learning samples' refers to the samples used in the meta-learning training process. We apologize for any confusion caused and have provided additional clarification on meta-learning samples in our paper.  \n  \n**Q2**: Why using Gaussian random projection to reduce the dimensionality?  \n**A2**: This is an excellent question. We employ Gaussian random projection for dimensionality reduction for the following reasons:\n1. **Computational Efficiency**: Compared to other dimensionality reduction methods, Gaussian random projection offers high computational efficiency. For instance, performing principal component analysis on a matrix of shape $(n, n)$ has a computational complexity of $\\mathcal{O}(n^3)$, whereas Gaussian projection has a complexity of $\\mathcal{O}(n^2)$.\n2. **Theoretical basis**: According to the Johnson-Lindenstrauss lemma[2], Gaussian random projection can effectively preserve the distance relationships between high-dimensional sample points with a certain probability. It also exhibits good robustness for non-linearity, noise, or unevenly distributed data. Taking the scenario discussed in this paper as an example, let $z$ be an embedding vector and $v$ a perturbation of $z$. When dimensionality reduction is performed on $z+v$ using the Gaussian random projection matrix $R$, if $v$ lies in the null space of $R$, then $R(z+v)=Rz=Rz+Rv$, meaning the perturbation has no effect. Since $R$ is used for downsampling and has a large null space (downsampling matrices typically reduce data dimensions, projecting many original data directions to zero), perturbations like $v$ are more likely to be eliminated.  \n[2]On variants of the Johnson–Lindenstrauss lemma.  \n  \n**Q3**: Explanation of why concatenating $G_T$ and $H_T^p$.    \n**A3**: **Information Integration**: According to Eq.2, $H_T$ is obtained by using attention to integrate current task node features with past task class prototypes stored in the SMU. We then reduce the dimensionality of $H_T$ using a Gaussian random projection to obtain $H_T^p$, which helps reduce subsequent computational complexity. For $G_T$, Eq.3 further extracts the current task's graph structure information. By concatenating these two types of information, we obtain richer and more comprehensive node features, which better capture the complex relationships between the current and past tasks' graph structures.   \n**Theoretical Support**: Concatenation is a common and effective feature fusion method, validated in many studies on graph neural networks and self-attention mechanisms. For example, previous works like MixHop[3] concatenate current node information with neighbor node information to integrate neighbor features and capture richer structural information.   \n**Experimental Support**: The ablation studies in Fig.3 demonstrated the effectiveness of using $G_T$ or $H_T^p$ alone (no graphinfo, no inter) respectively, as well as with and without the concatenation method(with MeCo, no MeCo). And the concatenation method performs the best.   \n[3] MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing.   \n    \n**Q4**: Difference between class prototype and class representation.    \n**A4**: Thank you for your meticulous review and valuable feedback. The class prototype $\\mathbf{m}$ is a typical representation of the class. In our paper, it is obtained by integrating the information of seen classes and local graph structure with node features through MeCo, representing the class centroid. The class representation $\\mathbf{p}$ contains probability information for each category and is used to determine the sample's category. In our paper, $\\mathbf{p}$  is randomly initialized and updated through the distillation process in GKIM. When a sample finds its closest class prototype $\\mathbf{m}$ via Euclidean distance, it further matches $\\mathbf{p}$ for classification.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jhiRudIex6", "reviewer": "Reviewer_cvjp", "summary": "The authors focus on graph few-shot class-incremental learning. The authors first introduce Mecoin to efficiently construct and preserve memory. To avoid extensive parameter finetuning and forgetting, the authors introduce a memory representation adaptive module called MRaM to separate the learning of prototypes and class representations. Besides, the authors propose Graph Knowledge Interchange Module (GKIM) to injects past knowledge information into GNN. Additional analyses illustrate the effectiveness of the methods.", "questions": "See Weaknesses.", "limitations": "The paper has a section discussing several limitations in the appendix.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written, and the motivations for each part are clear.\n2. The reported performance out-performs many baselines.\n3. The theoretical and experimental analyses successfully illustrate the effectiveness of the methods.", "weaknesses": "1. The paper does not report error bars, standard deviations or provide detailed information about the statistical significance of the experiments, which is important for understanding the reliability and variability of the results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1**: Lacks error bars, standard deviations.   \n**A1**: Thank you for the valuable feedback. In the initial version, we omitted detailed statistical information due to formatting and space constraints. However, these details are crucial for a comprehensive evaluation of model performance and result reliability. Based on the reviewer's suggestions, we have added error bars, standard deviations, and statistical significance analyses to the experimental results.    \n\n|methods|backbone|dataset||||number of codebook=100, 2-way-5-shot||||\n|-|-|-|-|-|-|-|-|-|-|\n| \\ | \\ | \\ |session 0|session 4|session 6|session 8|session 10|PD|AVE Acc|\n|ergnn|GCN|CoraFull|73.43±0.89|20.93±0.23|16.13±0.31|15.2±0.25|11.3±0.55|62.13±0.74|29.55±0.98|\n| |GAT|CoraFull|69.06±0.11|28.13±0.59|24.06±0.48|24.47±0.59|12.92±0.79|56.14±0.92|33.31±0.39|\n|lwf|GCN|CoraFull|73.43±0.48|19.71±0.82|10.95±0.86|9.90±0.45|7.73±0.37|65.70±0.32|22.26±0.39|\n| |GAT|CoraFull|73.60±0.84|19.95±0.57|13.76±0.66|10.83±0.46|7.46±0.89|66.14±0.58|24.02±0.57|\n|gem|GCN|CoraFull|73.43±0.45|19.44±0.67|11.70±0.91|9.86±0.21|8.27±0.31|65.16±0.78|22.38±0.65|\n| |GAT|CoraFull|69.06±0.66|19.44±0.15|12.79±0.33|9.90±0.29 |7.52±0.22 |61.54±0.27|22.65±0.48|\n|EWC|GCN|CoraFull|73.43±0.74|20.27±0.74|11.41±0.61|10.04±0.32|7.75±0.98 |65.68±0.48|22.52±0.26|\n| |GAT|CoraFull|69.06±0.39|20.39±0.66|14.86±0.67|19.70±0.68|12.55±0.74|56.51±0.26|27.14±0.81|\n|MAS|GCN|CoraFull|73.43±0.97|22.44±0.58|20.50±0.83|18.01±0.97|15.53±0.82|57.90±0.22|25.28±0.89|\n| |GAT|CoraFull|69.06±0.96|37.08±0.19|48.58±0.55|44.26±0.46|46.39±0.50|22.67±0.89|49.99±0.77|\n|TWP|GCN|CoraFull|70.54±0.79|24.16±0.20|18.77±0.25|19.37±0.17|14.48±0.19|56.06±0.43|25.54±0.17|\n| |GAT|CoraFull|69.06±0.53|21.40±0.34|15.04±0.54|14.86±0.12|13.77±0.27|55.29±0.84|27.84±0.52|\n|Geometer| \\ |CoraFull|72.23±0.63|36.24±0.94|29.97±0.99|21.01±0.11|16.32±0.89|55.91±0.23|35.52±0.56|\n|HAG-Meta| \\ |CoraFull|87.62±0.44|67.65±0.99|60±0.81|55.5±0.78 |51.47±0.46|36.15±0.91|66.57±0.83|\n|ours|GCN|CoraFull|82.18±0.17|70.43±1.26|68.78±1.04|66.7±0.35 |61.36±0.07|20.82±0.20|70.90±0.40|\n| |GAT|CoraFull|75.53±0.12|66.05±0.87|64.18±0.87|62.02±0.12|60.10±0.10|15.43±0.33|66.22±0.26|\n\n|methods|backbone|dataset|| | |number of codebook=100, 1-way-5-shot | | | |\n|-|-|-|-|-|-|-|-|-|-|\n| \\ | \\ | \\ |session 0|session 4|session 6|session 8|session 10|PD|AVE Acc|\n|ergnn|GCN|CS|100±0.01|20±0.02|14.29±0.08|11.11±0.03|14.02±0.09|85.98±0.03|27.901±0.05|\n| |GAT|CS|100±0.01|33.95±0.13|24.64±0.20|18.6±0.21|30.12±0.18|69.88±0.14|38.41±0.11|\n|lwf|GCN|CS|100±0.01|20±0.03|14.29±0.06|11.11±0.03|15.44±0.17|84.56±0.08|28.60±0.12|\n| |GAT|CS|100±0.01|36.28±0.12|28.92±0.12|21.23±0.10|32.51±0.12|67.49±0.14|38.30±0.13|\n|gem|GCN|CS|100±0.01|20±0.02|14.29±0.05|11.11±0.03|12.12±0.22|87.88±0.07|27.73±0.09|\n| |GAT|CS|100±0.01|20±0.02|14.85±0.08|14.17±0.15|18.09±0.12|81.91±0.07|30.30±0.13|\n|EWC|GCN|CS|100±0.01|20±0.01|14.29±0.08|11.11±0.03|12.12±0.10|87.88±0.05|27.73±0.07|\n| |GAT|CS| 100±0.01|38.60±0.19|25.64±0.12|16.19±0.27|38.63±0.17|61.37±0.11|38.74±0.15|\n|MAS|GCN|CS|100±0.01|20±0.01|14.29±0.33|11.11±0.03|9.70±0.21|90.30±0.16|28.32±0.18|\n| |GAT|CS|100±0.01|56.16±0.18|65.57±0.25|61.41±0.21|63.92±0.13|36.08±0.19|60.68±0.17|\n|TWP|GCN|CS|100±0.01|20±0.01|14.29±0.08|11.04±0.05|15.03±0.15|84.97±0.12|28.60±0.12|\n| |GAT|CS|100±0.01|38.14±0.14|39.34±0.13|30.52±0.19|52.02±0.17|47.98±0.14|44.43±0.15|\n|Geometer| \\ |CS|60.6±0.22|24.87±0.16|24.46±0.19|19.30±0.19|29.63±0.20|30.97±0.16|28.11±0.19|\n|HAG-Meta| \\ |CS|20±0.02|11.11±0.01|9.09±0.03|7.69±0.02|6.66±0.03|13.34±0.04|11.23±0.03|\n|ours|GCN|CS|98.07±0.27|79.95±1.27|74.13±1.12|69.48±2.74|59.66±0.79|38.41±0.56|77.96±1.1|\n| |GAT|CS|97.83±0.21|71.79±1.14|75.35±1.13|72.52±1.47|62.21±0.66|35.62±0.45|77.50±1.01|\n\n|methods|backbone|dataset| | | |number of codebook=100, 1-way-5-shot| | | | |\n|-|-|-|-|-|-|-|-|-|-|-|\n| \\ | \\ | \\ |session 0|session 1|session 2|session 3|session 4|session 5|PD|AVE Acc|\n|ergnn|GCN|Computers|100±0.01|50±0.01|33.33±0.01|25±0.01|20±0.01|16.67±0.01|83.33±0.01|40.83±0.01|\n| |GAT|Computers|100±0.01|50±0.01|33.33±0.01|25±0.01|20±0.01|16.67±0.01|83.33±0.01|40.83±0.01|\n|lwf|GCN|Computers|100±0.01|50±0.01|33.33±0.01|25±0.01|20±0.01|16.67±0.01|83.33±0.01|40.83±0.01|\n| |GAT|Computers|100±0.01|50±0.01|33.33±0.01|25±0.01|20±0.01|16.84±0.01|83.16±0.01|40.86±0.03|\n|gem|GCN|Computers|100±0.01|50±0.01|33.33±0.01|25±0.01|20±0.01|16.67±0.01|83.33±0.01|40.83±0.01|\n| |GAT|Computers|100±0.01|50±0.01|33.33±0.01|25±0.01|20±0.01|16.67±0.01|83.33±0.01|40.83±0.01|\n|EWC|GCN|Computers|100±0.01|50±0.01|33.33±0.01|25±0.01|20±0.01|16.67±0.01|83.33±0.01|40.83±0.01|\n| |GAT|Computers|100±0.01|50±0.01|33.33±0.01|25±0.01|20±0.01|16.67±0.01|83.33±0.01|40.83±0.01|\n|MAS|GCN|Computers|100±0.01|50±0.01|33.33±0.01|25±0.01|20±0.01|16.67±0.01|83.33±0.01|40.83±0.01|\n| |GAT|Computers|100±0.01|50±0.01|33.57±0.04|37.39±0.15|25.90±0.12|21.64±0.15|78.36±0.18|44.75±0.15|\n|TWP|GCN|Computers|100±0.01|50±0.01|33.33±0.01|25±0.01|20±0.01|16.67±0.01|83.33±0.01|40.83±0.01|\n| |GAT|Computers|100±0.01|50±0.01|33.33±0.01|25±0.01|20±0.01|16.67±0.01|83.33±0.01|40.83±0.01|\n|Geometer| \\ |Computers|59.40±0.11|33.00±0.28|23.57±0.11|18.56±0.17|15.39±0.17|13.20±0.19|46.19±0.13|27.19±0.14|\n|HAG-Meta| \\ |Computers|20±0.03|16.67±0.02|14.28±0.03|12.50±0.01|11.11±0.02|10±0.02|10±0.02|14.09±0.02|\n|ours|GCN|Computers|90.18±0.81|88.75±2.16|73.36±2.51|71.19±3.12|69.41±1.75|63.91±1.26|26.27±1.53|76.13±1.65|\n| |GAT|Computers|91.44±0.65|91.44±1.22|54.94±1.39|68.73±2.34|73.64±1.33|67.66±1.21|23.78±1.08|74.64±1.21|\n\nWe apologize that due to word limits, we cannot include the results for all sessions for CoraFull and CS. We will include this detailed statistical information in the revised version of the paper to better demonstrate the reliability and significance of the experimental results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FWRTceFdo9", "reviewer": "Reviewer_utSJ", "summary": "This paper tackles the problem of graph few-shot class-incremental learning, proposes a novel framework named Mecoin to address the challenges of catastrophic forgetting and overfitting. Mecoin includes two key componets: the structured memory unit (SMU) for storing and updating prototypes and the memory representation adaptive module (MRaM) for separating the learning of prototypes and class representations. Based on SMU, Mecoin efficiently maintain representative prototypes preventing the overfitting problem caused by few-shot setting, and rely on MRaM, Mecoin avoids heavily finetuning the model parameters which solved the catastrophic forgetting problem in class-incremental learning.", "questions": "Please check the weaknesses", "limitations": "The paper is poor-writing and many places lack detailed explanations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1.Stores past knowledge efficiently with low storage space by using the SMU.\n2.Avoids catastrophic forgetting by separating the learning of prototypes and class representations.\n3.The experiments show significant improvement compared to other state-of-the-art methods.", "weaknesses": "1.The abbreviations of the proposed method are confusing. For example, in some places, the Memory Representation Adaptive Module is abbreviated as MRaM, while in others, it is MRAM. Additionally, the full model name, Mecoin, is too similar to one of its submodules, the Memory Construction module (MeCo).\n2.Fig. 1 is confusing; it doesn’t show the details and advantages of the SMU and MRaM structures.\n3.In Sec. 3.1, Eq. 6 lacks explanatory information. How does Eq. 6 integrate edge information into the loss function?\n4.The proof of Theorem 1 is too brief. I recommend the authors provide detailed derivation rather than citing a lemma from another paper.\n5.In Sec. 3.2, Eqs. 8 and 9 are incomprehensible. I assume the GKIM maintains past knowledge and updates new knowledge with different elements 𝑝. For example, if the dataset has 100 classes, \n𝑝1:60 would be maintained for past knowledge and 𝑝61:100 for new knowledge. Thus, 𝑁 should be 60. It’s challenging to explain how new knowledge is updated in Eq. 9.\n6.In Sec. 4.1, the process of pretraining the GNN needs to be explained. Is the pretraining phase a standard graph node classification task or a few-shot node classification task? Additionally, in Tab. 3, the comparison method MAG-Meta is mentioned twice; I guess one should be Geometer. Also, if the proposed method is named Mecoin, why does Mecoin appear among the comparison methods in Tabs. 2, 3, and 4?\n7.In Sec. 4.2, the toy dataset is too simple. Sampling only 4 classes makes it difficult to demonstrate the ability of different settings to handle overlapping problems. I recommend sampling at least 10 classes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1**: The abbreviations of the proposed method are confusing.   \n**A1**: We appreciate your attention to our work and the valuable feedback, and sincerely apologize for the confusion caused by the use of abbreviations. According to your suggestions, we have re-examined the entire text and decided to standardize the abbreviation for \"Memory Representation Adaptive Module\" to ‘MRaM’, and the ‘’Memory Construction module’’ to ‘MeCs’ for minimizing the confusion. We have carefully reviewed our paper and ensured that all relevant abbreviations have been modified to avoid any potential confusion.  \n\n**Q2**: Fig. 1 is confusing; it doesn’t show the details and advantages of the SMU and MRaM structures.   \n**A2**: We appreciate your feedback regarding to Fig.1 and apologize for the confusion. The Structured Memory Unit (SMU) and the Memory Representation Adaptive Module (MRaM) are the key components of Mecoin, serving to learn representative class prototypes and interact with the Graph Neural Network (GNN) to retain past knowledge while updating new knowledge. In accordance with your advice, we have redesigned Fig.1 which include the structure and workflow of the SMU and MRaM. The redesigned Fig. 1 is included in the submitted PDF.  \n\n**Q3**: In Sec.3.1, Eq.6 lacks explanatory information. How does Eq.6 integrate edge information into the loss function?  \n**A3**: The first term in Eq.6 calculates the similarity between node features, thus reflects the strength of the connections between nodes[1,2]. This connection strength helps the SMU better distinguish the features of different classes of nodes, leading to more representative class prototypes.   \n[1]Confidence-based Graph Convolutional Networks for Semi-Supervised Learning.    \n[2]Geom-GCN: Geometric Graph Convolutional Networks.  \n\n**Q4**: The proof of Theorem 1 is too brief.   \n**A4**: Thank you for your suggestions.We included the proof outline and description of Theorem 1 in the appendix but not the detailed proof. We appreciate the reviewer's suggestion, as a detailed explanation would help readers better understand our theorem and conclusions.According to your advice, we have supplemented the proof,  due to the rebuttal's word limit, we apologize for not being able to present the complete proof here, but give the outline. Similar to the proof of Thm3.1 in [16], we first decompose the generalization error $\\mathcal{R}=\\mathbb{E}[\\ell(f(d_{\\epsilon}(x)),y)]-\\frac{1}{|\\mathcal{X}_T|}\\sum_i f(x_i,y_i)$ based on the probability $\\mathrm{Pr}((x,y) \\in \\mathcal{S}_c)$ and the Total Probability Theorem. After some calculation and modification of the above decomposition, by using Lemma 1 in paper reference [7], we obtain Eq.11. For simplicity, we denote the last term of Eq.11 as $R’$, which has a tighter upper bound for other models according to Lemma 4 in paper ‘Combined scaling for open-vocabulary image classification’, and is not necessarily 0. While it equals 0 for Mecoin based on the definition of $\\mathcal{S}_c,\\mathcal{I}_c$ and the matching method(i.e. based on the smallest distance) in Meocin, since the term $\\mathbb{E}_z[\\ell(f(x),y)|z\\in\\mathcal{S}_c] = \\frac{1}{|\\mathcal{I}_c|}\\sum_i\\ell(f(x_i),y_i)$. Thus we conduct that Mecoin has a lower generalizaiton bound than other models. Furthermore, to clarify the theorem result and the proof, we have modified the symbols and statements, and have updated in the revised paper.   \n\n**Q5**: The definition of $N$ makes it challenging to explain how new knowledge is updated in Eq.9  \n**A5**: Thank you for pointing out the issues and errors. In Eq.9, we incorrectly used symbols. To distinguish between seen classes and unseen classes, we will use 𝑁_𝑠 and 𝑁_𝑢 to represent the sample sizes for seen and unseen classes, and replace 𝑁 in Eq.8,9 accordingly.  When new class samples appear, Mecoin first calculates the class prototypes 𝑚61:100 for these new classes through the SMU module and identifies the corresponding 𝑝61:100, where 𝑝61:100 are randomly initialized and not updated. During the training process 𝑝61:100 are updated and integrated into MRaM via Eq.9, thus updating and storing new knowledge.  \n\n**Q6**: Explain the pre-training experimental setup. In Tab.3, the comparison method MAG-Meta is mentioned twice. Why does Mecoin appear among the comparison methods in Tabs.2, 3, and 4?   \n**A6**: Thank you for thorough examination and feedback. We used a standard graph node classification task in pre-training, where the specific samples and number of classes in each dataset are detailed in Tab.1. We apologize for the error in Tab.3. The second MAG-Meta should be Geometer and we have corrected it in the revised manuscript. Tab.2, 3, 4 show the comparison results of Mecoin with other methods across different datasets, which demonstrate the consistent performance and superiority of Mecoin under various experimental settings and across multiple benchmarks.  \n\n**Q7**: Sampling only 4 classes makes it difficult to demonstrate the ability of different settings to handle overlapping problems.   \n**A7**: Thank you very much for your suggestions. In Section 4.2, we used three commonly used graph-structured datasets: CoraFull, CS, and Computers. Our experiment aimed to demonstrate the impact of the SMU and related operations (Eq.2, Eq.3) on model performance and class prototype learning. We realized that sampling only four classes was insufficient to fully illustrate the handling of overlapping issues in different settings. Therefore, based on your suggestion, we increased the number of sampled classes to 10 and presented the results in the submitted PDF file. The experimental results with 10 classes were consistent with those obtained with four classes.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thank you very much to the reviewers for their valuable feedback on our paper. The reviewers acknowledged our strong motivation (**R3**), efficient and low-cost method design (**R1, R2, R3**), and insightful analysis (**R2**), which greatly encouraged us. We are pleased that the reviewers found our method significantly improves over existing baselines (**R1, R2, R3**). We are also glad that **R3** recognized our design of the SMU module, which effectively reduces memory usage and separates the learning processes of class prototypes and class representations to help prevent catastrophic forgetting.

Additionally, we greatly appreciate the constructive suggestions for further improving the paper. We have carefully reviewed and revised the paper to address the confusions of symbols, terminologies and presentation. We also redrew figures and supplemented experiments according to the reviewers' recommendations. Thank you again for your suggestions, which will be very helpful for our future research. We believe our revisions will address the reviewers' concerns.

Meta-review: This paper addresses the problem of graph few-shot class-incremental learning and proposes a novel framework named Mecoin to tackle the challenges of catastrophic forgetting and overfitting. Mecoin comprises two key components: the Structured Memory Unit (SMU) for storing and updating prototypes, and the Memory Representation Adaptive Module (MRaM) for separating the learning of prototypes and class representations. The SMU helps maintain representative prototypes, effectively preventing overfitting in few-shot settings. Meanwhile, MRaM mitigates catastrophic forgetting by avoiding extensive fine-tuning of model parameters in class-incremental learning.

During the rebuttal period, the authors satisfactorily addressed the reviewers' concerns, particularly those regarding the methodology presentation and experimental setup. The problem tackled in the paper is important, and the proposed solution is novel. Therefore, we recommend acceptance of this paper. However, the authors are strongly encouraged to (1) significantly improve the writing, especially in the method section, to enhance clarity and (2) include the additional empirical results provided during the rebuttal period to strengthen the paper’s contributions.
Decision: Accept (poster)