GraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation
Authors: Zihao Guo
Qingyun Sun
Ziwei Zhang
Haonan Yuan
Huiping Zhuang
Xingcheng Fu
Jianxin Li
Venue: NeurIPS 2025 poster ()
Keywords: Graph Neural Network
Incremental Learning
Multi-Domain
Primary area: deep_learning
TL;DR: In this paper, we propose a novel framework GraphKeeper to address the catastrophic forgetting in Graph Domain-Incremental Learning scenario.

Abstract: Graph incremental learning (GIL), which continuously updates graph models by sequential knowledge acquisition, has garnered significant interest recently. However, existing GIL approaches focus on task-incremental and class-incremental scenarios within a single domain. Graph domain-incremental learning (Domain-IL), aiming at updating models across multiple graph domains, has become critical with the development of graph foundation models (GFMs), but remains unexplored in the literature. In this paper, we propose **Graph** Domain-Incremental Learning via **K**nowledge Dis**e**ntangl**e**ment and **P**res**er**vation (**GraphKeeper**), to address catastrophic forgetting in Domain-IL scenario from the perspectives of embedding shifts and decision boundary deviations. Specifically, to prevent embedding shifts and confusion across incremental graph domains, we first propose the domain-specific parameter-efficient fine-tuning together with intra- and inter-domain disentanglement objectives. Consequently, to maintain a stable decision boundary, we introduce deviation-free knowledge preservation to continuously fit incremental domains. Additionally, for graphs with unobservable domains, we perform domain-aware distribution discrimination to obtain precise embeddings. Extensive experiments demonstrate the proposed GraphKeeper achieves state-of-the-art results with 6.5%\~16.6% improvement over the runner-up with negligible forgetting. Moreover, we show GraphKeeper can be seamlessly integrated with various representative GFMs, highlighting its broad applicative potential.

GraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation
Zihao Guo', Qingyun Sun'; Ziwei Zhang', Haonan Yuan', Huiping Zhuang", Xingcheng Jianxin Li² SKLCCSE, School of Computer Science and Engineering, Beihang University -Shien-Ming Wu School of Intelligent Engineering, South China University of Technology $ Key Lab of Education Blockchain and Intelligent Technology, Guangxi Normal University {guozh, sunqy, zwzhang, yuanhn, y edu. cn hpzhuang@scut. edu. cn, fuxc@gxnu.c edu. cttp:yuu.ddu..om
Abstract
Graph incremental learning (GIL), which continuously updates graph models by sequential knowledge acquisition, has garnered significant interest recently. How- ever, existing GIL approaches focus on cask-incremental and class-incrementa scenarios within a single domain. Graph domain-incremental learning (Domain- IL), aiming at updating models across multiple graph domains, has become critical with the development of graph foundation models (GFMs), but remains unex- plored in the literature. In this paper, we propose Graph CemnainnIncrementll Learning via Knowledge Disentanglement and Preservation (GraphKeeper), address catastrophic forgetting in Domain-ll. scenario from the perspectives embedding shifts and decision boundary deviations. Specifically, prevent em- bedding shifts and confusion across incremental graph domains, we first propose the Nomain-speciffe parameter-efficient fine-tuning together with intra- and inter- domain disentanglement objectives. Consequently, to maintain a stable decision boundary, introduce deviation-free knowledge preservation continuously fit incremental domains. Additionally, for graphs with unobservable domains, we perform domain- aware distribution discrimination obtain precise embeddings. Extensive experiments demonstrate the proposed GraphKeeper achieves state-of- the-art results with 6.5%~~16.6%% improvement over the runner-up with negligible forgetting. Moreover, we show GraphKeeper can be seamlessly integrated with various representative GFMs, highlighting its broad applicative potential.
Introduction
Graph incremental learning (GIL) 6 47 aims continuously update graph models as new graph data arrives, and has attracted increasing attention. Most existing methods target mask-incremental (Task-IL) and class-incremental (Class-IL) settings (Figure II. where new tasks or classes emerge within a single domain. However, as newly added graphs often come from different domains, the domain-increments. (Domain-IL) setting becomes essential. This is especially relevant with the rise graph foundation models (GFMs), which require integrating diverse graphs from multiple domains to build a comprehensive and evolving knowledge base. Unfortunately, current GIL methods are designed for single-domain scenarios and struggle with Domain-IL. As illustrated in Figure 2 representative method SSM [45) performs well under Class-TL but fails in the more challenging Domain-II. case. Despite its importance, effective GII. under Domain-II II remains an open problem.
*Coreesonding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Traditional GII Somario
Domain-II. Scenario
114
280
Domaia A
Derrain
Task-Il.
Classil
(a) Class-IL Scenario (b) Domain- IL Scenario
Same Domain
Same Donaln
Difffrrett Domaia
Figure I: An illustration of traditional GIL (i.e. Task- Figure 2: Performance of SSM 45 a rep-
IL and Class-IL) and Domain-IL scenarios within our resentative GIL method, in Class- IL scenario scope studied in this paper.
and the more challenging Domain- scenario.
To tackle this problem, we first analyze the underlying cause of performance degradation in the Domain-IL setting, which is the catastrophic forgetting problem. In typical graph learning models such as GNNs, learning new graphs causes changes in model parameters that manifest in two aspects: (1) shifts in the embeddings of previously learned graphs, and (2) deviations in the decision boundary. These changes together lead to the forgetting of prior knowledge. Existing GIL methods attempt to constrain or adapt to such changes, which relatively feasible in Task-II. and Class-II. scenarios, as the data still resides within a single domain. In contrast, Domain-IL. involves substantial structural and semantic discrepancies across domains, making it much harder for GNNs to retain knowledge across diverse graph distributions. This difficulty is also evidenced by the negative transfer observed in multi-domain graph pre-training [2].
Based on the analysis above, we tackle the GIL challenges in Domain-IL scenario from two sides: (1) Embedding shifts: How to learn stable and disentangled representations for different graph domains in GIL? The embedding shifts introduced by model parameter changes bring the risk of semantic confusion between incremental graphs. To adapt to new domains, more drastic parameter changes occur, making embedding shifts more uncontrollable. (2) Decision boundary deviations: How to effectively retain knowledge from previously learned graph domains? The deviation in decision boundary is also a significant cause of catastrophic forgetting [51], especially when the model adapts to new domains. Previous GIL methods, which treat embedding learning and prediction as integrated process, are difficult to constrain the decision boundary effectively.
In this paper, we propose novel GIL framework GraphKeeper address the catastrophic forgetting in Domain-IL. scenario. Specifically: (1) To address the embedding shifts challenge, we propose the domain-specinfe graph parameter-efficien
fine-tuning (PEFT) based on a pre-trained GNN to learn embeddings correspond to graph domains, which ensures parameters for previously learned graph domains remain unaffected when learning new domains. Additionally, we propose intra- and inter- domain disentanglement to avoid confusion in embedding space. (2) To address the decision boundary deviation challenge, we separate the decision module from the embedding model and continuously fit incremental domains through the analytical solution of ridge regression, which prevents drastic changes in the decision boundary while effectively retaining knowledge from previously learned graph domains. (3) Lastly, we perform domain-awan distribution discrimination on graphs with unobservable domain to facilitate the precise embedding and prediction. Our contributions are:
We propose a novel GIL framework GraphKeeper to tackle the catastrophic forgetting in Domain-IL scenario. To the best of our knowledge, we the first to explore this challenging problem.
We design the multi-domain graph disentanglement and deviation- knowledge preservation mechanism, which enables GraphKeeper learn stable and disentangled representations between different incremental graph domains and maintains stable decision boundary without deviations.
GraphKeeper achieves 6.5%~ 16.6% improvement over the runner-up with negligible forgetting. Besides, it can be seamlessly integrated with existing representative GFMs.
Related Work
2.1 Graph Incremental Learning
Existing graph incremental learning methods can be categorized into regularization-base memory replay-based, and parameter isolation-based Eegularization-base methods [16 31 aim to identify

---

Incremennall Training
Encreiional Domsins
Multi demnsin Graph Dicuttagkaenn
Deeisticn-Fe Kaunledge Prasevation
Damalivanary Bitribation Diserimination
specifie Graph
Predkt
Enbeeling
Presrained GNS e()
EC))
Stable Decision Space
G1
Flurt
Dernatn
Inrra- and Interr domail Disnttnngeement
High-disconinal imppits farction GhiM
Updaste the Ridge Regression Parameters by Eq.18)) and Eq.(1l)
Noda Emahudding
Jnter Derenin
Dossis
Graph Donata
Trikiiy
Infacner () Fouce e Tuxcl * Docain Protatgoe
Exbeckinn Prekeapo
Figure 3: The overall framework of GraphKeeper. (1) The Multi-domain Graph Disentanglement isolates parameters of different graph domains through the domain-specific graph PEFT prevent embedding shifts, and disentangles embeddings both intra- and inter-domain to prevent confusion. (2) The Deviation-Free Knowledge Preservation continuously fits incremental graph domains while maintaining a stable decision boundary without deviations. (3) The Domain- aware Distribution Discrimination matchs graphs with unobservable domain to prototypes of previous domains. Then our method embeds them with corresponding domain-specing PEFT module, and make predictions.
parameters that are important for previous tasks and constrain their changes through penalty terms. Memory replay-based methods [52 45 34 33
42 22 are widely recognized for their effectiivnness, explicitly preserving representative subsets of graphs from previous tasks and replaying them while learning new tasks. Parameter isolation- based methods [44 41 23 adapt to graph data in new tasks through adding trainable parameters. Despite the impressive performance in Task IL and Class-IL scenarios, these methods not consider the unique challenges in the Domain-Il scenario.
46
17
2.2 Multi-domain Graph Learning
Cross-domain Graph Pre-training. A promising approach for constructing GFMs is to pre-train on a large corpus of graphs, and transfer the pre-trained knowledge to downstream graphs 4% Although existing cross-domain graph pre-training methods [49 33 37 (50 can effectively adapt to downstream graphs, the model becomes fixed once pre-training complete. Continuously learning knowledge from new graphs is essential [47]. especially when dealing with multiple downstream graph domains, faces the problem catastrophic forgetting. Our method addresses the challenging Domain-IL. scenario in GIL and can be integrated with various graph pre-training methods.
Graph Domain Adaptation. Graph domain adaptation (GDA) (88 aims to leverage knowledge from the source domain to improve the performance in the target domain. Although multiple domains are considered, GDA and Domain-ll. scenario GIL are fundamentally different [36]. For GDA, the source and target domains are simultaneously accessible in most cases, with the focus primarily on target domain performance. In comparison, Domain-Il II scenario, only the domain current task is accessible, and the objective is to maintain performance across all previously learned domains.
3 Preliminary and Problem Formulation
Graph Incremental Learning. Given task sequence S (G GT. the objective of GIL is to sequentially learn the graphs while ensuring that the model parameters 0, after learning the current task maintain satisfactory performance on all previous tasks. The objective can be formulated as:
E'' == L.(0),
0 arg min
0
3

---

where C, is i-th task loss. Previous t- 1 tasks and their data are inaccessible when learning the t-th.
Problem Formulation. We primarily focus GIL in the Domain-II. scenario, i.e., graphs G. in S belong to different domains. This contrasts with Task-Il. and Class-II,, where all tasks belong to the same domain. Due significant domain gap in both structure and feature semantics, conflicts arise among incremental domains in Domain-IL scenario. Our goal is to learn an encoder & : V + Z, where different domains are disentangled in the embedding space Z, representing their respective semantics, and to learn a decoder V/- Z + P that accurately maps the disentangled embedding of different graph domains the label space P, while continuously updating without forgetting.
4 Method
In this section, we present GraphKeeper to address the catastrophic forgetting in the Domain IIL scenario. Our key insight is to decompose the cause of catastrophic forgetting into embedding shifts and decision boundary deviations. The framework is shown in Figure 33 In brief, we first propose multi-domain graph disentanglement (Sec. 4.1 to prevent embedding shifts and confusion across incremental graph domains. Then, we introduce deviation-free knowledge preservation (Sec. (42) to continuously fit incremental domains while maintaining a stable decision boundary without deviations. Lastly, we design a domain-aware distribution discrimination mechanism (Sec. 4.3) for graphs with unobservable domains obtain precise embeddings.
4.1 Multi-domain Graph Disentanglement
To prevent confusion across incremental graph domains, we introduce glement, which isolates parameters between different graph domains through the domain-specific graph PEFT to avoid embedding shifts, and disentangles embeddings both intra- and inter-domain.
graph disentan-
Multi-domain Graph Feature Alignment. Prior to further processing, we first align the feature dimension for graphs from different domains:
F, Proj (Fi), F.ERIGIxa
where F; is the original node features of G; |G,] denotes the number of nodes, Proj(.) represents projection operation, and d represents the unified feature projection dimension. The projection operation (-oj(-) is realized through truncated singular value decomposition (SVD) 29]
In addition to this dimension alignment, the feature semantics of different domains remain fundamen- tally distinct. On the one hand, drastic changes in model parameters can occur when adapting to new domains, causing the embedding shifts across graph domains. On the other hand, the embeddings of different graph domains may exhibit unintentional overlap that contradicts their semantics, leading to confusion in prediction. Both factors can lead to the catastrophic forgetting in the Domain-IL scenario. To address these issues, we introduce two core modules as follows.
To prevent potential catastrophic forgetting of previous domains due drastic parameter changes when adapting to new domains, we aim isolate parameters of different domains. Inspired by the success of low-rank adaptation (LoRA) 100 14 9 38 we propose graph domain-specific PEFT. Specifically, for graph domain in the task sequence, we equip the pre-trained GNN with LoRA module. For the i-th domain, the I-th layer of the model v/s is calculated as:
Domain-specific Graph PEFT.
h' h'= E'h w!! (,(h' W! iodo W! kmpp)
where E'(-) denotes the output of the frozen GNN, (.) denotes the LoRA module of the i-th graph domain, and W! i,down E IRd xxr and W Wh.np E Brxad are two low-rank learnable parameter matrices, i.e., is much smaller than the parameter matrix dimension d' and d'. When learning new graph domain, we freeze previous domain-specific LoRA parameters, ensuring that learned graph domains remain stable in the embedding space and do not shift with the adaptation to the new domain.
j.up
Intra- and Inter-domain Disentanglement. To distinguish semantic differences between different graph domains as well as different classes within the same graph domain, we propose objectives for both intra- and inter-domain disentanglement, aiming to prevent confusion.
(1) Intra-domain Disentanglement, Within a single domain, our goal is to enhance the discrim- inability of node embeddings across different classes. To achieve this, introduce intra-domain

---

disentanglement objective based on contrastive learning. Specifically, we first generate augmented views for given Gy::
G C Aug(G.), X; = (G.) X, =b,(G,
where Aug(-) denotes feature and structure augmentation, X, and X are learned node embeddings corresponding to the original view and the augmented view, respectively, and 1! represents the domain-specific PEFT module in Eq. (3). For each node, define the positive samples 5 as the set of nodes belonging to the same class, and the negative samples S as the nodes from different classes. The intra-domain disentanglement objective can be formulated as:
Gil
EoesTr exp(sim (2r Wo aue)
Lintra
Σ log ==1
En exp exp(sim(( (1timiimmm..)) (. (
where sim(::-) denotes cosine similarity. This objective promotes semantic similarity among nodes of the same class while maintaining distinction between nodes of different classes in single domain.
(2) Inter-domain Disentanglement. With the intra-domain disentanglement, the embeddings for node of each class within a domain are compact to ensure clear discriminability between classes. However, in the Domain-IL scenario, due the feature semantics gap across different graph domains, there exist risks of unintentional overlap in the embedding space across domains. To prevent catas- trophic forgetting caused by this confusion between domains, we next introduce the representation scattering based inter-domain disentanglement objective.
Through freezing of domain-specific LoRA parameters, the embeddings of previous domains remain stable. Consequently, the inter-domain disentanglement objective can be defined as ensuring that the current domain is sufficiently distant from previous domains in the embedding space. However, previous graphs are not accessible in GIL Considering that the embeddings within each domain are compact and unchanged, after learning an incremental domain, we capture representative embedding prototypes through clustering, which represent the distribution of domains in the embedding space. Therefore, the inter-domain disentanglement objective can be expressed by pushing the samples of the current domain away from the embedding prototypes of all previous domains:
G.
Linter
IG Σ C2
1
P
2 +c
where P represents the set of embedding prototypes, and is a small constant. By minimizing the inter-domain disentanglement objective, the semantics of different domains are clearly separated, with no interference between them.
Overall Objective. The overall optimization objective can be formulated as:
L - 7/2 Cinur + 72 Cinters
where 71 and 72 are hyperparameters for trading-off. By minimizing Eq. 7 we can obtain highly discriminative node embeddings without confusion for both across different domains and across classes within each domain, thereby facilitating accurate knowledge preservation.
4.2 Deviation-Free Knowledge Preservation
Previous GIL methods usually integrate embedding learning and classification through end-to-end training framework. However, when learning new tasks, the classifier is updated together with the embedding model through back- poock-ropaggtion,, leading the deviation in the decision boundary. Although previous methods have attempted to solve this issue through techniques like memory replay, the limited quantity of memory data is insufficient when faced with the drastic parameter changes required to adapt to new domains.
Ridge Regression for Stable Classifier Updates. We aim to solve the decision boundary deviation issue by separating the classifier from the embedding model, thereby maintaining stable decision boundary that remains unaffected by new incremental domains, which also aligns with the stable embeddings we introduce the last section. this end, we introduce a ridge regression-based knowl- edge preservation mechanism, which does not require gradient updates through back propagopp,,,
5

---

thus avoiding the catastrophic forgetting caused by decision boundary deviations, inspired by [44 53 Specifically, for the first incremental domain, denote the stable and disentangled embedding as X Then, we fit the class labels Y1 through ridge regression, which is equivalent to:
arg min | X W, ll W W
(8)
where I1-1 indicates the Frobenius norm and À is the regularization coefficient. We can obtain closed-form solution for the optimal weight as W (x X1 X
Similarly, for the i-th incremental domain, the optimal ridge regression parameters W; which can also retain the knowledge of the previous i domains, can be calculated as:
W;- = (x (1:4) X (1:1) + XT) - x(1s)) Y (1:))-
Recursive Update without Historical Data Access. However, historical domains are not accessible in GIL, i.e., we cannot directly calculate Eq. (9). To solve this, we recursively update the parameter matrix only using data from the current graph domain:
W W; M, X,W M,
(10)
where indicates the concatenation and M; is an intermediate matrix recursively updated as:
M. M i (M+ -x,T))>>MM,,,, "X,M;-I. 1-1,
(11)
where M1 = (x x + AT) The detailed derivations are provided in Appendix A
Through Eq. (10) and Eq. (TT). we can precisely update the model parameters without using historical data while guaranteeing the optimal solution as Eq. ( For inference, given a graph Gg from the k-th domain, we obtain the predicted node labels as:
Y prodict Softmax(Xx W),
(12)
where X is calculated by Eq. (3) For graph Grest with unobservable domain, the corresponding domain-specifer PEFT module Prest in Eq. (3) is determined by the domain-aware distribution discrimination in Sec. 4.3 Since both the embeddings and the decision boundary remain stable, this
significantly reduces the risk catastrophic forgetting caused by drastic parameter changes.
4.3 Domain-aware Distribution Discrimination
To obtain precise embeddings, it is necessary to match the graphs from different domains with the corresponding PEFT module, which is easy to achieve for the training process, the domains of the training graphs are observable. However, for the test graph with an unobservable domain, is difficult directly match with the corresponding PEFT module. Therefore, we propose simple yet effective method for domain-aware distribution discrimination. The core idea is to match test graphs with prototypes of different domains.
Primarily, the features across multiple graph domains are at risk of being excessively similar or even overlapping, shown in Figure G.3] which may cause the test graph to be matched with the wrong domain prototype. To reduce the risk of prototype confusion, we aim to obtain domain prototypes with sufficient discriminability. Specifically, we first transform the features into high-dimensional space through a randomly initialized and frozen GNN for random mapping:
F GNN poog W ramdor))
(13)
where F denotes the transformed features. Though the GNN peoj is not trained, this high- sigh-dimensional random mapping [30 19 helps to separate prototypes between different domains, particularly when utilizing the feature distance the criterion for domain distribution discrimination. The confusion matrix of domain prototypes before and after mapping by Eq. (1) is shown in Figure E.2
Then, we determine the domain of the test graph by associating it with the nearest domain prototype. Specifically, we calculate the correlation between the test graph and domain prototypes based on the distances, and the domain with the highest correlation is chosen as the domain of the test graph:
G&
D2
F
Ctest = arg max(exx(- Dreat D2 D2
(14)
k

---

where Cest indicates the domain index, Des and Dj are the prototype of the test and k-th incremental graph domain, obtained through average pooling. Our digh-siiensiinal random mapping guarantees sufficiently distance between the prototypes of different domains, which results in minimal correlation between the prototype of the test graph and the prototypes of other domains, effectively preventing confusion in domain discrimination.
The overall algorithm and complexity analysis of GraphKeeper are shown in Appendix B
Experiments
5.1 Experimental setup
Datasets and Baselines.
To evaluate GraphKeepe + conduct comprehensive experiments 15 We choose variety of baselines,
real-world datasets, where detailed descriptions are in Appendix C.I
including general II. methods EWC [13]. MAS [1. GEM [18], LWF
15] and representative GIL. methods TWP [6]. ER-GNN (52), SSM 45 DeLoMe (22). PDGNNS 42 TPP [23] Additionally,
include self-designed baselines: Fine-Tune, which continuously training without any II.
techniques, and Joint, which utilizes all previous graph domains for training.
Implementation Details. We implement all methods under the GIL benchmark averaged over 5 independent runs for a fair comparison. More details are provided Appendix D
43 All results
Experimental Settings. To comprehensively evaluate in the Domain-IL -L scenario, we adopt multiple groups of graph domains. Due to the sigmificant differences between domains, the performance varies to some extent under different incremental orders. Therefore, set multiple incremental orders for each group and report the average results. Further details are provided in Appendix C.2
Evaluation Metrics, We adopt two commonly used metrics GIL: Average Accuracy (AA) and Average Forgetting (AF). Higher AA indicates better performance, and higher AF indicates less forgetting, as detailed in Appendix C.3
5.2 Performance on Domain-IL Scenario
Analysis. We can observe the following findings from the results of Domain-IL. scenario in Table I
While existing GIL methods can handle Task-IL and Class-IL scenarios, they show significant decline in performance in Domain-IL scenario, which indicates the inability of traditional GIL methods to adapt to the more challenging Domain-IL scenario. In contrast, Graph achieves state-of-the-an art results with 6.5%~~6..6% improvement over the runner-up and with negligible forgetting, which benefits from disentangled and stable representation for different incremental graph domains through the domain- speciffc graph PEFT to prevent confusion, and performing high- fideliiiylity knowledge preservation.
GraphKeeper outperforms the Joint baseline even though it has access to all previous graph domains. The results indicate that single GNN struggles to effectively integrate knowledge from multiple domains. In contrast, GraphKeeper tackles this issue by isolating parameters of different domains through the domain-specific graph PEFT.
DeLoMe and PDGNNs exhibit relatively passable performance. These methods utilize SGC [35] and APPNP [ their backbones, respectively, which sacrifices plasticity to mitigate forgetting. However, this limits their flexibility and applicability. We also try replacing their backbones as GCN and the results are summarized in Table E.3] showing a significant decline in performance. TPP utilizes prompts to adapt to different incremental graphs. However, its capability for modeling incremental graph domains is relatively constrained.
5.3 Studies of Integration with GFMs
To further explore the potential of GraphKeeper, we integrate it with two representative GFMs. GCOPE [49] and MDGPT [39], to equip them with the continuous updating capability. Specifically, we first pre-train the GFMs and then incorporate GraphKeeper with them evaluate the performance in the few-shot Domain-IL, scenario. The results are summarized in Table

---

Table I: The average performance (%) across different incremental orders in Domain-IL. scenario. AA indicates Average Accuracy and AF indicates Average Forgetting. + represents the standard deviation. The best results are indicated in bold and the runner-ups underlined.
Group
Group 2
Group 3
Group 4
Group 5
Group 6
Method
AA
AFT
AAT
AFT
AAT
AFT
AAT
AFT
A
AFT
AA
AF
Fine-Tune
23.9
-72.7t03
17.1 -74.4-07 67.6r04
20.9t0.1 -81.3-02
21.6:06 -77.8 19.7-a.1 74.5.0.
-77.8ta3
18.6.06
-77.9ta6
Joint
66.6.0.
78.0+0.
75.7-0x
71,5a05
EWC
23.3a0.1 -72.0ao.
17.3s04
-74,8at0
20.8
-80.9
22,4
1
20.6
-77.9au6
18.0
-76.6199
MAS
24.0a24 -70.5a43
16.8s51 -69.8a50
18.9225
-77.4
21.1a50 -75.5.77
18.9a34
-74.0a32
19.8x23 -71.5
GEM
23.3=32 -71.7456
10. -69.2-2s
21.7:23
-80.6-21
21.1
-77.4
20.0.25
-77.4s21
19,4+18 1
LWF
23.7.2. -72.5
17.0+07 -73.8-01 20.2
-80.7-02
10.1 -78.6
19.8-02
-77.9k04
18.8+03 -76.0aa7
TWP
23.5a0.1 -71.0a05
16.0a33
-71.0116
20.0a22
-79.0a07
20.2a2.
-79.0a1.
18.8a34
-74.4a12
18.2
-75.2s10
ER-GNN
23.3a14 -64.8
22.7x22
-66.3a29
28,7:30
-66.3a17
23.9st.7
-72.9a28
24.8a27
-68.8s3 8.3.9
24.7s47
-67. 65.9
SSM
24.1 -36.4s13.1
22. 1a36
-35.7
15.6
-31.0s4.1
25.3
-45.7a3.
25.7=28
-39.3x33
16.1457
-33.9248
TPP
52.6a18
0.0g0.
49.7a15
0.0ano
$7.1
0.0g0.0
53.0229
0.0ga0
56.72t.
0.0gou
48,
3424
0.0gn0
DeLoMe
49.3a0.1
-7.7a0.
58.225.1
-9.621.
70.2s4.1
-1.0g0.1
73.4
-1.1
63.225.7
-3.3
64.2a12
-4.8259
PDGNNs
52,4
1508
53,5
-7.4
65.5an6
-11.4 65.5
-7.2a08
64.3
-8.2203
60.2a02
-7.2a04
GraphKeeper 69.229.3
-0.4a02
73.1a1a
-2.8 80.6294 0.0go.1
79.9.a.
1.0
75.5
0.1 77.5 -0.1
Table 2: The results of integrating GraphKeeper with existing GFMs in the few-shot Domain-ll senario. AA indicates Average Accuracy and AF indicates Average Forgetting. + represents the standard deviation. Better results are indicated in bold.
Group 1
Group
Group 3
Group 4
Group 5
Group 6
Method
A
AF1
AA
AF1
AA
AF
AA
AF
AA
AF1
AA
AF
GCOPE 20.6:a9
-53.5
10.5a03
-36.7410
13.2a12
47.722
12.6a1
-510018
13.6.14
41.7s29
12.6209
-43.7a1.
GCOPE+Ours 56.8119
0.2g03
36.6214
0.3g03
47.4su
-1.6223
51.6216
-0.4206
44.3:40
0,8a05
44.9a09
-0.8g08
MDGPPT
19.9+1.
-59,4.2.
10.7.0,5
-42.7k2
12,1
-50.4.21
12.8a14
-52.9ks1
12.8
.47.6s99
12.3,0s
-48.8.33
MDGPT+Ours 59.7a12 -1,5a07
32.7a1e
-0.4207
49.5a21
-2.8a11
62.7a17 -14212
50.9216
-0.7a04
43.921.1 -0.2s10
Analysis. We observe that the original GFMs exhibit low AA and high AF. The high AF indicates that they perform well when trained separately on each graph domain, demonstrating strong few-shot capability. However, they lack the continuous updating capability with severe catastrophic forgetting issues, resulting in low AA. After integrating the GraphKeeper, they show significantly higher AA with negligible forgetting. This demonstrates that GraphKeeper can be seamlessly integrated into existing GFMs, combining their few-shot strength with the continuous updating capability to construct more powerful GFMs. Additionally, GraphKeeper does not require memory replay, which avoids the potential memory explosion problem, particularly on a large corpus of graphs.
5.4 Ablation Study
Ablation Study. We analyze three variants of Graph- Keeper. The results are summarized in Figure 4 and Figure E.
RX08 Curs
(DOC We DT
CQ Wif PEFT
68S8 w/o KP
GraphKeeper (w/o DT)): We replace the disen- tanglement objectives in Eq. ? with the vanilla contrastive objective that ignores domain-specific semantics.
ACC (86)
45
GraphKeeper (w/o PEFT): We remove the domain-specific graph PEFT module and perform continuous training on the pre-trained GNN model. GraphKeeper (w/o KP): We remove the deviation- free knowledge preservation module and utilize an MLP as the classifier for end-to-end training.
15-
Group
Group 2
Group 3
Figure 4: The ablation study results.
Analysis. Overall, we observe that removing any key component of GraphKeeper results in notice- able drop in performance, highlighting the importance of each module. (1) For GraphKeeper (w/o
8

---

DT), removing both intra-domain and inter-domain disentanglement objectives significantly weakens the model's ability to distinguish between classes and domains. This results in entangled embeddiings that cause confusion across domains and hinder generalization. (2) For GraphKeeper (w/o PEFT), excluding parameter-effiieent fine-tuning removes the isolation of domain-specifie parameters. As result, adaptation to new domains induces large parameter updates that affect previously learned domains, leading to embedding shifts and inconsistency with the preserved knowledge. (3) For GraphKeeper (w/o KP), the absence of knowledge preservation allows the classifier parameters to drift throughout training. This drift causes deviations in the decision boundary, disrupting the alignment between the learned embeddings and their corresponding labels. These results confirm that both embedding stability and decision boundary consistency are essential to mitigate catastrophic forgetting in the Domain-IL IIL setting, and that GraphKeeper achieves this through the synergistic effect of disentanglement, parameter isolation, and knowledge preservation.
5.5 Visualisation of Disentangled Embeddings
Embedding confusion is an important cause of catastrophic forgetting in Domain-IL scenario. To provide more intuitive comparison the advantages of GraphKeeper, we visualize the embeddings of different incremental graph domains with t-SNE 32 in Figure G.3 and Figure G.4
Analysis. As shown Figure G.3] the embeddings produced by GraphKeeper are highly compact with clear boundaries between domains, thanks to the introduction of multi-domain graph disentanglement, which effectively reduces interference across incremental domains. contrast, PDGNNs and DeLoMe exhibit noticeable domain overlap. Although they replay previous data, they still struggle to avoid conflicts between different domains. To further investigate the evolutions, we also visualize previously learned domains at different learning stages (Figure G.4). It can be observed that the embeddings of PDGNNs and DeLoMe become increasingly entangled over time, while those of GraphKeeper maintain high separability throughout, further corroborating our analysis.
5.6
Hyperparameters Analysis
We analyze the impact of hyperparameters Figure where r denotes the rank of the parameter matrix in domain-specinn graph PEFT module, and n//22 plays the trade-off role between intra- and inter-domain disentanglement objectives.
Analysis. (1) For hyperparameter r, a small value limits
the number of tunable parameters, making diffieult for the model to adapt effectively to new graph domains. As increases, the performance improves and achieves strong results while still using far fewer parameters than full-model tuning. (2) For hyperparameters Y1 and 722 when the ratio /1/12 decreases, meaning the inter-domain disentanglement objective becomes dominant, the model shows relatively poor performance due to insufficient class discrimination within each domain. On the other hand, when the intra- domain objective dominates, the performance also degrades because the lack of inter-domain disentanglement causes partial confusion between domains. A balanced trade-off between Y1 and ?2 yields the best performance.
(9)
75.8
75.8
76.5
72.9
16
32
64
128
725
ACC
0.01
0.1
100
Yu/r2
Figure 5: Hyperparameter analysis.
Conclusion
In this paper, we propose novel GII. framework, GraphKeeper, which effectively addresses catas- trophic forgetting in the Domain-ll. setting by mitigating embedding shifts and decision boundary deviations. The multi-domain graph disentanglement module learns stable and disentangled repre- sentations, thereby reducing interference among domains. In parallel, the deviation-free knowledge preservation mechanism maintains consistent decision boundary across tasks. Moreover, the domain-aware distribution discrimination enables precise embedding of graphs with unknown domain labels. Extensive experiments validate the effectiveness of GraphKeeper and highlight its strong potential for seamless integration into graph foundation models.

---

Acknowledgements
The corresponding author Qingyun Sun. Authors of this paper are supported by the National Natural Science Foundation of China through grants No.62302023 and No.62225202, the Fundamental Research Funds for the Central Universities JK2024-07, and the Guangzhou Basic and Applied Basic Research Foundation 2024A04J3681. We owe sincere thanks to all authors for their valuable efforts and contributions.
References
[1] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) forget. In ECCV, pages 139-154. 2018.
[2] Yuxuan Cao, Jiarong Xu, Carl Yang, Jiaan Wang, Yunchao Zhang, Chunping Wang, Lei Chen, and Yang Yang. When pre-train graph neural networks? from data generation perspective! In SIGKDD, pages 142-153, 2023.
[3] Ines Chami, Zhitao Ying. Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks. NewrIPS, 32, 2019.
[4] Yuanning Cui, Yuxin Wang, Zequn Sun, Wenqiang Liu, Yiqiao Jiang, Kexin Han, and Wei Hu. Lifelong embedding learning and transfer for growing knowledge graphs. In AAAI, volume 37, pages 4217-4224, 2023.
[5] Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. A density- algorithm for discovering clusters large spatial datahases with noise. In kdd, volume 96, pages 226-231, 1996.
[6] Falih Gozi Febrinanto, Feng Xia, Kristen Moore, Chandra Thapa, and Charu Aggarwal. Graph lifelong lcarning: A survey. IEEE Computational Intellignnce Magazine, (6(1))32-51, 2023.
[7] Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King, Magun: Metapath aggregated graph neural network for heterogeneous graph embedding. In The Web Conference, pages 2331 2331-1 2020.
[8] Johannes Gasteiger, Aleksandar Bojehevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:/810.05999, 2018.
[9] Anchun Gui, Jinqiang Yc, Han Xiao. G-adapter: Towards structure- aware srrameter-efficieee transfer learning for graph transformer networks. In AAAI, volume 38, pages 12226 12234, 2024.
[10] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2)3, 2022.
[11] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
[12] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ICIR, 2017.
[13] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- -arekk-kaaiikka et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-335 -3526, 2017.
[14] Shengrui Li, Xueting Han, and Jing Bai. Adaptergnn: Parameter-efficient fine-tuning improves generaliza- tion in gnns. In AAAI, volume 38, pages 13600-13608 2024.
[15] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions pottern analysis and machine intelligence, 40(12):2935 2947, 2017.
[16] Huihui Liu, Yiding Yang. and Xinchao Wang. Overcoming catastrophic forgetting in graph neural networks. In AAAI, volume 35, pages 8653-8661, 2021.
[17] Yilun Liu, Ruihong Qiu, and Zi Huang. Cat: Balanced continual graph learning with graph condensation. In ICDM, pages 1662. IEEE, 2023.
[18] David Lopez- Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. NewrIPS, 30, 2017.
10

---

[19] Mark D McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton van den Hengel. Ranpac: Random projections and pre-trained models for continual learning. Advances in Neural Information Processing Systems, 36, 2024.
[20] Péter Mernyei and Cätälina Cangea. Wiki-es: A wikipedia-base based benchmark for graph neural networks. arXiv preprint arXiv:2007,02900 2020.
[21] Galileo Namata, Ben London, Lise Getoor, Bert Huang, and U Edu. Query-driven active surveying for collective classification. In 10th international workshop on mining and learning with graphs, volume 8, page 1, 2012.
[22] Chaoxi Niu. Guansong Pang, and Ling Chen. Graph continual learning with debiased lossless memory replay. In ECAI 2024-27th European Conference on Artificial Intelligence, Including 1.thh Conference on Prestigious Applications of Intelligent Systems, PAIS 2024, Proceedings, pages 1808-1815. IOS Press BV, 2024.
[23] Chaoxi Niu, Guansong Pang. Ling Chen, and Bing Liu. Replay-and-forget-- graph class-incremental learning: A task profiling and prompting approach. Advances in Neural Information Processing Systems, 37:87978- 88002,, 2024,
[24] Benedek Rozemberczki, Carl Allen, and Rik Sarkar. Multi-scalc attributed node embedding. Journal of Complex Nenworks, 9(2)enab014, 2021.
[25] Benedek Rozemberezki and Rik Sarkar. Characteristic functions graphs: Birds of feather, from statistical descriptors parametric models. In CIKM, pages 1325-1334, 2020.
[26] Prithviraj Sen. Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data, Al magazine, 29(3):93-99) 2008
[27] Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprint arXiv: 181 1.05868, 2018
[28] Bo-Shen Shi, Yong-Qing Wang. Fang-Da Guo, Bing-Bing Xu, Hua- Wei Shen, and Xue-Qi Cheng. Domain adaptation for graph representation learning: Challenges, progress, and prospects. Journal of Computer Science and Technology, pages 1-18, 2025.
[29] Gilhert W Stewart. On the early history of the singular value decomposition. SIAM review, 35(4):551-566, 1993.
[30] Jiaqi Sun, Lin Zhang, Guangyi Chen, Peng Xu, Kun Zhang, and Yujiu Yang. Feature expansion for graph neural networks. In ICML, pages 33156-33176. PMLR, 2023.
[31] Li Sun, Junda Ye, Hao Peng, Feiyang Wang. and Yu Philip. Self Supe-super continual graph learning in adaptive riemannian spaces. In AAAI, volume 37, pages 4633-4642, 2023.
[32] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne, Journal of machine learning research, 9(11), 2008
[33] Junshan Wang, Wenhao Zhu, Guojie Song, and Liang Wang. Streaming graph neural networks with generative replay. In SIGKDD, pages 1878-1888, 2022.
[34] Di Wei, Yu Gu, Yumeng Song, Zhen Song, Fangfang Li, and Ge Yu. Incregnn: Incremental graph neural network learning by considering node and parameter importance. In DASFAA, pages 739-746. Springer, 2022.
[35] Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplifying graph convolutional networks. In ICML, pages 6861-6871. PMLR, 2019.
[36] Man Wu, Xin Zheng, Qin Zhang, Xiao Shen, Xiong Luo, Xingquan Zhu, and Shirui Pan. Graph learning under distribution shifts: comprehensive survey on domain adaptation, out distr--dittibttinn, and continual learning. arXiv preprint arXiv:2402. 16377 2024.
[37] Lianghao Xia and Chao Huang. Anygraph: Graph foundation model the wild. arXiv preprint urXiv:2408. 10700, 2024.
[38] Zhe- Rui Yang. Jindong Han, Chang-Dong Wang. and Hao Liu. Graphlora: Structure- aware contrastive low-rank adaptation for cross cross-graph transfer leaming. In Proceedings of the 31 ACM SIGKDD Conference Knowledge Discovery and Data Mining 1. pages 1785 1776, 2025.
11

---

[39] Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Text-free multi-domain graph pre-training: Toward graph loundation models. arXiv preprint urXiv:2405.139333 2024.
[40] Qiao Yuan, Sheng-Uei Guan, Pin Ni, Tianlun Luo, Ka Lok Man, Prudence Wong, and Victor Chang. Continual graph learning: A survey, arXiv preprint arXiv:2301.12233, 2023.
[41] Peiyan Zhang, Yuchen Yan, Chaozhuo Li, Senzhang Wang. Xing Xie, Guojie Song, and Sunghun Kim. Continual learning dynamic graphs via parameter isolation. In SIGIR, pages 601-611, 2023.
[42] Xikun Zhang, Dongjin Song. Yixin Chen, and Dacheng Tao. Topology-aware embedding memory for continual learning expanding networks. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Duta Mining, pages 4326-4337, 2024.
[43] Xikun Zhang. Dongjin Song, and Dacheng Tao. Cglb: Benchmark tasks for continual graph learning. NeurIPS, 35:13006 13021, 2022.
[44] Xikun Zhang, Dongjin Song, and Dacheng Tao. Hierarchical prototype networks for continual graph representation learning. IEEE Transactions on Puttern Analysis and Machine Intelligence, 45(4):4622- 4636, 2022.
[45] Xikun Zhang, Dongjin Song, and Dacheng Tao. Sparsified subgraph memory for continual graph represen- tation learning. In ICDM, pages 1340. IEEE, 2022.
[46] Xikun Zhang, Dongjin Song, and Dacheng Tao. Ricci curvature-base graph sparsification for continual graph representation learning. IEEE Transactions on Neural Networks and Learning Systems, 2023.
[47] Xikun Zhang, Dongjin Song, and Dacheng Tao. Continual learning on graphs: Challenges, solutions, and opportunities. arXiv preprint arXiv:2402. 1165, 2024.
[48] Ziwei Zhang, Haoyang Li, Zeyang Zhang. Vijian Qin, Xin Wang, and Wenwu Zhu. Graph meets Ilms: Towards large graph models. In NeurlPS 2023 Workshop: New Frontiers in Graph Learning, 2023.
[49] Haihong Zhao, Aochuan Chen, Xiangguo Sun, Hong Cheng, and Jia Li. All in one and one for all: simple yet effective method towards cross-domain graph pretraining. In SIGKDD, pages 4443-4454, 2024.
[50] Jianan Zhao, Hesham Mostafa, Mikhail Galkin, Michael Bronstein, Zhaocheng Zhu, and Jian Tang. Graphany: A foundation model for node classification on any graph, arXiv preprint AXiv:2405.20445 2024.
[51] Junhao Zheng, Shengjie Qiu, and Qianli Ma. Learn or recall? revisiting incremental learning with pre- trained language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14848-14877, 2024.
[52] Fan Zhou and Chengtai Cao. Overcoming catastrophic forgetting in graph neural networks with experience replay. In AAAI, volume 35, pages 4714-4722, 2021.
[53] Huiping Zhuang, Yizhu Chen, Di Fang, Run He, Kai Tong, Hongxin Wei, Ziqian Zeng, and Cen Chen. Gacl: Exemplar-free free generalized analytic continual learning. In NeurlPS, 2024.
[54] Huiping Zhuang, Zhenyu Weng, Hongxin Wei, Renchunzi Xie, Kar-Ann Toh, and Zhiping Lin. Acil: Analytic class-incremental learning with absolute memorization and privacy protection. NewrlPS, 35:1 1602- 11614, 2022.
12

---

A Derivation of Eq. (10) and Eq.
We first restate Eq. (10) and Eq.
for reference.
Eq. TT):
W. W --1 M; X Y,),
where indicates the concatenation and M; is an intermediate matrix. Eq.
M. M,-1 M [X] X1 +
(1+X, M
",,,)))""MM,,, M-M,
where M1
Proof.
We begin by defining the closed-form solution of ridge regression at step as:
W = X (1:k) Y (1:)).
(A.1)
where M = (X X (188) + XI) This can be recursively expressed using the matrix inversion
lemma as:
M = (M X Xx)
(A.2)
Applying the Woodbury matrix identity, we derive the recursive update of M
M M M + X M
(A.3)
which corresponds to Eq. [T]. We then turn to the update of W.- By definition:
Wx k = M, (1:4) Y (1sk)
T (1
M [ [TTT 1) Y ((1=-1)) X Y
= x (1:k-1) Y (1=k-1) || M Y
(A.4)
We now focus on computing the first term. Substituting Eq. AA.3 into this term yields:
M M (1ik-1) Y (1:--1) MR-1 X (1:k-1) Y (1ik-1)
-M k X (I X xI) X k- X (1 (1:K-1)
W M M (I - W-
(A.5)
To simplify the matrix expression, we define:
K, (I X
(A.6)
From the identity K2 + M x = I, we can derive:
M
K₁ = --=--K -
(A.7)
Using this, we rewrite the matrix product as:
M M X
M
M
(A.8)
Substituting Eq. (A.8) into Eq. A.5 gives:
M (1: (1ik-1)
W M,X, X.W WA-1-
(A.9)
Finally, substituting Eq. A.9 back into Eq. A.4 obtain:
Wx #
W
M X X k-1 MMMX Y R).
(A.10)
where MA recursively updated via Eq. A.3
This completes the derivation of Eq. (10) and Eq.
13

---

B Algorithms and Comlexity Analysis
B.1 Algorithms
The overall training and inference process GraphKeeper are given in Algorithm and Algorithm
Algorithm 1: The overall training process of GraphKeeper
Input: Graph domain sequence S [GI,.... Gr Pre-trained GNN & Number of training epochs E.
Output: Ridge regression parameters W; Domain-spectinn LoRA parameters { p..... Φ
Domain prototypes , Random projection function GNN pmj--
Randomly initialize and frozen the parameters of GNN proj:
Frozen the parameters ξ:
Initialize the P of the embedding prototypes;
for ? ,,.,.,,,1
T do
// Multi-domain Graph Disentanglement
Align the feature dimensions of G, by Eq. (2); Initialize the LoRA parameters Φ, for G₁;
for epoch I..... E do Train the LoRA parameters Φe by Eq. 7 end
10
Calculate the embedding X, by Eq. 33:
11
Add new embedding prototypes into set P by clustering;
// Deviation Free Knowledge Preservation
12
Update the ridge regression parameters W by Eq. (10) and Calculate the domain prototype Dú;
13
14 end
Algorithm 2: The inference process GraphKeeper
Input: Test graph Grea; Ridge regression parameters W; Pre-trained GNN &:
Domain -peein- LoRA parameters {ΦI..... Φr Domain prototypes
[Di...., Dr Random projection function GNN proj.
Output: Predicted result of the test graph Grest-
// Domain- aware iistributinn Discrimination
Calculate the test prototype Dieat'
Discriminate the domain of Gwest by Eq.
// Inference
Calculate the embedding X test with & and diea by Eq.
Get the predicted result by Eq. [2);
B.2 Comlexity Analysis
For brevity, d donates the unified feature dimension, |G,] donates the number of nodes in the i-th graph domain, and donates the model's hidden layer dimension. For a single graph domain, with the model containing 2 layers, the complexity of the forward propagation of the frozen pre-trained GNN is O(G)dd + |G'dd)) dP), and the complexity of LoRA forward and backward propagation is (2(( rd) + (drr) O (2)) + 3d)), where is the rank of LoRA module. The complexity of intra-domain disentanglement loss calculated by Eq. (5) is ( and the complexity of inter- domain disentanglement loss calculated by Eq. (6) is P (d), where [P] donates the number of embedding prototypes. Given the number of the training epochs E, the combined complexity of these components (E( + + + GG IP]) + 2rd))). For Eq. (T)) and Eq. (TT), the primary computational cost arises from matrix inversion, with complexity of ((G)) but this performed only once. The overall computational complexity is O [[Gldl(ddd + d 6r [G.] + P|) + 2rd)) G As the feature dimension d, the rank r, hidden layer dimension the number of embedding prototypes IP]. and the training epochs E are relatively small compared to
14

---

the number of nodes, the overall computational complexity can then be approximately reduced to ( 1 )
- IG,
C
Experiment Details
C.1 Datasets
We conduct experiments on 15 real-world datasets, including academic networks (Cora [26]. Cite-
seer [2]. PubMed 21I. CoauthorCS [27), and DBLP (7). co-purchase networks (Photo 22 and
Computer (27)), web networks (WikiCS [20], Facebook (24]. Chameleon [24], and Squirrel (24)..
social networks (GitHub (44, LastFMAsia [25], and DeezerEurope (251), and airline networks (Air-
port B. Statistics of datasets are concluded in Table [C. All the datasets are consented to by the
authors for academic usage. All the datasets do not contain personally identifable information offensive content.
Table C.I: Statistics of datasets.
Dataset
# Nodes
# Edges
# Features
# Classes
Homophily
Cora
2,708
5,429
1,433
7
0.81
Citeseer
3,327
4,732
3,703
6
0.74
PubMed
19,717
44,338
500
3
0.80
CoauthorCS
18,333
163,788
6,805
0.81
DBLP
17,716
105,734
1,639
4
0.83
Photo
7,650
119,081
745
8
0.83
Computer
13,752
245,778
767
0.78
WikiCS
11,701
431,726
300
10
0,65
Facebook
22,470
342,004
128
4
0.89
Chamelcon
2,277
36,101
2,325
5
0.24
Squirrel
5,201
217.073
2,089
5
0.22
GitHub
37,700
578,006
128
0.85 2
LastFMAsia
7,624
55,612
128
0.87
DeezerEurope
28,281
185,504
128
2
0.53
Airport
3,188
18,631
4
0.72
C.2
Experiment Setting
C.2.1 Domain-IL. Scenario
To comprehensively evaluate the effectiveness of methods in Domain-IL scenario, we obtained 6 increment groups of graph domain from 15 datasets, as shown in Table C.2 Group to Group consist of graph domains from the same type (e.g., all are social networks in group), while Group to Group include graph domains from mixed types. For the group containing graph domains A, B, C. and D, evaluate in incremental orders of A B C - D. + D + A, C
A + B, and D A B + C, respectively. For each graph domain, we set the unified dimension of the features 512 and split the training set, validation set, and test set in proportions of 60%, 20%, and 20%. For the few-shot setting, we sample 10 labeled nodes from all classes within each graph domain for training.
C.3 Evaluation Metrics
The accuracy matrix is denoted as M, where Mi,j represents the accuracy on domain j after learning domain i. We adopt two commonly used metries: Average Accuracy (AA) and Average Forgetting (AF). The AA is computed as + E MT.j, which represents the average performance across domains after learning all domains. On the other hand, AF quantifies forgetting across domains through - E (Mr, M2.). which represents the difference between the final performance on each domain and the performance on the domain when it was first learned. Higher AA indicate better performance and higher AF indicates less forgetting.
15

---

Table C.2: Statistics of incremental groups of graph domain.
Group
Graph Domains
Group
GitHub, LastaMAsia, DeczerFurope
Group
WikiCS, Facebook, Chameleon, Squirrel
Group 3
Citescer, Pubmed, CoauthorCS, DBL.P
Group 4
Pubmed, Photo, WikiCS, Airport
Group 5
CoauthorCS,, Computer, Chameleon, DeezerFurope
Group
Cora, Facebook, LastFMAsia, Squirel
GitHub, LastFMAsia, DeezerEurope, WikiCS, Faccbook, Chameleon, Squirrel, Citeseer, Pubmed, CoauthorCS, DBLP
Group 7
Pubmed, Photo, WikiCS, Airport, CoauthorCS, Computer, DeezerEurope, Cora, Facebook, LastFMAsia, Squirrel
Group 8
C.4
Running Environment
We conduct the experiments with:
Operating System: Ubuntu 20.04 LTS.
CPU: Intel(R) Xeon(R) Platinum 8358 CPU@2.60GHZ with 1TB DDR4 of Memory,
GPU: NVIDIA Tesla V100 with 32GB of Memory.
Software: CUDA 11.7, Python 3.8.0, Pytorch 1.7.1, DGL 0.6.1.
D
Implementation Details
D.1 Implementation Details of GraphKeeper
We set the number of model layers to for all methods, with the learning rate set 5e-2, the weight decay coefficient set to 5e-4, and 200 training epochs per incremental graph domain. The parameters of models are optimized by Adam [I]. For GraphKeeper, the pre-trained GNN model is trained through link prediction on the first incremental domain (except for Sec. 5.3). the feature and structure augmentation are achieved by randomly masking few features and dropping few edges, and the embedding prototype sampling is implemented through DBSCAN [5]. The hyperparameter is chosen from (4, 8, 16, 32, 64, 128), " chosen from (0.01, 0.1. 1.0, 5.0, 10.0). ?2 chosen from (0.001, 0.01, 0.10, 0.50, 1.0).
D.2 Implementation Details of Baselines
We implement all methods under the GIL benchmark
43
EWC [3), MAS M. GEM 18) LWF 15), TWP 16). ER-GNN [2]
https://ggtthubb.com/uuuuuuu
[CC BY 4.0 License].
SSM 45):
htttee///giihubb..co////ee///// [CC BY 4.0 License].
TPP 23
http:://gitthbb.cooamaaaaaaaaaap [MIT License].
PDGNNs httpp:///tttuuu..comm/////AAA/1 |with license unspecified].
DeLoMe 22:
httgs///giaabb.cmmmaaaaaaa/////p
[CC BY 4.0 License].
E
Additional Experiment Results
E.1
Comparison of Methods with Unified GCN Backbone
Considering that some baselines adopt simplified GNN such as SGC 35 and APPNP [ as their backbone, they achieve stability at the cost of sacrificing model plasticity. However, constraining the backbone of the model imposes limitations on practical applications, especially when the model architecture cannot be replaced. Thus, unity the backbone of related methods to GCN for fair
16

---

Table E.3: Comparison of performance with the unified GCN backbone.
Group 1
Group
Group 3
Group 4
Group
Group 6
Method
AA f
AF
AA
AF
AA
AFT
AA
AFT
AA
AF1
AF
DeLoMe
49.2,21
20,9 32.2+14
-51.8-12
56.9103
1.38 40.9.13 44.8+1. 49.1+13
-31,722
45,4+15 -35.3x24
PDGNNs
48.0s24
-33.4
42.3+25
+39,1229
$3.4
40.9a12 -35,5a09
49,44.24 49.6.18 136,1
45.1er5 138.1
GraphKeeper 69.2an3
-0.4a02 73.1a1.1
-2.8.07 80.6.0.4
0.0aa11 79.9aus
1.0a0. 75.5sa.9
O.1ann
77.5a07
-0.1
Table E,4: Performance comparison on longer Domain -IL sequences.
Group (11 Domains)
Group Domains)
Method
AA f
AF
AA
AF
TPP
50.2a0.
0.0as.
52.8-25
0.0.00
DeLoMe PDGNNS
> day
> day
37.7.0.1
-13.1
45.0.02
-13.6-03
GraphKeeper
73.0.1.1
-0.7a05
76.5.n9
-0.8.02
comparison. The results summarized in Table E.3 It show that after unifying the backbone, the performance of the baselines declines to a certain extent, significantly reducing their usability. This also highlights the advantage of our approach, which imposes no restrictions model architecture, achieving dual benefit of plasticity and stability.
E.2 Longer Incremental Learning Sequence
We conduct evaluation two longer incremental sequences, Group 7 and Group 8, containing 11 and 12 graph domains, respectively, which are more challenging. As shown in Table [E.4] the advanced baselines suffer significant performance drops, while GraphKeeper maintains strong performance and outperforms the runner-up by 22.8%~~3..7%% This demonstrates the scalability of GraphKeeper and its robustness to larger data scales.
E.3 Ablation study
Due to space limitations, the ablation study from Group 4 Group 6 are presented in Figure E.T showing that removing any component of Graph- Keeper leads to degradation of the performance, especially for GraphKeeper (w/o PEFT) and Graph- Keeper (w/o KP) variants. Specifically, the full model consistently outperforms all ablated variants across the three groups. Notably, removing PEFT or KP causes the most significant drops (e.g., from 79.9% to 21.3% in Group 4), highlighting their crucial roles in preserving prior knowledge and supporting cross-domain alignment. The performance of w/o DTT is also inferior, confirming the importance of domain transfer modules. These results verify the necessity of each component for robust generalization.
PEFT
90
77.5
75
Group
Group
Group 6
Figure E.1: Ablation study.
E.4 Domain Discrimination
Through random high-dimensional mapping, we achieve linear separability between domain features that may potentially overlap. As shown in Figure all domains exhibit a significant reduction in pairwise confusion after mapping. This provides stability guarantee for our subsequent domain prototype matching.
17

---

1.0
0x
0
Dy
Pa
0, 0, 9, 6, Py Pa.
9 9. 6, 0, 4, P,000
.0.0
Figure E.2: Confusion matrix of domain prototypes. Deeper color indicates higher proximity. Left: Original. Right: Ours.
Further Discussion on Related Work
In Sec. we introduce the domain-aware distribution discrimination match graphs with unob- servable domains to previous domain prototypes. Recent work (23] discusses similar issue, they propose applying Laplace smoothing on original features and matching the test graph with the class prototypes through cosine similarity. Unfortunately, as shown in Figure G.3] the features across multiple domains are risk of being excessively similar even overlapping, which may cause the test graph to be matched with the wrong domain prototype. We guarantees sufficiently distance between the prototypes of different domains through high-dimensional random mapping, use distance metrics in high-dimensional space the criteria for matching, effectively preventing confusion in domain discrimination.
G Limitations
The limitations of GraphKeeper include the following aspects. First, when evaluating the performance of GraphKeeper in Domain-IL scenario, we followe the traditional setting, where the classes the new incremental domain are assumed to be entirely novel. However, more general and practical cases, such as partially overlapping classes across domains, have not been thoroughly explored. Additionally, although GraphKeeper their continual updating capabilities based on their few-shot learning strengths, it does not fully leverage the zero-shot learning potential of GFMs. This limitation tied to the scenario design, does not account for previously unseen domains, an aspect that has been overlooked by all existing methods. In summary, future work will extend the GraphKeeper's applicability, and investigate more general scenario in GIL.
be seamlessly integrated into existing GFMs, enhancing
18

---

Raw Features
PDGNNs
DeLoMe
Group 1
Group 2
Group 3
Group
Group 5
Group 6
Figure G.3: Visualization of the node embeddings of different graph domains after incrementally learning all graph domains. Each color corresponds graph domain. The embeddings produced by GraphKeeper are highly compact with clear boundaries between different graph domains.
19

---

1 Domain Learned
2 Domains Learned
3 Domains Learned
Domains Learned
PDGNNS
Del MM
ours
Figure G.4: Visualization the node embeddings of previously learned domains at different stages of the learning process on Group 6.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately represent the paper's contributions and scope, with all results and findings clearly and correctly reflected.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
See Appendix for discussion on limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
21

---

We provide detailed proof in Appendix A for two theoretical derivations in Eq. (TO) and Eq.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have provided the code, datasets, training scripts, and algorithms (See Sec 5.1 and Appendix [B. which collectively provide comprehensive details for fully repro- ducing the experiments and verifying the results presented in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
22

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The code and datasets available at:
attps://anooymouss 4open
science/r/GrrapKeeppe
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ https/cccppllligggddeeeeeeeeeeee Cooee///ddeeebmmmcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All experimental setting and details are provided in Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We have provided the variance in the experimental results tables, which allows for an understanding of the variability and reliability of the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
23

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
See Appendix C.4 for discussion on experiments compute resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code ethics
Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Question:
Answer: [Yes]
The research conducted this paper adheres to aspects of the NeurIPS Code of Ethics. No ethical issues were identified in the methodology, data collection, or result interpretations.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This paper enables adaptive and scalable graph learning, promotes knowledge sharing, and supports privacy-friendly applications without storing past data.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The paper does not involve the release of data models that have high risk for misuse, hence the question of safeguards does not apply this context.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
The paper appropriately cites and respects the licenses and terms of use of the datasets and baseline code used the work.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
25

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces GraphKeeper method and provides code access via an anonymous repository.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
26

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
27


Reviews:
[{"review_id": "S3bIX7D7Z2", "reviewer": "Reviewer_HUJf", "summary": "This paper addresses the problem of Graph Domain-Incremental Learning (Domain-IL), where a graph model is required to learn sequentially from graphs belonging to different domains without forgetting previous knowledge. The authors identify two key challenges, embedding shifts and decision boundary drift, that lead to catastrophic forgetting in this setting. To address these, they propose GraphKeeper, a framework that combines domain-specific parameter-efficient fine-tuning, intra- and inter-domain disentanglement, and a ridge regression-based classifier to stabilize knowledge across domains. Experiments on 15 graph datasets show that the proposed method achieves consistent improvements over existing graph incremental learning baselines.", "questions": "- The proposed method focuses on preventing forgetting through domain-specific parameter isolation but does not address how knowledge can be shared or transferred across related domains. Have the authors considered mechanisms to enable forward transfer or knowledge accumulation in Domain-IL settings?\n\n- Can the authors provide further analysis or experiments to assess the robustness of this domain assignment and its impact on downstream performance?", "limitations": "The key limitations are summarized in the weaknesses section above.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- The paper is the first to systematically address Graph Domain-Incremental Learning (Domain-IL), a practically relevant yet underexplored setting in graph learning.\n\n- The proposed combination of domain-specific parameter-efficient tuning (PEFT) and ridge regression-based knowledge preservation is well-motivated, computationally efficient, and compatible with graph foundation models.\n\n- Extensive experiments on 15 diverse graph datasets, combined with careful ablations and visualization, consistently demonstrate the effectiveness of the method in mitigating catastrophic forgetting.\n\nWeaknesses\n\n- The proposed framework isolates parameters for each domain to prevent forgetting, but this design inherently limits positive transfer across related domains. There is no mechanism to extract, share, or accumulate transferable knowledge when structural or semantic similarities exist between domains. While the method effectively mitigates forgetting, it overlooks forward transfer and knowledge accumulation, which are essential components of continual learning.\n\n- The method assumes access to domain labels during training and uses heuristic prototype matching for domain assignment at test time. This approach can be fragile when domains are similar, potentially leading to incorrect parameter selection and degraded performance. The paper does not analyze or address the impact of such domain misclassification.\n\n- While the paper provides formal derivations for its classifier update and disentanglement objectives, it lacks a deeper theoretical justification or generalization analysis explaining why these mechanisms prevent forgetting in the Domain-IL setting.\n\n- Maintaining separate PEFT modules for each domain may incur significant memory and computational costs as the number of domains grows, which is not discussed.", "was_revised": true, "final_justification": "I decided to raise my score as the authors have provided thorough theoretical and empirical justifications addressing my concerns, particularly regarding knowledge transfer, robustness under domain similarity, and scalability. Their clarifications support the paper's contributions to domain-incremental learning on graphs.", "rebuttal": "We appreciate your comments. To address your concerns, below we provide further analysis and experiments.\n\n---\n\n> **Q1: The knowledge transfer across related domains (Weakness 1, Question 1)**\n> \n\n**A1:** We sincerely appreciate your insightful comment on cross-domain knowledge transfer in domain-IL settings.\n\n- We fully agree that forward transfer is promising for incremental learning. But importantly, Graph Domain-IL has no prior work, where the fundamental challenge is catastrophic forgetting caused by structural and semantic gaps, which is the most essential capability for incremental learning. As the first exploration in this underexplored setting, GraphKeeper establishes a foundational anti-forgetting framework, which is the main focus and contribution of our work. \n\n- Regarding cross-domain knowledge transfer, we observed that our method exhibited positive transfer in the incremental learning process of some graph domains (AF was positive, as shown in Table 1 of the paper). It should be emphasized that, despite domain-specific parameter isolation, nodes from all incremental domains are embedded into a unified semantic space. Consequently, as our closed-form ridge regression classifier employs recursive updates in the new domain, it triggers adaptive adjustments to the decision boundaries of previous graph domains. This collaborative boundary evolution demonstrates the potential for implicit positive transfer. We fully agree with your vision regarding positive transfer and are actively exploring mechanisms for cross-domain parameter sharing while combining them with domain-specific parameters. Since this requires addressing new challenges in domain-invariant knowledge and negative transfer, it constitutes future work to be pursued and falls outside the scope of this work.\n\n---\n\n> **Q2: The impact of similar domains for Domain-aware Distribution Discrimination (Weakness 2, Question 2)**\n> \n\n**A2:** We thank the reviewer for raising this insightful concern. Below, we provide a systematic response through theoretical theorems and supplementary experiments. The supplementary analyses will be incorporated into the revised version.\n\n**Theorem 1. Linear Separability via Random High-Dimensional Mapping:** Let $D_i$ and $D_j$ be feature distributions of two graph domains with overlap $\\kappa \\in [0,1]$. For a fixed random mapping $\\phi:\\mathbb{R}^{d} \\rightarrow \\mathbb{R}^{m}$, where $m \\gg d$, the mapped features satisfy:\n\n$$\n\\mathbb{P}\\left[\\left\\| \\phi(\\boldsymbol{x}_i \\sim \\mathcal{D}_i) - \\phi(\\boldsymbol{x}_j \\sim \\mathcal{D}_j) \\right\\|_2 \\geq \\Delta \\right] \\geq 1 - \\delta\n$$\n\nwhere $\\Delta >0$ is a separation margin, and $\\delta$ decays exponentially with $m$. Even when $D_i$ and $D_j$ overlap to a large extent, random projection amplifies distributional differences in high-dimensional space, ensuring linear separability with high probability if $\\kappa \\neq 1$. This stems from the Johnson-Lindenstrauss Lemma and geometric properties of random projections.\n\n**Theorem 2. Stability of Prototype Matching:** Let $\\boldsymbol{x}\\_{\\text{test}} \\sim \\mathcal{D}\\_{k}$ be the test node features from domain $k$, and $\\boldsymbol{p}\\_{k} = \\mathbb{E}\\_{\\boldsymbol{x}\\_{\\text{train}} \\sim \\mathcal{D}\\_{k}}[\\phi(\\boldsymbol{x}\\_{\\text{train}})]$ be the true domain prototype. Given a fixed random mapping $\\phi$ and identical train and test distributions:\n\n$$\n\\left\\| \\boldsymbol{p}_{\\text{test}} - \\boldsymbol{p}_k \\right\\|_2 \\leq \\epsilon_1\n$$\n\nFor mismatched domain prototypes $\\boldsymbol{p}_{j}$, where $j \\neq k$:\n\n$$\n\\left\\| \\boldsymbol{p}_{\\text{test}} - \\boldsymbol{p}_j \\right\\|_2 \\geq \\Delta - \\epsilon_2\n$$\n\nwhere $\\Delta$ is the separation margin from Theorem 1. Since the test graph's nodes share the same distribution as the training nodes of its source domain, test graphs converge to their true domain prototype $\\boldsymbol{p}\\_k$ (i.e. $\\epsilon_1$ tends towards zero). Meanwhile, even when the feature distributions across domains exhibit substantial overlap but are not identical (i.e. $\\kappa \\neq 1$), coupling this with a sufficiently high mapping dimension ensures that $\\Delta - \\epsilon_2$ tends to be sufficiently large. This guarantees that the lower bound of $\\left\\| \\boldsymbol{p}\\_{\\text{test}} - \\boldsymbol{p}\\_j \\right\\|\\_2$ is always greater than the upper bound of $\\left\\| \\boldsymbol{p}\\_{\\text{test}} - \\boldsymbol{p}\\_k \\right\\|\\_2$.\n\n**Supplementary experiments.** We conducted robustness evaluations under different domains with high distributional overlap. Specifically, we randomly sampled graphs from four mutually similar distributions as four graph domains, where the pairwise domains feature distribution overlap exceeded 80%. For each domain, we partitioned training and test nodes to derive domain prototypes and test graph prototypes. The confusion matrix below shows the probability of our method correctly predicting the source domain of each test graph.\n\n||$\\boldsymbol{p}_{1}$|$\\boldsymbol{p}_{2}$|$\\boldsymbol{p}_{3}$|$\\boldsymbol{p}_{4}$|\n|---|---|---|---|---|\n|$\\boldsymbol{p}_{1}^{\\text{test}}$|**38.16%**|22.77%|18.91%|20.14%|\n|$\\boldsymbol{p}_{2}^{\\text{test}}$|21.02%|**38.94%**|21.30%|18.72%|\n|$\\boldsymbol{p}_{3}^{\\text{test}}$|19.57%|23.72%|**39.79%**|16.91%|\n|$\\boldsymbol{p}_{4}^{\\text{test}}$|23.49%|20.47%|16.49%|**39.52%**|\n\nThese results demonstrate that even when confronted with highly similar cross-domain distributions, our method ensures that the matching probability between test graph prototypes and their source domain prototypes is significantly higher than that with other domains, thereby enabling accurate and robust domain-specific parameter assignment.\n\n---\n\n> **Q3: The analysis for GraphKeeper to prevent forgetting in the Domain-IL setting (Weakness 3)**\n> \n\n**A3:** The stability of GraphKeeper in mitigating catastrophic forgetting stems from the synergistic operation of its core mechanisms. \n\n**Theorem 3. Stability Guarantee of GraphKeeper:** Let $\\mathcal{Z}$ denote the disentangled semantic space. For any two domains $\\mathcal{D}\\_u, \\mathcal{D}\\_v$ and classes $\\mathcal{C}\\_i^u,\\mathcal{C}\\_j^u \\in \\mathcal{D}\\_u$, the disentangled embeddings $\\mathbf{z} \\in \\mathcal{Z}$ satisfy:\n\n$$\n\\min\\_{\\substack{\\mathbf{z}\\_1 \\in \\mathcal{C}\\_i^u, \\mathbf{z}\\_2 \\in \\mathcal{C}\\_j^u}} \\|\\mathbf{z}\\_1 - \\mathbf{z}\\_2\\|_2 \\geq \\tau\\_{\\text{intra}}, \\quad \\min\\_{\\substack{\\mathbf{z}\\_1 \\in \\mathcal{C}\\_i^u, \\\\ \\mathbf{z}\\_2 \\in \\mathcal{C}\\_j^v}} \\|\\mathbf{z}\\_1 - \\mathbf{z}\\_2\\|_2 \\geq \\tau\\_{\\text{inter}}\n$$\n\nwhere intra- and inter-domain disentanglement objectives in Eq. (5) and Eq. (6) ensure that $\\tau\\_{\\text{intra}}$ and $\\tau\\_{\\text{inter}}$ are sufficiently large, which collocates nodes of distinct classes into unique semantic location, forming compact intra-class clusters with clear boundaries. The domain-specific PEFT then freezes these semantic regions, stabilizing historical domain embeddings.\n\nThe ridge regression classifier $\\psi$ establishes a stable mapping from the disentanglement semantic space $\\mathcal{Z}$ to the label space $Y$:\n\n$$\n\\psi\\_{t}(\\mathbf{z},\\boldsymbol{W}\\_{t}):\\mathcal{Z}\\rightarrow Y, \\quad \\boldsymbol{W}\\_{t} = \\arg\\min\\_{\\boldsymbol{W}}\\left\\|\\boldsymbol{Y}\\_{(1:t)}-\\mathcal{Z}\\_{(1:t)}\\boldsymbol{W}\\right\\|\\_{\\mathrm{F}}^2+\\lambda\\left\\|\\boldsymbol{W}\\right\\|_{\\mathrm{F}}^2,\n$$\n\nwhere the parameters $\\boldsymbol{W}$ of the ridge regression classifier $\\psi$ satisfy the optimal solution at any incremental stage, which is guaranteed by the derivation of the ridge classifier’s exact historical knowledge retention in Appendix A. The intra- and inter-domain disentanglement and historical embedding stability enable the ridge regression classifier to map disentangled and stable embeddings to labels via closed-form recursive updates.\n\nFor inference, given test data $\\boldsymbol{x}\\_{\\text{test}} \\sim \\mathcal{D}\\_k$, the training data  $\\boldsymbol{x}\\_{\\text{train}}$ of source domain is also identically distributed. Due to Theorem 1-2 guarantee domain-specific parameter assignment is accurate and robust, the prediction error:\n\n$$\n\\lim_{\\tau\\_{\\text{intra}},\\tau\\_{\\text{inter}} \\to +\\infty} \\mathbb{P}\\left( \\psi(\\mathbf{z}\\_{\\text{train}}) \\neq y\\_{\\text{train}} \\right) = 0, \\quad  \\lim_{\\epsilon \\to 1} \\mathbb{P}\\left( \\psi(\\mathbf{z}\\_{\\text{test}}) \\neq y\\_{\\text{test}} \\right) = \\mathbb{P}\\left( \\psi(\\mathbf{z}\\_{\\text{train}}) \\neq y\\_{\\text{train}} \\right)\n$$\n\nwhere the prediction error of training data converges to zero when $\\tau\\_{\\text{intra}}$ and $\\tau\\_{\\text{inter}}$ are sufficiently large. Meanwhile, the prediction error of the test data converges to the same level as that of the training data of source domain when the distribution matching degree $\\epsilon$ in the embedding space $\\mathcal{Z}$ approaches 1.\n\n---\n\n> **Q4: The analysis of memory and computational costs as the number of domains grows (Weakness 4)**\n> \n\n**A4:** We are grateful for the reviewer's concern to this detail.\n\nOur experiments in Figure 7 of the paper validate that GraphKeeper achieves competitive performance even with low-rank PEFT modules, indicating that domain-specific parameters account for only a minor fraction compared to the pre-trained backbone. To quantify this, we tracked the total parameters and trainable parameters across incremental stages.\n||Pretrained|1 domain|5 domains|10 domains|20 domains|\n|---| --- | --- | --- | --- | --- |\n|Total Parameters|196.61K|206.85K|247.81K|299.01K|401.41K|\n|Tuned Parameters||10.24K|10.24K|10.24K|10.24K|\n\nResults show that adding a new domain increases total parameters by merely 5.2% relative to the pre-trained backbone, with only the specific PEFT parameters of current domain requiring training. This confirms that parameter growth slightly as the number of domains grows, and only a small subset of parameters is ever trained, ensuring memory and computational costs stay manageable even for large incremental domain sets."}, {"review_id": "rM9CmmRGy3", "reviewer": "Reviewer_E6bP", "summary": "Most existing work in graph incremental learning focuses on task- and class-increment settings within the same domain, while graph domain-incremental learning (DomainIL) remains challenging and underexplored. This paper focuses on the DomainIL scenarios, and proposes a framework, GraphKeeper, to address the catastrophic forgetting issue in DomainIL. The DomainIL framework includes a domain-specific parameter-efficient fine-tuning (PEFT) module to avoid embedding drift, a deviation-free knowledge preservation model to stabilize the decision boundary across domains, and a domain-aware distribution discrimination module to produce precise embeddings for unseen domains.", "questions": "- Are the reported results based on inference conducted on an unseen domain?\n- In the integration studies with GFMs, why do you first pre-train the GFMs and how?\n- Is it always beneficial or fair to push embeddings from different domains apart? Could you provide more discussion and explanation? \n- How is the domain prototype selected? Is it computed as the average of all node embeddings within a domain?", "limitations": "N/A.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper is the first to explore the challenging problem of graph domain-incremental learning (DomainIL) and effectively motivates its importance.\n- The proposed methods are technically sound. To mitigate catastrophic forgetting in DomainIL, the authors introduce domain-specific PEFT modules to prevent embedding shifts, and use Intra- and inter-domain disentanglement to make embeddings more discriminative across domains. On the other hand, the framework decouples the embedding model from the task classifier to preserve a stable decision boundary, which the classifier is updating via ridge regression and without historical data access.\n- The proposed framework can be adapted to new, unseen domains, by the effect of a domain-aware distribution discrimination module.\n- The experimental results are promising. In particular, it is interesting to observe that the proposed method brings significant improvement to GFMs in the DomainIL setting.\n\nWeaknesses:\n- The paper does not discuss Mixture-of-Experts (MoE) approaches for graph incremental learning. It would be helpful to compare with such methods, especially in relation to the proposed domain-specific graph PEFT design.\n- The domains used during the incremental training process, their ordering, and the domain chosen for inference are not clearly specified.\n- The explanation of how GraphKeeper integrates with GFMs lacks clarity. Specifically, it is unclear how GFMs are pre-trained, and how few-shot learning is conducted.\n- It would be better to include the comparison between using GFMs with GraphKeeper and directly fine-tuning GFMs on new domains.\n- Minor: Some references to figures or sections (e.g., Figure G.2, G.3) are broken or lead to incorrect locations.", "was_revised": true, "final_justification": "The authors addressed my concerns in the rebuttal, and I have decided to keep my score at 4 (borderline accept) as it is already positive.", "rebuttal": "We appreciate your efforts and insightful comments. Below, we provide more detailed explanations to address your concerns.\n\n---\n\n> **Q1: More discussions on related work (Weakness 1)**\n> \n\n**A1:** We are grateful for the reviewer's constructive suggestion. We have further investigated recent Mixture-of-Experts and parameter-efficient fine-tuning approaches relevant to graph incremental learning and incorporated comparisons into our analysis. Below, we clarify distinctions between GraphKeeper and existing methods:\n﻿\n\n- DyMoE [1] memorizes knowledge of incremental classes through distinct experts, integrating the relevant knowledge via sparse MoE activation to enable positive transfer from previously learned knowledge. However, DyMoE operates exclusively within single domains, prioritizing task-level knowledge transfer, where its expert related integration fails under Domain-IL's core cross-domain structural and semantic gaps. GraphKeeper's domain-specific PEFT structurally resembles MoE but fundamentally differs, where we isolate domain parameters to prevent embedding shifts in incremental stages and achieve knowledge disentanglement between different domains, addressing Domain-IL's unique challenge of inter-domain interference.\n- FastKGE [2] leverages incremental LoRA (IncLoRA) to efficiently learn new knowledge graph embeddings while preserving old knowledge. However, its adapter allocation relies on entity-level influence within a single KG, overlooking the domain-level shifts critical to Domain-IL, which is a more challenging scenario requiring knowledge stability across heterogeneous domains. Although both adopt structure similar to incremental LoRA, GraphKeeper’s PEFT is fundamentally domain-centric, designed to stabilize and disentangle multi-domain knowledge. Meanwhile, its implementation is easily compatible (e.g., substituting LoRA with corresponding prompt strategy in GFMs), demonstrating flexibility and adaptability.\n- TPP [3] predicts task-ID via Laplacian smoothing, sharing conceptual similarity with our domain assignment. However, it faces significant risks in Domain-IL due to cross-domain semantic gaps ignored. In contrast, GraphKeeper’s high-dimensional distribution discrimination resolves prototype confusion, as shown in Figure 7 of the paper. What's more, TPP’s simplistic prompt design shows limited adaptability to multi-domain graphs, which evidenced by its average accuracy in Table 1 of the paper.\n\n---\n\n> **Q2: More detailed explanation about experimental setup (Weakness 2, Question 1)**\n> \n\n**A2:** The incremental training domains, their ordering, and the inference settings are comprehensively specified in Appendix C.2.1 of the paper. To further clarify these aspects:\n\n- We constructed 8 incremental groups (Table C.2 in Appendix) from 15 diverse graph domains. Group 1-3 consist of graph domains from the same type (e.g., all are social networks in a group), while Group 4-6 include graph domains from mixed types. Groups 7–8 include over 10 graph domains to evaluate performance under longer incremental learning sequences.\n- To eliminate order bias, each group of domains was evaluated under all cyclic permutations of the task sequence. Such as for group of domains {A, B, C, D}, we evaluate in incremental orders of A → B → C → D, B → C → D → A, C → D → A → B, and D → A → B → C, where each order is independently executed 10 times. This design ensures robustness against sequence-dependent performance variance, with results averaged over all permutations.\n- For the evaluation protocol, we strictly adhere to incremental learning principles. During the process of incremental learning, an accuracy matrix ${M}$ is generated, where ${M}\\_{i,j}$ represents the accuracy on $j$-th domain after learning $i$-th domain. Average Accuracy (AA) is computed as $\\frac{1}{T}\\sum\\_{j=1}^{T} M\\_{T,j}$, which represents the average performance across all domains after learning all domains. Average Forgetting (AF) is computed as $\\frac{1}{T-1}\\sum\\_{j=1}^{T-1}(M\\_{T,j} - M\\_{j,j})$, which represents the difference between the final performance on each domain and the performance on the domain when it was first learned.\n- To ensure absolute fairness in incremental learning, the model is unaware of which domain the data to be evaluated belongs to during evaluation (i.e., graphs with unobservable domains in the paper). It should be emphasized that, the test graph belongs to one of the domains in the training process, rather than entirely new domains.\n\n---\n\n> **Q3: More detailed explanation about the integration studies with GFMs (Weakness 3 & 4, Question 2)**\n> \n\n**A3:** We are grateful for the reviewer's concern to this detail.\n\n- GraphKeeper integrates with GFMs during the downstream fine-tuning phase, without altering their pretraining procedures, preserving their pretrained knowledge. GFMs are pretrained following the method proposed in corresponding paper, and pretraining datasets remain disjoint from downstream data, adhering to standard GFM settings.\n- Our experiments integrating GFMs focus on preserving their few-shot capability under low-resource constraints while endowing them with incremental learning and memory abilities, enabling continuous few-shot adaptation across multiple graph domains without forgetting. We replace the pre-trained GNN in our framework with pre-trained GFMs and adapt domain-specific PEFT modules. Concretely, for MDGPT, we replace the LoRA in PEFT module with its prompt-based strategy during fine-tuning. For GCOPE, we retain the original PEFT module and fine-tune the pre-trained GFM, consistent with its original design.\n- For the few-shot experimental setup, we sample 10 nodes per class per graph domain to effectively conduct incremental multi-domain downstream fine-tuning (as stated in Appendix C.2.1).\n- Table 2 in the paper already includes the performance of GFMs in continuous multi-domain downstream fine-tuning with/without the GraphKeeper framework. It shows that:\n    - Without GraphKeeper: The low AA and low AF of GFMs indicate that although GFMs achieve excellent performance in single-domain few-shot learning, they suffer from severe forgetting of prior domain knowledge during domain incremental few-shot learning.\n    - With GraphKeeper: GFMs perfectly integrate continual learning and memory abilities, while retaining few-shot capabilities, exhibiting high AA and near-zero AF.\n\n---\n\n> **Q4: Broken reference locations (Weakness 5)**\n> \n\n**A4:** Thanks for the careful reminder. We will correct it in the revised version.\n\n---\n\n> **Q5: More detailed explanation about disentanglement mechanism (Question 3)**\n> \n\n**A5:** We appreciate the reviewer's insightful comment. Pushing embeddings from different domains apart is synergistic with other components of GraphKeeper. First, intra-domain disentanglement in Eq. (5) and inter-domain disentanglement in Eq. (6) map nodes of distinct classes to unique positions in the semantic space, eliminating semantic confusion. Second, after learning each incremental domain, its domain-specific parameters are frozen, locking the embeddings within stable regions of the semantic space. This stability aligns perfectly with the deviation-free ridge regression classifier in Eq. (10) and Eq. (11), which maps different semantic regions to labels via closed-form updates, leveraging the disentangled and stable embeddings. Consequently, the synergy among these mechanisms ensures both zero forgetting of old domains and effective memory of new domain knowledge. Further explanation can be obtained in Theorem 3 of the response to reviewer #HUJf.\n\n---\n\n> **Q6: More detailed explanation about the domain prototype (Question 4)**\n> \n\n**A6:** We are grateful for the reviewer's concern to this detail.\n\n- As described in Eq. (13) and Eq. (14), the domain prototype is computed as the average of node features within a high-dimensional space. Specifically, to mitigate assignment errors caused by potential overlaps among multiple domains, we first project node features into a more discriminative high-dimensional space via random mapping in Eq. (13), a concept similarly explored in literature [4-6]. The prototype for each domain is then obtained by averaging these high-dimensional features of all its training nodes. During evaluation, for a test graph with an unobservable domain, we similarly average the high-dimensional features of its test nodes to form a unified prototype, which is then matched against prototypes of all previously trained domains for precise domain-specific parameter assignment. The distance between prototypes is then used to estimate the probability of the test graph belonging to each domain. The more explanation can be obtained in Theorem 1-2 of the response to reviewer #kEra. \n\n- Furthermore, although using node embeddings for domain prototype calculation is also feasible due to the stability of node embeddings in our method, this requires performing inference on the test graph with all domain-specific parameters to compute multiple matching scores for assignment during testing, which incurs unnecessary computational overhead. Therefore, we adopt node features rather than the embeddings output by the model.\n\n---\n\n> **References**\n\n[1] Kong L, et al. Dynamic Mixture-of-Experts for Incremental Graph Learning.\n\n[2] Liu J, et al. Fast and continual knowledge graph embedding via incremental lora. IJCAI 2024.\n\n[3] Niu C, et al. Replay-and-forget-free graph class-incremental learning: A task profiling and prompting approach. NeurIPS 2024.\n\n[4] McDonnell M D, et al. Ranpac: Random projections and pre-trained models for continual learning. NeurIPS 2023.\n\n[5] Sun J, et al. Feature expansion for graph neural networks. ICML 2023.\n\n[6] Zhuang H, et al. Acil: Analytic class-incremental learning with absolute memorization and privacy protection. NeurIPS 2022."}, {"review_id": "tUl4QQUKd6", "reviewer": "Reviewer_4hfB", "summary": "The paper introduces GraphKeeper, a framework for Graph Domain-Incremental Learning that prevents catastrophic forgetting by stabilizing embeddings and decision boundaries across graph domains. It combines domain-specific fine-tuning, disentanglement objectives, and knowledge preservation. Experiments show it outperforms existing methods, offering improved accuracy and minimal forgetting for multi-domain graph learning.", "questions": "1. How does GraphKeeper perform when dealing with graph data from domains that have very little overlap in their feature semantics? Are there any specific domains where the framework struggles?\n\n2. In the experiments, you compare GraphKeeper with methods like TPP and DeLoMe, which also address catastrophic forgetting. Could you discuss in more detail how GraphKeeper is fundamentally different in terms of model architecture or training procedure?\n\n3. How does the method scale to a large number of domains, and can the PEFT modules be shared or compressed across similar domains?\n\n4. What's the limitation of existing related works? Please declare the majoir difference between GraphKeeper and these.\n\n5. For other questions, please refer to Weakness.", "limitations": "NA", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n1. GraphKeeper fills a gap in graph incremental learning by addressing the challenge of catastrophic forgetting in domain-incremental learning, which is an area that has been less explored.\n2. The combination of domain-specific PEFT, disentanglement objectives (intra- and inter-domain), and deviation-free knowledge preservation offers a comprehensive framewrk to mitigate forgetting and improve model stability across domains.\n3. The framework's ability to integrate with existing GFMs enhances its potential for use in real-world applications involving dynamic, multi-domain graph data.\n\n**Weaknesses:**\n1. The paper doesn't provide an ablation study for the domain-aware distribution discrimination mechanism (Section 4.3), which could offer deeper insights into its effectiveness.\n2. The paper lacks comparison with the CaT [1], which would be a relevant baseline in the context of graph continual learning.\n3. The proposed inter-domain loss pushes samples away from previous domain prototypes using inverse distance. This is a heuristic formulation, lacking theoretical grounding on why this regularization ensures optimal knowledge retention.\n4. Missing related references, e.g., [2]\n\n[1] Cat: Balanced continual graph learning with graph condensation. ICDM 2023.\n\n[2] Dink-net: Neural clustering on large graphs", "was_revised": true, "final_justification": "Most of the concerns had been addressed", "rebuttal": "We appreciate your positive feedback and insightful comments. We provide responses below to address your concerns.\n\n---\n\n> **Q1: The ablation study for Domain-aware Distribution Discrimination and comparison with CaT (Weakness 1 & 2)**\n> \n\n**A1:** We sincerely thank the reviewer for this valuable suggestion. To validate the effectiveness of domain-aware distribution discrimination, we remove it to obtain GraphKeeper(w/o Discrim) variant. And we further incorporate CaT into the comparison. The supplementary results will be incorporated into the revised version.\n\n| Method | Group 1 |  | Group 2 |  | Group 3 |  | Group 4 |  | Group 5 |  | Group 6 |  |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | **AA ↑** | **AF ↑** | **AA ↑** | **AF ↑** | **AA ↑** | **AF ↑** | **AA ↑** | **AF ↑** | **AA ↑** | **AF ↑** | **AA ↑** | **AF ↑** |\n| CaT | 26.7±3.3 | -39.1±4.8 | 43.3±1.1 | -7.3±3.3 | 40.6±1.6 | -9.8±3.5 | 45.6±1.9 | -31.8±3.7 | 39.1±3.1 | -11.5±4.1 | 47.9±4.9 | -21.7±3.3 |\n| **GraphKeeper** | **69.2±0.3** | -0.4±0.2 | **73.1±1.1** | -2.8±0.7 | **80.6±0.4** | 0.0±0.1 | **79.9±0.5** | 1.0±0.5 | **75.5±0.9** | 0.1±0.5 | **77.5±0.7** | -0.1±0.5 |\n| w/o Discrim | 39.8±1.9 | -31.4±2.3 | 37.1±1.5 | -30.7±2.3 | 58.5±4.8 | -27.5±7.2 | 30.4±4.3 | -38.0±1.3 | 35.6±7.1 | -24.8±2.7 | 38.7±1.6 | -28.6±3.4 |\n\nAs shown in the Table:\n\n- The performance of GraphKeeper(w/o Discrim) degrades significantly due to the lack of prior knowledge for associating the graph with corresponding PEFT module, resulting in less precise embeddings.\n- GraphKeeper still achieves the best performance with incorporating CaT into the comparison, as CaT did not consider Domain-IL's core cross-domain challenges. \n\n---\n\n> **Q2: More detailed explanation about disentanglement mechanism (Weakness 3)**\n> \n\n**A2:**  We appreciate the reviewer's insightful comment. The inter-domain disentanglement is not a standalone heuristic but a synergistic component of GraphKeeper’s unified framework. Intra- and inter-domain disentanglement collocates nodes of distinct classes into unique semantic location, forming compact intra-class clusters with clear boundaries. The domain-specific PEFT then freezes these semantic regions, stabilizing historical domain embeddings. The intra- and inter-domain disentanglement and historical embedding stability enable the ridge regression classifier to map disentangled and stable embeddings to labels via closed-form recursive updates, facilitating incremental updates while ensuring optimal knowledge preservation. Thus, distancing new domains from old prototypes prevents overlap in the stabilized embedding space, ensuring optimal knowledge retention by isolating domain semantics while facilitating incremental updates. Further explanation can be obtained in Theorem 3 of the response to reviewer #HUJf.\n\n---\n\n> **Q3: Missing related references (Weakness 4)**\n> \n\n**A3:** Thank you for reminding this omission. We will add the suggested reference in the revised version.\n\n---\n\n> **Q4: Performance in domains with little overlap and limitations in specific domains (Question 1)**\n> \n\n**A4:** We appreciate the reviewer's insightful comment.\n\n- For graph data from domains that have very little overlap in their feature semantics, please refer to Group 1 of Figure G.2 in Appendix G. Although very little overlap in raw features, other methods suffer from domain confusion due to embedding shifts caused by parameter updates across stages. In contrast, our method consistently maintains clear boundaries between domains in each stage.\n- Regarding limitations, GraphKeeper is agnostic to domain semantics and performs robustly across divergent domains. However, when fine-tuning pre-trained homogeneous GNNs on heterogeneous graph domains, structural disparities, which unrelated to feature semantics, causing a marginal performance drop. This is a universal challenge in heterogeneous and homogeneous graph learning, independent of our framework and beyond the scope of our discussion.\n\n---\n\n> **Q5: More discussion of related work limitations (Question 2 & 4)**\n> \n\n**A5:** Thank you for highlighting this point. \n\n- GraphKeeper fundamentally differs from prior methods through its distinctive design for Domain-IL scenarios. Existing approaches (e.g., DeLoMe, PDGNNs) are confined to Task-IL or Class-IL settings, relying on end-to-end updates of a single GNN model. However, in Domain-IL, significant structural and semantic gaps across graph domains prevent a unified GNN from retaining multi-domain knowledge, rendering conventional designs ineffective. We attribute their failure to embedding shifts (distortion of prior domain embeddings) and decision boundary deviations (classifier instability). \n\n- GraphKeeper addresses this via: (1) Domain-specific PEFT atop a pre-trained GNN, isolating parameters to protect prior domains from interference; (2) The decision modules updated through ridge regression’s closed-form solution, rather than end-to-end backpropagation to update the entire model, preventing boundary drift while preserving knowledge; (3) Domain-aware distribution discrimination in high-dimensional space to assign PEFT parameters for inference on graphs with unobservable domain. Although TPP introduced a similar concept, its design remains confined to traditional scenarios, where the simple task-ID prediction mechanism and prompt design substantially diminish its effectiveness in the Domain-IL setting, which is validated by the performance comparisons in Table 1 of the paper.\n\n---\n\n> **Q6: Discussion on the scalability of GraphKeeper (Question 3)**\n> \n\n**A6:** We appreciate the reviewer's insightful comment.\n\n**(1) Regarding the scalability to a large number of domains:**\n- The Table E.4 in Appendix demonstrate that GraphKeeper maintains superior performance even with 10+ incremental graph domains. \n- In terms of memory and computational costs, our experiments in Figure 7 of the paper validate that GraphKeeper achieves competitive performance even with low-rank PEFT modules, indicating that domain-specific parameters account for only a minor fraction compared to the pre-trained backbone. To quantify this, we tracked the total parameters and trainable parameters across incremental stages. Results show that adding a new domain increases total parameters by merely 5.2% relative to the pre-trained backbone, with only the specific PEFT parameters of current domain requiring training. This confirms that parameter growth slightly as the number of domains grows, and only a small subset of parameters is ever trained, ensuring memory and computational costs stay manageable even for large incremental domain sets.\n\n|  | Pretrained | 1 domain | 5 domains | 10 domains | 20 domains |\n| --- | --- | --- | --- | --- | --- |\n| Total Parameters | 196.61K | 206.85K | 247.81K | 299.01K | 401.41K |\n| Tuned Parameters |  | 10.24K | 10.24K | 10.24K | 10.24K |\n\n**(2) Regarding the sharing of PEFT modules:** this is an insightful comment that highlights the potential for leveraging similar knowledge from prior domains to facilitate forward transfer in incremental learning. As the first work to explore Graph Domain-IL, GraphKeeper prioritizes addressing catastrophic forgetting in this scenario, which is the most fundamental and critical capability for incremental learning. The cross-domain parameter sharing mechanism introduces new challenges, such as extracting domain-invariant knowledge and mitigating negative transfer. Therefore, while promising, this direction constitutes future work beyond the scope of this work."}, {"review_id": "C6ZLIgMY4n", "reviewer": "Reviewer_kEra", "summary": "This paper pioneers graph domain-incremental learning (Domain-IL), where models continuously learn from diverse graph domains unlike traditional task/class-incremental scenarios. The authors trace catastrophic forgetting to embedding shifts and decision boundary deviations when acquiring new domain knowledge. GraphKeeper addresses these issues through three components: domain-specific PEFT with intra/inter-domain disentanglement prevents embedding confusion; ridge regression-based knowledge preservation maintains stable decision boundaries without historical data; high-dimensional random mapping matches unknown test domains to correct modules.", "questions": "See above weaknesses", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "# **Strengths**\n\n**S1: Domain-Specific Parameter Isolation**\n\nDomain-specific graph PEFT using LoRA modules where each incremental domain gets its own low-rank adaptation parameters while freezing previous domains' parameters, preventing cross-domain parameter interference.\n\n**S2: Dual-Level Disentanglement Mechanism**\n\nNovel intra-domain and inter-domain disentanglement objectives that simultaneously enhance class discriminability within each domain through contrastive learning and ensure domain separation by pushing current domain embeddings away from previous domain prototypes.\n\n**S3: Gradient-Free Decision Boundary Preservation**\n\nThe framework separates classifier from embedding model and employs ridge regression with recursive matrix updates (Equations 10-11) to maintain stable decision boundaries without gradient-based updates, eliminating decision boundary drift.\n\n# **Weaknesses**\n\n**W1: Missing Theoretical Convergence Analysis**\n\nThe paper lacks theoretical analysis for recursive ridge regression (Equations 10-11). Authors should prove convergence conditions, demonstrate equivalence with batch ridge regression, and provide theoretical justification for how LoRA parameter isolation ensures previous domain embedding stability.\n\n**W2: Insufficient Analysis of Domain Discrimination Failures**\n\nSection 4.3's domain-aware distribution discrimination mechanism lacks examination of failure scenarios. Authors should conduct additional experiments addressing: (1) strategies for handling test graphs whose domain features differ significantly from all trained domains; (2) conditions where high-dimensional random mapping fails to separate domain prototypes effectively; (3) quantifying how domain discrimination errors impact overall performance and whether confidence thresholds or rejection mechanisms are necessary for uncertain domain assignments.", "was_revised": true, "final_justification": "My final justification is recommendation **4** (Borderline Accept) and my confidence is 3\nThe paper's core technical strengths and rigorous methodology compensate for the relatively limited scope of its evaluation. Although the reviewers raised concerns about the theoretical justification and experimental procedures, the authors' rebuttals and subsequent discussions with the reviewers clarified that the narrow but conclusive evaluation effectively validated the core arguments. While some uncertainty exists regarding the understanding of some technical details, the fundamental soundness of the study is evident. Therefore, the technical contributions sufficiently outweigh the acknowledged limitations to warrant only a grudging recommendation. The authors are advised to cautiously expand the theoretical comparison in future work.", "rebuttal": "We are grateful for your positive feedback and detailed comments. We provide responses below to address your concerns.\n\n---\n\n> **Q1: Supplementary analysis of recursive ridge regression and embedding stability (Weekness 1)**\n> \n\n**A1:** We appreciate the reviewer's insightful comment.\n\n**(1) Analysis of recursive ridge regression:** The detailed derivation proving the equivalence between incremental updated ridge regression (Eq. (10) and Eq. (11)) and global ridge regression (Eq. (9)) has been provided in Appendix A. To illustrate, we list the main derivation steps here:\n\n*Proof.*  We begin by defining the closed-form solution of ridge regression at step $k$ as:\n$$\n\\boldsymbol{W}\\_{k} =  \\boldsymbol{M}\\_{k} \\boldsymbol{X}\\_{(1:k)}^{\\top} \\boldsymbol{Y}\\_{(1:k)}.\n$$\nwhere $\\boldsymbol{M}\\_{k}=\\big(\\boldsymbol{X}\\_{(1:k)}^{\\top} \\boldsymbol{X}\\_{(1:k)}+\\lambda \\boldsymbol{I}\\big)^{-1}$. This can be recursively expressed using the matrix inversion lemma as:\n$$\n\\boldsymbol{M}\\_{k}=\\big(\\boldsymbol{M}\\_{k-1}^{-1} + \\boldsymbol{X}\\_{k}^{\\top} \\boldsymbol{X}\\_{k}\\big)^{-1}.\n$$\nApplying the Woodbury matrix identity, we derive the recursive update of $\\boldsymbol{M}\\_k$:\n$$\n\\boldsymbol{M}\\_k = \\boldsymbol{M}\\_{k-1}  \n        - \\boldsymbol{M}\\_{k-1} \\boldsymbol{X}\\_k^{\\top} \\big(\\boldsymbol{I} + \\boldsymbol{X}\\_k \\boldsymbol{M}\\_{k-1} \\boldsymbol{X}\\_k^{\\top} \\big)^{-1} \\boldsymbol{X}\\_k \\boldsymbol{M}\\_{k-1}.\n$$\nWe then turn to the update of $\\boldsymbol{W}\\_k$. By definition:\n$$\n\\boldsymbol{W}\\_{k} = \\big[\\boldsymbol{M}\\_k \\boldsymbol{X}\\_{(1:k-1)}^{\\top} \\boldsymbol{Y}\\_{(1:k-1)}  ||  \\boldsymbol{M}\\_k \\boldsymbol{X}\\_{k}^{\\top} \\boldsymbol{Y}\\_{k}\\big],\n$$\nwhere the first term can be simplified through a series of derivations:\n$$\n\\boldsymbol{M}\\_k \\boldsymbol{X}\\_{(1:k-1)}^{\\top} \\boldsymbol{Y}\\_{(1:k-1)} = \\boldsymbol{W}\\_{k-1} - \\boldsymbol{M}\\_k \\boldsymbol{X}\\_k^{\\top} \\boldsymbol{X}\\_k \\boldsymbol{W}\\_{k-1}.\n$$\nSubstituting it back into $\\boldsymbol{W}\\_{k}$, we obtain:\n$$\n\\boldsymbol{W}\\_{k} = \\big[ \\boldsymbol{W}\\_{k-1} - \\boldsymbol{M}\\_k \\boldsymbol{X}\\_k^{\\top} \\boldsymbol{X}\\_k \\boldsymbol{W}\\_{k-1} || \\boldsymbol{M}\\_k \\boldsymbol{X}\\_{k}^{\\top} \\boldsymbol{Y}\\_{k} \\big].\n$$\n\nThis proves the equivalence of Eq. (10) and Eq. (11) with global ridge regression in Eq. (9). The complete derivation process can be found in Appendix A.\n\n**(2) Analysis of embedding stability:** Each incremental domain uses domain-specific PEFT modules fine-tuned on a pre-trained GNN backbone. After training, these parameters are frozen and stored. During inference, domain-aware distribution discrimination selects the correct frozen PEFT parameters to combine with the backbone. Since both the pre-trained backbone and domain-specific PEFT parameters remain fixed after training, embeddings for previous domains are stable as long as the domain-aware distribution discriminator assigns the correct PEFT parameters, ensuring stability under the identically distributed data in the same domain. Further explanation can be obtained in Theorem 3 of the response to reviewer #HUJf.\n\n---\n\n> **Q2: Supplementary analysis of Domain Discrimination (Weekness 2)**\n> \n\n**A2:** We thank the reviewer for raising this insightful concern. Below, we provide a systematic response through theoretical theorems and supplementary experiments. The supplementary analyses will be incorporated into the revised version.\n\n**Theorem 1. Linear Separability via Random High-Dimensional Mapping:** Let $D_i$ and $D_j$ be feature distributions of two graph domains with overlap $\\kappa \\in [0,1]$. For a fixed random mapping $\\phi:\\mathbb{R}^{d} \\rightarrow \\mathbb{R}^{m}$, where $m \\gg d$, the mapped features satisfy:\n\n$$\n\\mathbb{P}\\left[\\left\\| \\phi(\\boldsymbol{x}_i \\sim \\mathcal{D}_i) - \\phi(\\boldsymbol{x}_j \\sim \\mathcal{D}_j) \\right\\|_2 \\geq \\Delta \\right] \\geq 1 - \\delta\n$$\n\nwhere $\\Delta >0$ is a separation margin, and $\\delta$ decays exponentially with $m$. Even when $D_i$ and $D_j$ overlap to a large extent, random projection amplifies distributional differences in high-dimensional space, ensuring linear separability with high probability if $\\kappa \\neq 1$. This stems from the Johnson-Lindenstrauss Lemma and geometric properties of random projections.\n\n**Theorem 2. Stability of Prototype Matching:** Let $\\boldsymbol{x}\\_{\\text{test}} \\sim \\mathcal{D}\\_{k}$ be the test node features from domain $k$, and $\\boldsymbol{p}\\_{k} = \\mathbb{E}\\_{\\boldsymbol{x}\\_{\\text{train}} \\sim \\mathcal{D}\\_{k}}[\\phi(\\boldsymbol{x}\\_{\\text{train}})]$ be the true domain prototype. Given a fixed random mapping $\\phi$ and identical train and test distributions:\n\n$$\n\\left\\| \\boldsymbol{p}_{\\text{test}} - \\boldsymbol{p}_k \\right\\|_2 \\leq \\epsilon_1\n$$\n\nFor mismatched domain prototypes $\\boldsymbol{p}_{j}$, where $j \\neq k$:\n\n$$\n\\left\\| \\boldsymbol{p}_{\\text{test}} - \\boldsymbol{p}_j \\right\\|_2 \\geq \\Delta - \\epsilon_2\n$$\n\nwhere $\\Delta$ is the separation margin from Theorem 1. Since the test graph's nodes share the same distribution as the training nodes of its source domain, test graphs converge to their true domain prototype $\\boldsymbol{p}\\_k$ (i.e. $\\epsilon_1$ tends towards zero). Meanwhile, even when the feature distributions across domains exhibit substantial overlap but are not identical (i.e. $\\kappa \\neq 1$), coupling this with a sufficiently high mapping dimension ensures that $\\Delta - \\epsilon_2$ tends to be sufficiently large. This guarantees that the lower bound of $\\left\\| \\boldsymbol{p}\\_{\\text{test}} - \\boldsymbol{p}\\_j \\right\\|\\_2$ is always greater than the upper bound of $\\left\\| \\boldsymbol{p}\\_{\\text{test}} - \\boldsymbol{p}\\_k \\right\\|\\_2$.\n\n**Supplementary experiments.** We conducted robustness evaluations under different domains with high distributional overlap. Specifically, we randomly sampled graphs from four mutually similar distributions as four graph domains, where the pairwise domains feature distribution overlap exceeded 80%. For each domain, we partitioned training and test nodes to derive domain prototypes and test graph prototypes. The confusion matrix below shows the probability of our method correctly predicting the source domain of each test graph.\n\n||$\\boldsymbol{p}_{1}$|$\\boldsymbol{p}_{2}$|$\\boldsymbol{p}_{3}$|$\\boldsymbol{p}_{4}$|\n|---|---|---|---|---|\n|$\\boldsymbol{p}_{1}^{\\text{test}}$|**38.16%**|22.77%|18.91%|20.14%|\n|$\\boldsymbol{p}_{2}^{\\text{test}}$|21.02%|**38.94%**|21.30%|18.72%|\n|$\\boldsymbol{p}_{3}^{\\text{test}}$|19.57%|23.72%|**39.79%**|16.91%|\n|$\\boldsymbol{p}_{4}^{\\text{test}}$|23.49%|20.47%|16.49%|**39.52%**|\n\nThese results demonstrate that even when confronted with highly similar cross-domain distributions, our method ensures that the matching probability between test graph prototypes and their source domain prototypes is significantly higher than that with other domains, thereby enabling accurate and robust domain-specific parameter assignment.\n\n---"}, {"review_id": "g42RuaMZp0", "reviewer": "Reviewer_CzLz", "summary": "This paper designs GraphKeeper for graph domain incremental learning. It tackles the challenge of catastrophic forgetting by mitigating embedding shifts and decision boundary deviations. GraphKeeper employs parameter-efficient fine-tuning and intra/inter-domain objectives, along with deviation-free knowledge preservation via ridge regression. Furthermore, it incorporates domain-aware distribution discrimination using high-dimensional random mapping. Experiments across 15 real-world datasets demonstrate its performance.", "questions": "Please see the above weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The analysis of the graph domain incremental learning task is praiseworthy, with the challenges well defined. \n2. The method's components appear effective, and the experiments are fairly comprehensive. The testing on the GFM model and the ablation experiments are conducted.\n\nWeaknesses:\n1. The author proposed a series of components to address the graph domain incremental learning task. But the components are not specially designed for graph data. LoRA-based learning, contrastive learning, and ridge regression can also be adapted to other data types. I worry about the innovativeness of this method for graph domain incremental learning tasks. \n2. In formula 7, why are there two loss weights instead of one? \n3. In formula 2, the prediction is made using a weight matrix and input. But why is there only a weight matrix and no bias matrix? \n4. In formula 13, how can random weight matrices compute stable and accurate node representations to get domain prototypes? \n5. In Figure 1, what's the accuracy of the proposed method in predicting domain prototypes? If using domain-specific parameters, whether the forgetting rate is zero like the TPP [1] method?\n\n[1] Replay-and-forget-free graph class-incremental learning: A task profiling and prompting approach. NeurIPS 2024.", "was_revised": true, "final_justification": "The analysis of the graph domain incremental learning task is praiseworthy. The author's response resolved my concerns about the method's innovation. So I raised the score from 3 to 4 accordingly.", "rebuttal": "We appreciate your comments. We provide responses below to address your concerns.\n\n---\n\n> **Q1: More Discussion on Method Design and Novelty (Weakness 1)**\n> \n\n**A1:** We appreciate the reviewer’s concern. Firstly, our method is designed around the significant structural and semantic differences across graph domains, as well as the limitations of vanilla GNN architectures in capturing multi-domain knowledge, which are the core challenges for graph data and models in Domain-IL scenario. A more detailed analysis of topology will be explored in future work. Additionally, the main contribution of this paper focuses on addressing the problems in the challenging scenario we proposed. While some techniques employed are not originally proposed in this paper, we have specifically adapted and refined them based on the characteristics of the scenario, making them suitable for graph data and models, and effectively addressing the challenges in Domain-IL scenario.\n\n---\n\n> **Q2: The explanation of Formulas 7 and Formulas 12 (Weakness 2 & 3)**\n> \n\n**A2:** We are grateful for the reviewer's concern to this detail.\n\n- **Regarding Formula 7:** it is mathematically equivalent to the single-weight form $\\mathcal{L} = \\gamma \\mathcal{L}\\_{\\text{intra}} + (1-\\gamma) \\mathcal{L}\\_{\\text{inter}}$, as both serve to balance the contributions of the intra-domain and inter-domain disentanglement objectives.\n- **Regarding Formula 12:** the bias term is omitted for mathematical and operational consistency. The weight matrix $W$ is derived via ridge regression in Eq. (9) is recursively updated in Eq. (10) and Eq. (11) to optimally fit all incremental graph domains without requiring historical data. Crucially, adding a bias term would disrupt the closed-form solution of ridge regression and the recursive update derivation in Appendix A, which is one of the key points to ensure that the weight matrix $W$ can recursively fit the new domain without forgetting the previous domains.\n\n---\n\n> **Q3: The explanation of Formulas 13 and Domain-aware Distribution Discrimination (Weakness 4)**\n> \n\n**A3:** We appreciate the reviewer's insightful comment. The random weight matrices in Eq. (13) compute stable domain prototypes through high-dimensional random projection, rather than pursuing precise node representations. The frozen, randomly initialized $GNN_{proj}$ non-linearly projects raw features into a high-dimensional space. This scattering effect amplifies differences between domains while suppressing potiential semantic overlaps, as shown in Figure 4 of the paper. Crucially, the fixed random weights ensure deterministic mappings across all domains, guaranteeing stability. Prototypes derived from averaged embeddings in Eq. (14) thus reliably discriminate domains based on relative distances.\n\nIn addition, we also provide theoretical theorems and supplementary experiments below.\n\n**Theorem 1. Linear Separability via Random High-Dimensional Mapping:** Let $D_i$ and $D_j$ be feature distributions of two graph domains with overlap $\\kappa \\in [0,1]$. For a fixed random mapping $\\phi:\\mathbb{R}^{d} \\rightarrow \\mathbb{R}^{m}$, where $m \\gg d$, the mapped features satisfy:\n\n$$\n\\mathbb{P}\\left[\\left\\| \\phi(\\boldsymbol{x}_i \\sim \\mathcal{D}_i) - \\phi(\\boldsymbol{x}_j \\sim \\mathcal{D}_j) \\right\\|_2 \\geq \\Delta \\right] \\geq 1 - \\delta\n$$\n\nwhere $\\Delta >0$ is a separation margin, and $\\delta$ decays exponentially with $m$. Even when $D_i$ and $D_j$ overlap to a large extent, random projection amplifies distributional differences in high-dimensional space, ensuring linear separability with high probability if $\\kappa \\neq 1$. This stems from the Johnson-Lindenstrauss Lemma and geometric properties of random projections.\n\n**Theorem 2. Stability of Prototype Matching:** Let $\\boldsymbol{x}\\_{\\text{test}} \\sim \\mathcal{D}\\_{k}$ be the test node features from domain $k$, and $\\boldsymbol{p}\\_{k} = \\mathbb{E}\\_{\\boldsymbol{x}\\_{\\text{train}} \\sim \\mathcal{D}\\_{k}}[\\phi(\\boldsymbol{x}\\_{\\text{train}})]$ be the true domain prototype. Given a fixed random mapping $\\phi$ and identical train and test distributions:\n\n$$\n\\left\\| \\boldsymbol{p}_{\\text{test}} - \\boldsymbol{p}_k \\right\\|_2 \\leq \\epsilon_1\n$$\n\nFor mismatched domain prototypes $\\boldsymbol{p}_{j}$, where $j \\neq k$:\n\n$$\n\\left\\| \\boldsymbol{p}_{\\text{test}} - \\boldsymbol{p}_j \\right\\|_2 \\geq \\Delta - \\epsilon_2\n$$\n\nwhere $\\Delta$ is the separation margin from Theorem 1. Since the test graph's nodes share the same distribution as the training nodes of its source domain, test graphs converge to their true domain prototype $\\boldsymbol{p}\\_k$ (i.e. $\\epsilon_1$ tends towards zero). Meanwhile, even when the feature distributions across domains exhibit substantial overlap but are not identical (i.e. $\\kappa \\neq 1$), coupling this with a sufficiently high mapping dimension ensures that $\\Delta - \\epsilon_2$ tends to be sufficiently large. This guarantees that the lower bound of $\\left\\| \\boldsymbol{p}\\_{\\text{test}} - \\boldsymbol{p}\\_j \\right\\|\\_2$ is always greater than the upper bound of $\\left\\| \\boldsymbol{p}\\_{\\text{test}} - \\boldsymbol{p}\\_k \\right\\|\\_2$.\n\n---\n\n> **Q4: The accuracy in predicting domain prototypes and the performance of forgetting rate (Weakness 5)**\n> \n\n**A4:** We appreciate the reviewer's insightful comment.\n\n**(1) Regarding the accuracy of domain prototype prediction:**\n- As illustrated in Figure 4 of the paper, our high-dimensional random mapping significantly enhances the discriminability between domain prototypes. This ensures precise matching between any test graph’s prototype and its belonging domain prototype, effectively eliminating inter-domain semantic confusion and guaranteeing reliable domain discrimination during testing. \n\n- To further verify this, we also provide experimental validation of robustness under extreme conditions (i.e. under different domains with high distributional overlap). Specifically, we randomly sampled graphs from four mutually similar distributions as four graph domains, where the pairwise domains feature distribution overlap exceeded 80%. For each domain, we partitioned training and test nodes to derive domain prototypes and test graph prototypes. The confusion matrix below shows the probability of our method correctly predicting the source domain of each test graph.\n\n|  | $\\boldsymbol{p}_{1}$ | $\\boldsymbol{p}_{2}$ | $\\boldsymbol{p}_{3}$ | $\\boldsymbol{p}_{4}$ |\n| --- | --- | --- | --- | --- |\n| $\\boldsymbol{p}_{1}^{\\text{test}}$ | **38.16%** | 22.77% | 18.91% | 20.14% |\n| $\\boldsymbol{p}_{2}^{\\text{test}}$ | 21.02% | **38.94%** | 21.30% | 18.72% |\n| $\\boldsymbol{p}_{3}^{\\text{test}}$ | 19.57% | 23.72% | **39.79%** | 16.91% |\n| $\\boldsymbol{p}_{4}^{\\text{test}}$ | 23.49% | 20.47% | 16.49% | **39.52%** |\n\n- These results demonstrate that even when confronted with highly similar cross-domain distributions, our method ensures that the matching probability between test graph prototypes and their source domain prototypes is significantly higher than that with other domains, thereby enabling accurate and robust domain-specific parameter assignment.\n\n**(2) Regarding the forgetting rate:** GraphKeeper exhibits near-zero forgetting and even demonstrates positive forward transfer on some graph domains. This fundamentally differs from TPP, while TPP achieves zero forgetting by completely isolating prompt parameters and classifiers across incremental tasks under accuracy task-ID prediction, it severely restricts cross-task knowledge transfer. In contrast, GraphKeeper maps embeddings from different domains into a unified semantic space to preserve domain-specific semantics. When new domains are introduced, the boundary adaptively adjusts when recursively updating the ridge regression classifier, triggering implicit positive transfer. It is worth noting that the current framework demonstrates promise for positive transfer among incremental domains, but not yet consummates. We are actively exploring to achieve that and addressing challenges such as domain-invariant knowledge extraction and negative transfer mitigation, which constitutes future work beyond the scope of this work."}]

Author rebuttal:

Meta-review: This paper introduces GraphKeeper, a framework for Graph Domain-Incremental Learning (Domain-IL), where models must continuously learn from graphs across different domains without forgetting prior knowledge. The authors identify embedding shifts and decision boundary drift as the main causes of catastrophic forgetting and address them through domain-specific parameter-efficient fine-tuning, intra- and inter-domain disentanglement, and ridge regression–based knowledge preservation. Experiments on 15 datasets demonstrate that GraphKeeper consistently outperforms existing incremental learning baselines, offering stable embeddings, robust decision boundaries, and improved accuracy in multi-domain graph learning.

Strengths:
1. Most reviewers find the paper well-motivated and recognize its novelty in addressing graph domain-incremental learning.
2. Most reviewers consider the experiments extensive and the results promising.

Weakness:
1. Some reviewers raised concerns about the transferability of the proposed method across similar domains
2. Some reviewers ask for theoretical justification.

During the rebuttal, the authors have provided:
1. New theoretical justification
2. Additional experiments to justify model's scalability, knowledge transferability and robustness under domain similarity.

The authors addressed all concerns during the rebuttal, and all reviewers responded with positive ratings, indicating unanimous support for acceptance.
Decision: Accept (poster)