GCAL: Adapting Graph Models to Evolving Domain Shifts
Authors: Ziyue Qiao
Qianyi Cai
Hao Dong
Jiawei Gu
Pengyang Wang
Meng Xiao
Xiao Luo
Hui Xiong
Venue: ICML 2025 poster (2025)
Keywords: Continual Learning
Domain adaptation
Graph neural network
Primary area: deep_learning->graph_neural_networks

Abstract: This paper addresses the challenge of graph domain adaptation on evolving, multiple out-of-distribution (OOD) graphs.
Conventional graph domain adaptation methods are confined to single-step adaptation, making them ineffective in handling continuous domain shifts and prone to catastrophic forgetting. This paper introduces the Graph Continual Adaptive Learning (GCAL) method, designed to enhance model sustainability and adaptability across various graph domains. GCAL employs a bilevel optimization strategy. The "adapt" phase uses an information maximization approach to fine-tune the model with new graph domains while re-adapting past memories to mitigate forgetting. Concurrently, the "generate memory" phase, guided by a theoretical lower bound derived from information bottleneck theory, involves a variational memory graph generation module to condense original graphs into memories. Extensive experimental evaluations demonstrate that GCAL substantially outperforms existing methods in terms of adaptability and knowledge retention.

GCAL: Adapting Graph Models to Evolving Domain Shifts
Ziyue Qiao 11
Qianyi Cai
Hao Dong² Jiawei Gu
Pengyang Wang"
Meng Xiao
Xiao Luo Hui Xiong 26
Abstract
Graph Model
Graph Model
Continual Adoptarton
This paper addresses the challenge of graph do- main adaptation on evolving, multiple out-of- distribution (OOD) graphs. Conventional graph domain adaptation methods are confined single- step adaptation, making them ineffective in han- dling continuous domain shifts and prone to catas- trophic forgetting. This paper introduces the Graph Continual Adaptive Learning (GCAL) method, designed to enhance model sustainabil- ity and adaptability across various graph domains. GCAL employs a bilevel optimization strategy. The "adapt" phase uses information maximiza- tion approach to fine-tune the model with new graph domains while re-adapting past memories to mitigate forgetting. Concurrently, the "gener- ate memory" phase, guided by a theoretical lower bound derived from information bottleneck theory, involves variational memory graph generation module to condense original graphs into memo- ries. Extensive experimental evaluations demon- strate that GCAL substantially outperforms exist- ing methods in terms of adaptability and knowl- edge retention. The code of GCAL is available at https: http//gtttttbbb com/ joe811 817 /CAL.
1
20.0
0CB
Prevent Fargetring
Ge-k
Evolving OOD graphs (a)
Comakaa
(b)
Figure 1. The challenge of continual adaptation of graph mod- els on evolving OOD graph sequences. (h) Empirical evaluations of the SOTA graph adaptation method across four OOD graph datasets in continual adaptation setting.
tricate connections, have been pivotal in advancing data min- ing and knowledge discovery. Classic graph models such as Graph Convolutional Networks (GCNs) (Kipf & Welling, 2016) and Graph Attention Networks (GATs) (Veličković et al., 2017) have been successfully applied in numerous applications ranging from social network analysis (Dong et al., 2023; Qiao et al., 2022; Sun et al., 2024) bioinfor- matics (Huang et al., 2024; Wang et al., 2024c) and recom- mendation systems (He et al., 2020; Wu al., 2024b).
Despite the successes, the sustainability of graph models in handling ever- increascng volumes of graph data presents unique challenges, particularly in scenarios involving new, unseen graphs. Such OOD scenarios commonly arise when model trained on one set of graph data is applied a different and novel set. This discrepancy underscores pivotal issue in graph machine learning: Domain Adaptation in Graph Models. The objective is to enhance the model's inference ability to generalize across different but related graph distributions without substantial retraining.
1. Introduction
Graphs are ubiquitously present the real world, serving as fundamental structures for representing complex systems in a multitude of domains. Graph models, leveraging these in-
'qqual contribution 'School Computing and Information Technology. Great Bay University 'Thrust of Artificial Intel- ligence. The Hong Kong University of Science and Technol- ogy (Guangzhou) CComputer Network Information Center, Uni- versity of the Chinese Academy of Sciences "University of Macau 'Department of Computer Science, University of Cali- fornia, Los Angeles 'Department of Computer Science and En- gineering, The Hong Kong University of Science and Technol- ogy Hong Kong SAR, China. Correspondence to: Hui Xiong <xionghuiie hk? Xiao Luoo <xiaoluo es udla.edu>, Meng Xiao shaoow @cnic.cn.>.
However, the current body of research often limits its focus to single-step adaptations employing techniques like Max- imum Mean Discrepancy (MMD) (Dziugaite et al., 2015) and adversarial learning (Dan et al., 2024; Qiao et al., 2023; Zhang et al., 2018). While useful, these techniques fall short when the model is subjected to continual domain shifts over time or domains: As graph datasets grow and evolve, models encounter new domains that necessitate ongoing adaptation but lose their ability to adapt to previous graphs, resulting
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
in catastrophic forgetting. The problem is illustrated in Fig- ure (a), and as the empirical results depicted in Figure (b), the state-of-the-art (SOTA) graph domain adaptation method EERM (Wu et al., 2022b) experiences continuous and serious decline in performance across four evolving OOD graph datasets.
2. Preliminary
We present the formulation for the continual adaptive learn- ing on graphs. Given a graph model f(O,) : G. + y pre-trained on one or multiple source graphs for a specific classification task, where O₂ represents the pee- pre-trained pa- rameters, G. is the source graph and Y = {vi, 1/2, 1/2,..., ...; Vc) is the set of C' classes. The sequence of m target domain graphs is defined as {Gi, G2, Gm where each graph G = Xi} belong to the f-th domain. A E R N. N.X.M is the adjacency matrix and N₂ are the numbers of nodes. X, E RN is the attribute matrices and are the dimen- sions of node attributes. In this scenario, the target graphs arrive one by one sequentially, and each target graph may exhibit a different distribution from previous ones due to changes in the underlying data over region and time, i.e., GG.) # p(G * j. where p(-) is the data distribu- tion. We aim at sustainable reusing of the graph model for the continual adaptation and inference on multiple out-of- distribution target domains within the same task in an online fashion. As the target domain graph G. arrives, we feed the model on G2 to adapt the parameters Q,-1 + O, and make the prediction accordingly. There are two purposes in the process of test-time training: (1) Adapting: we aim to ensure that the model adapts effectively to new target do- main graphs as they arise; (2) Avoid Forgetting: we aim to retain the model's performance on previously encountered target graphs after each adaptation, all without the need for complete retraining.
Continual learning (Wang et al.. 2024a; 2022; Zhang et al., 2022arb; 2023b), or lifelong learning, offers a promising so- lution to mitigate catastrophic forgetting. A widely adopted strategy within this paradigm is the replay mechanisms, where the model periodically revisits selected or generated old data to reinforce past knowledge. This practice helps in maintaining a balance between training cost and knowledge retention. However, existing approaches typically rely on labeled data select and replay memories, presenting a sub- stantial barrier in many applications where such labels are either unavailable or prohibitively expensive to obtain. This introduces a significant research gap: the development of continual adaptive methods for unsupervised memory gen- eration and replay in graph models. Such methods would need to autonomously identify critical features and struc- tural patterns within graphs that are essential for the model's long-term adaptability and robustness.
In this paper, we introduce novel method named Graph Continual Adaptive Learning (GCAL), specifically designed to tackle the challenges of catastrophic forgetting in graph models' continual adaptation. Our approach employs an "adapt and generate memory" bilevel optimization strat- egy, activated each time new graph data is introduced. For "adapt," utilize information maximization approach to adapt the model to new domain graphs, simultaneously re-adapting the previous memory graphs to prevent forget- ting. For "generate memory," we theoretically derive a lower bound for preserving informative and generalized memory graphs from the current graph, leveraging the principles of the information bottleneck. The main contributions of this research are outlined follows:
3. Methodology
As shown in Figure 2, our method first utilizes an infor- mation maximization approach to adapt the current model θ--1 to the newly arrived graph G2 while simultaneously conducting memory replay on the previous memory graphs to avoid forgetting, which will be introduced in Sec. 3.1. Then the updated model parameter θ is used to learn the small memory graph G. for G., which will be introduced in Sec. 3.2. To generate the memory graph, we develop a variational memory graph gemeration module comprising variational GNN, trainable selector, and novel graph structure learning and reparameterization technique. To optimize the memory graph, we follow the lower bound to introduce three learning objectives. The memory graph learning loss uses a graph condensation technology to learn task-related memory graphs. The regularization losses are proposed to ensure the stability and informativeness of the memory graph. The generation loss enhances the relevance of the memory graph to the original graph.
We introduce the GCAL framework to effectively man- age catastrophic forgetting and enhance the sustainable reuse of graph models during their continual adaptation across evolving OOD graph data.
We derive a theoretical lower bound that ensures the preservation of informative and generalized memory graphs. Based on this foundation, we design a memory graph generator equipped with three tailored losses to effectively guide the memory graph learning process.
We conduct extensive experiments on various graph datasets, demonstrating that GCAL significantly out- performs of-e-of-thheann art across domain shifts.
3.1. Adaptation with Memory Replay
In the I-th adaptation step, the goal is to refine the current model f(0,-1) to make the prediction to enhance its pre-
2

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
Adaptation with Memory Replay
Memory Graph Learning
GNN
Classiiee
GNN
Classifier
E
L
Step
1+
u(G,)
Poromater shoring
L AMR
GNN
Chassifies
Bi
Vartational Memorr
LM
Gen
Forward
Graph
Generator
Pool
Backpropagation
81
R2
...
Variational Memory Graph Generator
a(0+))
K
G
GNN
Classifer
GNNa,
&
Adp
Regg
logm
A
8
TopKSclector
Figure 2. The GCAL framework involves several steps: Starting with the graph model f(O:-1), the current graph Gs, and the accumulated memory graph pool G (= GCAL first applies the Adaptation with Memory Replay method using loss CAMe for model adaptation. Next, Variational Memory Generator creates new memory graph G. for Gt, which is refined using the memory graph learning loss LMGL, the regularization losses Leeg to ensure stability and informativeness, and the generation loss LGen to enhance the memory graph's relevance G.. Finally, G2 is added G for future adaptation.
dictive accuracy on target domain graphs. Since there is no label on the target graphs, the model adaptation is primar- ily conducted self-supervised. We adopt the Information Maximization (Liang et al., 2020) leveraged on the output probability of the model. This approach rests on the fun- damental premise: A model that effectively discriminates target data will exhibit high inferential confidence, char- acterized by output probabilities that closely resemble a one-hot vector. With the output probability pv of each node encoded from pre-trained model f(0,-1), the objective to minimize risk (see Eq. 1) on is as follows:
By combining the adaptation loss on the new graph G2 with the adaptation losses the graphs in the memory pool g, the model f(Ot-1) is continually refined to adapt to new graphs while reinforcing performance on historical graphs.
3.2. Variational Memory Graph Generation
In our model, meanwhile adapting to the new graph G2 with memory-aware replay, our goal is also to learn the memory G corresponding to Gr, forming a series of memories G. that can be replayed when the next adaptation task arrives to prevent forgetting. We consider three factors: (1) the mem- ory size should significantly smaller than the original graph, (2) the memory should be informative, retaining as much important information from the source graph possi- ble, and (3) the memory should be generalizable, capable of being stored across diverse graph distributions.
Caap(G; 0,-1) = -
EE Pork log(pw,))) + +hh De log Pu. (1)
where Pi,k denotes the k-th element of pt. The expected probability P&s is calculated as 2% = EevelPe.s) The sec- ond term introduces diversity regularization designed to enhance the variety output probabilities. This regulariza- tion helps prevent the issue where a few high integrity scores might dominate during training, potentially causing all un- labeled nodes to converge towards the same pseudo-label, resulting in overfitting.
3.2.1. DERIVING INFORMATION BOTTLENECK ON MEMORY GRAPHS
To achieve the above motivations, we proposed varia- tional information bottleneck based memory graph genera- tion method. Graph information bottleneck (Sun et al., 2022; Wu et al., 2020b) usually aims to maximize the below:
To prevent the model from forgetting previously learned graphs while adapting to new graphs, known as memory replay in continual learning, we apply the information max- imization loss not only to the graph but also to all previous graphs. For efficient replay, we use graph mem- ory pool, denoted as g = ( G.,... G.-1). This pool contains a sequence of smaller synthetic graphs, each repre- senting a previously encountered graph. We then perform adaptation using memory replay, formulated as follows:
G, = arg max & [1 - -
(3)
where I(=:-) denotes the mutual information and Y, repre- sents the training signals associated with nodes in G. The first term, G.. Y.). aims to preserve task-related informa- tion within the memory, while the second term, I(G Gt), focuses on compressing information from the original graph into the smaller memory graph, effectively filtering out irrel-
LAMR LGr 0.-1)+ + Σ CG: (2)
3

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
evant information. acts as a trade-off parameter, balancing the compression of input data with the preservation of task- relevant information.
where p IRA is its trainable parameters. We compute the logarithm of the standard deviation (i.e., log Ø) rather than directly calculating which smoothly scales the deviation, enhancing numerical stability and interpretability.
Despite the goal of G. to preserve information from the graph, directly generating it from G, is challenging. Instead, we generate G. from the original graph through a varia- tional latent representation Z,, expressed as Pg = Pa (G, P, (Z.(Ge) where g is the generator with parameter o. Utilizing this chain rule, we can reformulate the second term Eq.3 as: T(GG;G) = T(G;; G, Z,)
We first generate the latent variable of each node of the mem- ory graph from the distribution via the reparameterization trick:
3 N (Ziluss o,) = p. + +o O
(7)
where i = 1,.., K and E E NN 1) is random variable drawn from a standard normal distribution. This reparam- eterization ensures that the sampling process remains dif- ferentiable, allowing the gradients to be backpropagated through the sampling step during training.
I[:;; Ze Gt). Consequently, the optimization objective for generating G. can be reformulated as follows:
[(Φ) max [IG 8.) BI G., Z,) + B Z.)
Then, we assume 2; as the node features in the memory graph and obtain the feature matrix via X, {d) where id(-) is the identity function. We further generate the edges of the memory graph from 3, We assume that each edge follows an independent Bernoulli distribution, with each edge characterized by binary random variable a:,j ~ Bernoulliww..) for each edge. We use 2 to generate the learnable Bernoulli weights for each edge. Given that di,j is non-differentiable to wi.j we approximate it as continuous variable within the interval [0.1]. To facilitate gradient- based optimization, the Gumbel-Max reparameterization trick, as detailed by (Maddison et al., 2017), is employed to update the edges as follows:
(4)
To optimize this objective in parameterized manner, we derive a lower bound in the following Theorem:
Theorem 3.1. Let G. be a generated graph conditioned on the latent representation Z, of the original graph Gt. Suppose ((G.) is a variational approximation of the true posterior P(G)), Then, the following lower bound on the optimization objective for & holds:
(()) BBloog Py (f(().) SE[KL Q(G.))
I BE]oog(P C.. Z.)).-
(5)
Here, KL(- indicates the Kullback-Leibler divergence. Pj is considered as the classifier f(O,).
Wi.j = (MLP((E;;))) + MLP((
2
(8)
The proof can be found in Appendix A. Thus, the mem- ory graph learning objective can be maximizing the above lower bound. In the following sections, we first introduce the variational memory generator. Then, we introduce the optimization objectives for each item in Equation 5 in detail.
(WL)) + +og log 8 1/T)
a.j Sigmoid
1-8)/7)
where 5 ~ Uniform 1) and T represents the temperature hyperparameter. As T approaches 0, a; becomes increas- ingly binary. The reparameterization enables a well-defined gradient, das,, allowing for effective training of w. Con- sequently, ai can be obtained through the training process and used as the edge weight in constructing the adjacency matrix A, for the memory graph.
3.2.2. VARIATIONAL MEMORY GRAPH GENERATOR
In Theorem 3.1, we define the memory graph generator as g( P(G.) - P(G.). We first employ a GNN archi- tecture process the input graph G,, transforming it into latent distributions:
In this way, by combining the above modules together, we obtain the variational memory graph generator g and the memory graph is obtained by G. = g(G,, b) ( (A,, X.). Leveraging the variational approach not only aids in the gen- eration of nodes and edges but also helps in managing and optimizing the underlying distributions of these elements efficiently.
[p; log o] = Top a.a (Ac,X)),,
Xp)
TopKSelector( X)= = argsort Sigmoid x
[: K],
|pl|
(6) where j E KKh and logo E k represent the mean and variance components for each node, respectively. K < N2 is the number of nodes in the generated graph, and h is the hidden dimension. GNN nr.or (-) is parameterized to output a vector of dimensions 2 * h, divided into mean and variance components. To manage graph dimensionality and emphasize significant distributions, a top-k selector layer Telette))))r() reduces the number of distributions,
3.2.3. MEMORY GRAPHI LEARNING VIA CONDENSATION Loss
The first term in Eq.5, E llg (f,Y,).)), involves maxi- mizing the expected log- ligellieloood of the predicted out- comes given the generated memory graphs G.. This can

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
be reframed as minimizing the condensation loss (Jin al., 2021), novel objective that enhances the fidelity and rele- vance of the generated graphs to downstream tasks:
the predefined probability parameter. Additionally, consis- tent with prevailing approaches in the literature, we define Q(X,) for node features a Normal Gaussian distribution (00, I). where I represents the identity matrix in R Then, the overall regularization loss can be defined as fol- lows:
min 9.), P.). G.E.G G. 2
(9)
s.t. = arg min (
K
where L is the task-related loss on the graphs and corre- sponding training signals. As P, and Y, are not directly observable, we instead use the adaptation loss in Eq. I with the soft pseudo-labes labuls as training signals. In previous ap- proaches, the gradient matching scheme was often employed to minimize this loss. This method aligns the gradients of the model trained on the generated memory graph G, with the gradients from the true graph G2 with respect to the network parameters θ, By doing so, it ensures that the model's 's behavior on the generated data closely mirrors its behavior on actual data, facilitating better generalization:
Reg
min b
2
(p 3 + oi.5 - -oog(o - 1) +
1= ==1
K
Wi.j + (1 - -wi))))) ) log 1 Wi,j
Wi,
log
q
1 - 9
+,==
(13)
This approach acts as the regularization term that ensures that the variability introduced during the generation of new memory graphs is effectively controlled, leading more stable adaptations over successive domains.
3.2.5. GENERATION LOsS
LMGL minn (DLAer 00:
(BLad)(G.: (CAap(Gr:((O.) 0e,
For the last terms Eq.5, the objective becomes minimiz- ing -Ellog the graph G,. In conventional methods, the original graph usually serves as reference for optimizing the generated graph, typically employing a reconstruction-bant -based discrep- ancy loss to quantify the differences between the generated and the original graphs. However, our approach deviates from traditional methods due to the reduced size of the gen- erated graph, challenging direct structural and feature-based based comparisons. To address this, we adopt distribution- discrepancy measure, specifically designed to assess differ- ences in the aggregate properties of the graphs:
(10)
Z,))], the likelihood of generating
where -) is the sum of the distance between gradients at each layer. Given two gradients E Iddixda and g E
xd2 at specific layer, the distance between them is
defined as:
d2
D g)
gi g
(11)
g
where gi g: are the i-th column vectors of the gradient matrices. With this optimization, we are able to achieve
task-related memory graph learning through the efficient gradient-matching strategy.
K Σ (,( Σ (
N
min LGe = min Gis(Gr, Gt) =
3.2.4. REGULARIZATION Loss
(14)
The second term in 5q. (G. Z)) Q()))) corresponds to the KL divergence measure the learned dis- tribution Pg of the predicted graph G,, given the actual graph G, and latent variables Z, deviates from simpler prior Q(G.). We refine the prior distribution Q(G.) by dis- tinguishing between the components of node features and edges, i.e., Q(G,) (A,,, X,) = Q(A,) Q(x,) Thus, the optimization objective of the term can be rewritten as minimizing the following:
where Dis(-) means the discrepancy between the memory graph and the original graph, ū,))), u,,)) E TRA are the hidden representations of nodes encoded from the model f()) before the final classification head. Note that the node representations are different from those in Eq.7 which are encoded by the variational generator. || ||2 denotes the L2 normalization distance. We did not use MMD or adversarial alignment methods but more concise measure minimize the distribution discrepancy because the detailed distribution learning has already been effectively optimized in the former modules, and additionally, our method is more efficient and robust.
(P, Z:) Q(ĀM) 2
(12)
+ EJKL(P, G.,, Z)) |I Q((.))
3.3. Optimization
For the first term, as outlined in Section 3.2.2, we define the edges to follow an independent Bernoulli distribution. Accordingly, we specify Q(A,) such that each edge a,, ad- heres to a Bernoulli distribution Bernoulli(g), where q is
Combining the loss of adaptation with memory replay and the three losses in memory generation, we can establish the overall learning objective as a bi-level optimization frame-
5

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
work. When the 1-th graph G2 arrives, the learning contains
Table 1. The statistics of datasets with distribution shifts, where # denotes "the number of".
two stages, the inner loop aims to adapt the model f(O+-1) on G2 while the outer loop aims to learn a memory graph Go based on the adapted model f())):
Category
Datasets
#Nodes
#Edges
#Domains
Regional Shifls
Twirch-explicit
1,912-9,498 9,498
31,299 153,138
7
min (G, 0; + Re +) + JeCaenGG, Ox; P),
Facebook- 100
769 41,554
33,312 13,312-2,724,455
6,
Temporal Shifts
Elliptic
1,168- 9,,6991644
4]
s.L. θ = aig LAMR(GG, (. 1),
OGB-Arxiv
14227-397111
1,225 38,735
11
e
(15)
where W1, 22 are the loss weights. For the generator, for each timestep, a new generator is utilized to create the memory graph, and only the memory graphs are preserved in the memory buffer. We also use an exponential moving average (EMA) strategy ( (Wang et al., 2022) to update the mode parameters to smooth the parameter updates.
tic dataset, the metric used is the Fl Score, while for the OGB-Arxiv dataset, Accuracy is used. To compute a single numeric value upon completing all domains, we calculate the Average Performance (AP) + Σ MT....l primar- ily assessing adaptation ability, and the Average Forgetting (AF) as --1 Σis (MF.. - -M,sulh, ) primarily evaluating the ability to avoid forgetting. Each experiment is repeated five times, with results reported as the mean and standard deviation. Detailed introduction for datasets, baselines, and experimental settings is in Appendix B.
4. Experiments
4.1. Experimental Setup
Datasets Our paper involves two primary categories of graph datasets, differentiated by regional and temporal shifts. For regional shifts, Facebook-1000g 100(Traud et al., 2012) and et al., 2021) datasets consist of multiple social networks from different regions. For tem- poral shifts, OGB-Arxiv(Hu al., 2020) a paper citation network dataset, and Elliptic(Pareja et al., 2020) is Bitcoin transactions network dataset, both of which include graphs from different time steps. In these datasets, each graph is treated as separate domain. We select certain domains to pre-train a graph model and then adapt it continuously using the remaining graphs.
4.2. Experimental Results
Switch-Explicit(Rzzenle
1.2.1. OVERALL PERFORMANCE COMPARISON.
This experiment aims to answer: How does GCAI, perform in the unsupervised continual adaptation setting across evolving graph data? We compare GCAL with various baselines divided by different domain adaptation strategies and report the experimental results in Table 2. Note that Test does not involve modifying the model; EERM and GTrans train a new parameter at each graph, inapplicable to previous ones. Thus, their AF results are not applicable. Ob- viously, we observe that GCAL sets a new state-of-the-an surpassing all baseline methods across all datasets.
Baselines. We evaluate the performance of our continual adaptive learning framework against a diverse of base- line methods. Test employs pretrained graph model to perform direct inference on the target dataset without any adaptation, serving as the lower bound. The category "No Rehearsal Based Test-Time Adaptation" comprises one-step test-time adaptation methods, including DANN (Ganin et al., 2016), Tent (Wang et al., 2021). BN Stats Adapt (Li al., 2016), EERM (Wu et al., 2022b), and GTRANS (Jin al., 2022). The category "Continual Test Time Adaptation" refers to continual test-time training methods, including CoTTA (Wang et al., 2022) and EATA (Niu et al., 2022). For baselines not originally designed for graphs, their archi- tectures have been adapted to GCNs to ensure consistency in evaluation.
Specifically, certain baseline methods, especially traditional domain adaptation methods, demonstrate poor results. This can be attributed to the difficulty of the problem, which involves various timesteps of unsupervised continual adap- tation. This setting requires the model to dynamically adapt to new domains while concurrently retaining knowledge of previous ones. Specifically, when model fails preserve knowledge from the current domain, accumulating errors may disrupt performance in future domains and even cause the model to degrade over time. Our approach mitigates this challenge and collectively enhances the sustainable reuse of graph models and effectively alleviates catastrophic for- getting across evolving graph data via the Adaptation with Memory Replay framework. Among the advanced base- lines, the most comparable method GCAL is COTTA. Our method outperforms COTTA primarily by integrating the variational memory graph generation module. Instead of solely using EMA for model updates, we employ the Varia- tional Memory Generator to generate previous graphs that enhance knowledge transfer. This strategy not only lever-
Evaluation Metries. We report the performance matrix Mresuit E IR TxT, which is lower triangular matrix where Mrssult (for i 2 j) represents the performance on the do- main j after training on the domain i. Specifically, similar to (Jin et al., 2022; Wu et al., 2022b), for the "witch-Explici and Facebook- 100 datasets, the results are measured us- ing ROC-AUÇ and Accuracy respectively. For the Ellip-
6

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
Table Data performance comparison across four datasets, evaluated against eight haseline methods and GCAL. Test represents the lower bound, while Full indicates the upper bound. N/A: Not Applicable.
Txitch-explicit
Facebook- 100
Elliptic
OGB-ArXiv
Methods
AP-AUC/9t
AF/
AP-ACC/#t
AF/
AP-FI/K1
AF/%
AP-ACC%61
AF/
Test 53.88+0.00
N/A
50.55=0.000
N/A
53,97=0.00
N/A
42.43++0.00
N/A
DANN
51.50++0.39
0.19++0.55
50.63+0.64
0.38+1.13
53.84=0..56
-0.95=0.54
42.62++0.23
11.28=0.16
Norm
52.65++0.00
-2.30+0.00
47.62++0.00
0.53++0.00
54.67+0.00
0.02++0.00
40.21+00.00
0.11=0.00
TENT
52.84-00.88
-1,83+1005
46.63++0.00
0.61+00.00
46.54+00.00
0.55++0.00
36.72+00.37
-0.02-0.18
EERM
52.08+0.000
N/A
49.70+0.03
N/A
46.51+00.00
N/A
36.26+00.00
N/A
GTrans
53.55++0.29
N/A
0OM
N/A
54,25+0.02
N/A
39.69+0.04
N/A
CoTTA
53.94+00.36
0.34++0.4
50.12+0.11
0,59++0.12
54.08+0.05
-1.92+0.06
40.28+0.01
-1,96+0.01
RATA
53.56++0.10
-0.72+0.07
49.02+3.116
0057
50,48+0.077
-0.76+00006
40.91++0.70
-1,35+0.59
GCAL
55.65:10.09
0.42:10.13
52.72:10.36
0.72:10.19
56.57:10.14
0.113
45.22:10.17
0.76=0.05
3ua
1om
16m
COAL
Nom
0C4
Ascerage ACCP%
Aoversge ACC.
Averge
10
12
10
20
30
40
10 11
Domaits
Domain
Damaira
(a) Twitch-explicit
(b) Facebook- 100
(c) Elliptic
(d) OGB-Arxiv
Figure 3. Dynamics of the average performance during continual adaptation on evolving OOD graphs.
ages past insights for improved adaptation but also enriches the training process, boosting the model's adaptability and robustness across evolving domains. DANN achieves rela- tively high performance among the baselines as it uses the source graph to guide the adaptation, while others only use the target graph data. Notably, our method surpasses DANN in four datasets, particularly AP. This improvement is attributed to GCAL's enhanced ability adapt effectively to new target domains, leverage knowledge from past periences for better adaptation, and maintain performance on previously encountered domains, all without the need for complete retraining. This novel approach significantly bolsters the model's performance and adaptability.
accuracy matrices of GCAL and CoTTA on the Twitch and Elliptic datasets. The results are presented in Figure 4. In these matrices, each row represents the performance across all domains upon learning a new one, while cach column captures the evolving performance of a specific domain as all domains are learned sequentially. In the visual repre- sentation, darker shades signify better performance, while lighter hues indicate inferior outcomes.
From the results, we observed that as the number of do- mains increases, the learning objectives grow increasingly complex, resulting in reduction in performance across all examined methods. That is because as domains accumulate and the learning objectives become multifaceted, it becomes challenging for models to maintain optimal performance across all domains. Notably, most baselines experienced substantial decline, with the model collapsing with the arrival of merely few new domains, demonstrating that catastrophic forgetting occurs almost immediately when the model fails to access previous domain data. This rein- forces the need for effective continual learning techniques on the sequential graphs where new domains frequently emerge. While the performance drop was observed across all methods, GCAL demonstrated resilience and outper- formed the top-performing bascline CoTTA. Also, GCAL predominantly displays lighter shades across the majority of blocks compared to CoTTA in Figure 4. Moreover, its
4.2.2. IN-DEPTH ANALYSIS OF CONTINUOUS PERFORMANCE,
This experiment aims answer: How does GCAL's fine- grained performance evolve after continuously learning each domain? To present more fine-grained demonstration of the model's performance in continual adaptive learning on graphs, we analyzed the average performance across all previously encountered domains each time a new do- main was learned. The comparative results of Test, DANN, GCAL, and the top-performing baseline are depicted in Fig- ure 3. The curve represents the model's performance after t in terms of AP on all previous / tasks. Also, we visualize the

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
Domairs
Comal
20
10
339
10
20
30
40
20
20 Domains
30
40
Damainss
Domains
Domains
(a) Twitch-CoTTA
(b) Twitch-GCAL
(c) EllipticCoTTA
(d) Elliptic-CAAL
Figure 4. Performance matrices of GCAL and CoTTA in different datasets.
Table The results the ablation study.
35.00
13..00
Methods
Twitch-explicit
Facebook- 100
Flliptic
OGB-Arsiv
fverage
$2,00
w/o Citeg & Liien
54.0342..3
$2.05=0.31
46.53-20.01
44.70=0.06
$5.40
Caay
45.34+0441
52.37-00.66
55.23++0.22
44.76+00.48
wlo Erien
35.37=0.38
52.14=0.32
55.64+0.57
44.91#0.11
win EMA
54.79+00.04
47.66++0.06
53.83++0.20
43.19+0..88
95.00
50.00
52.72.10.36
45.22.10.17
Ratio
Ratio
Twitch-explich
(b) Facebook-
58.00
46,00
competitive performance in specific datasets signifies its robustness and capability. This could be attributed to the "adapt and generate memory" framework, which not only re- tains critical knowledge from previous tasks but also adapts to new ones.
$5.00
55.00
44.50
$4.00
46.00
0.23
0.05
0.02
0.23
2.05
0.37
0.50
Prompt Ratio
Prompt Ratin
4.2.3. ABLATION STUDIES. (RQ3)
(c) Elliptic
(d) OGB-Arxiv
This experiment aims to answer: Do all the proposed compo- nents of GCAL contribute esfectively to continual adaptation on graphs? For that, we design four variant methods for GCAL to verify the EMA in the model parameter updat- ing module, regularization loss, and generation loss in the variational memory graph generation module: w/o Reg & LGen. w/o Reg. w/o LGen. and w/o EMA, where "w/o" means "without" the corresponding losses or components in model continual adapting. The results are presented in Table Firstly, we can observe that when each of the losses or components removed, the model's performance de- creases across all datasets; while combining all modules, the method achieves the best results, providing straightforward evidence that all the proposed techniques contribute to our method. For w/o LGen the generation loss, the method still achieves remarkable results, primarily due to regularization loss and EMA updates. These components help optimize the latent space representation and ensure smooth model updates, thereby enhancing generalization across diverse data distributions.
Figure 5. The model performance with different synthetic ratios.
with K nodes for memory replay, where K is much less than the number of nodes in the initial graph. Thus, we set K as 0.01, 0.03, 0.05, 0.07, 0.09 of the num- ber of nodes for the datasets Facebook- 100 and OGB- Arxiv, and as 0.01, 0.02, 0.03, 0.04, 0.05 for Elliptic, and 0.09, 0.10, 0.11, 0.12, 0.13 for Twitch, respectively. From the results in Figure 5, we observe that even with a low ratio of synthetic nodes, our model performs well across all datasets. In partieular, for Facebook- 1 100 and OGB-Arxiv, which have larger number of nodes, the model continues to achieve promising results even at lower synthetic node ratios. This demonstrates that our method effectively preserves the information from previous source distributions, as captured by μ and a, highlighting its efficiency and robustness.
5. Conclusion
In conclusion, GCAL addresses the critical challenge of un- supervised continual adaptation to out-of distti--ittiittionn graph sequences. By employing a bilevel optimization strategy, GCAL effectively manages domain shifts and prevents catas- trophic forgetting. The method fine-tunes models on new
4.2.4. HYPERPARAMETE EXPERIMENTS.(ROL
This experiment aims to answer: How do synthetic ra- tio impact the performance of GCAL? With the Varia- tional Memory Generator, we generate synthesized graphs
8

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
domain graphs while reinforcing past memories through the adaptation phase, and generates informative and relevant memory graphs guided by a theoretical framework rooted in information bottleneck theory. Experimental results indicate significant improvements over existing methods in terms of adaptability and knowledge retention, enhancing model sus- tainability and adaptability, A potential limitation of GCAL its lack of improvement the graph model architecture itself. This could potentially hinder performance when the base model itself is less capable or inadequate for complex graph data scenarios.
Ding, Z., Li, S., Shao, M., and Fu, Y. Graph adaptive knowledge transfer for unsupervised domain adaptation. In Proceedings of the European Conference Computer Vision (ECCV), pp. 37-52, 2018.
Döbler, M., Marsden, R. A., and Yang. B. Robust mean teacher for continual and gradual test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7704-7714, 2023.
Dong, H., Ning, Z., Wang, P., Qiao, Z., Wang. P., Zhou, Y., and Fu, Y. Adaptive path-memory network for tempo- ral knowledge graph reasoning. In Proceedings of the Thirty-Second International Joint Conference on Artifi- cial Intelligence, pp. 2094-2094, 2023.
Impact Statement
This paper presents work whose goal to advance the field of Machine Learning, with particular focus on model trans- fer learning under continual distribution shifts in structured data. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here. This research has been conducted with commitment to ethical standards and presents no ethical concerns.
Dziugaite, G. K., Roy, D. M., and Ghahramani, Z. Training generative neural networks via maximum mean discrep- ancy optimization. arXiv preprint 1505:35505.0390 2015.
Gan, Y., Bai, Y., Lou, Y., Ma. X., Zhang, R., Shi, N., and Luo, L. Decorate the newcomers: Visual domain prompt for continual test time adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 7595-7600 2023.
Acknowledgments
The work is partially supported by the National Natural Science Foundation of China (No. 62406056), the Guang- dong Basic and Applied Basic Research Foundation (No. 2024A15151400 11111511111144 14, No. 2023B 1551 10057),, the Beijing Natural Science Foundation (No. 4254089), the National Key R&D Program of China (No. 2023YFF0725001), the National Natural Science Foundation of China (No. 92370204), and Guungzhou-HKUSTTGGZ Joint Funding Program (No. 2023A03J0008), Education Bureau of Guangzhou Municipality, The computational resources are supported by SongShan Lake HPC Center (SSL-HPC) in Great Bay University.
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., March, M., and Lempitsky, V. Domain- adversarial training of neural networks. Journal of ma- chine learning research, 17(59):1-35, 201,
Gao, X., Chen, T., Zang, Y., Zhang, W., Nguyen, Q. V. H., Zheng, K., and Yin, H. Graph condensation for inductive node representation learning. In 2024 IEEE 40th Inter- national Conference on Data Engineering (ICDE), pp. 3056-3069. IEFF, 2024.
Hamilton, W., Ying, Z., and Leskovec, J. Inductive repre- sentation learning on large graphs. Advances in neural information processing systems, 30, 2017.
References
Cai, J., Wang, X., Guan, C., Tang, Y., Xu, J., Zhong, B., and Zhu, W. Multimodal continual graph learning with neural architecture search. In Proceedings of the ACM Web Conference 2022, pp. 2292-130,, 2022.
Hashemi, M., Gong, S., Ni, J., Fan, W., Prakash, B. A.. and Jin, W. A comprehensive survey on graph reduc- tion: Sparsification, coarsening, and condensation. arXiv preprint arXiv:2402.033358 2024.
He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., and Wang, M. Lightgen: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp. 639-648, 2020.
Dai, Q., Wu, X.-M,, Xiao, J., Shen, X., and Wang, D. Graph transfer learning via adversarial domain adaptation with graph convolution. IEEE Transactions on Knowledge and Data Engineering. 2022.
Dan, J., Liu, W., Xie, C., Yu, H., Dong, S., and Tan, Y. Tígda: Exploring topology and feature alignment in semi- supervised graph domain adaptation through robust clus- tering. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., and Leskovec, J. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118 18-22133, 2020.

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
Huang, X., Ma, Z., Meng, D., Liu, Y., Ruan, S., Sun, Q.. Zheng, X., and Qiao, Z.. Praga: Prototype-aware graph adaptive aggregation for spatial multi-modal omics anal- ysis. In AAAI 2025, 2024.
In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 8266-8276, 2019.
Maddison, C., Mnih, A., and Teh, Y. The concrete dis- tribution: A continuous relaxation of discrete random variables. In Proceedings of the international conference learning Representatiins. International Conference on Learning Representations, 2017.
Jin, W., Zhao, L., Zhang, S., Liu, Y. Tang. J., and Shah, N. Graph condensation for graph neural networks. arXiv preprint arXiv:2 00.0780, 2021.
Jin, W., Zhao, T., Ding, J., Liu, Y., Tang, J., and Shah, N. Empowering graph representation learning with test- time graph transformation. In The Eleventh International Conference on Learning Representations, 2022.
Niu, C.,, Pang. G., Chen, L., and Liu, B.
free graph glass-nncrementll learning: A task profiling and prompting approach. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
Kipf, T. N. and Welling, M. Semi-supervised classifica- tion with graph convolutional networks. arXiv preprint (eli:v6600029000 2016.
Niu, S., Wu, J., Zhang, Y., Chen, Y., Zheng, S., Zhao P., and Tan, M. Efficient test-time model adaptation with- out forgetting. In International conference on machine learning, pp. 18888-16905. PMLR, 2022.
Li, J., Wang. Y., Zhu, P., Lin, W., and Hu, Q. What mat- ters in graph class incremental learning? an information preservation perspective. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Pareja, A., Domeniconi, G., Chen, J., Ma, T., Suzumura, T., Kanezashi, H., Kaler, T., Schardl, T., and Leiserson, C. Evolvegen: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI confer- ence on artificial intelligence, volume 34, pp. 5363-5370, 2020.
Li, Y., Wang, N., Shi, J., Liu, J., and Hou, X. Revisit- ing batch normalization for practical domain adaptation. arXiv preprint arXiv: /603.0476007777 2016.
Qiao, Z., Fu, Y., Wang, P., Xiao, M., Ning, Z., Zhang, D., Du, Y., and Zhou, Y. Rpt: toward transferable model on heterogeneous researcher data via pre-training. IEEE Transactions on Big Data, 9()): 2022.
Liang, J., Hu, D., and Feng. J. Do we really need to access the source data? source hypothesis transfer for unsuper- vised domain adaptation. In International conference machine learning, pp.
PMLR, 2020.
Qiao, Z.,, Luo, X., Xiao, M., Dong, H., Zhou, Y., and Xiong, H. Semi-supervised domain adaptation in graph trans- fer learning. In Proceedings of the Thirty-Second Inter- national Joint Conference on Artificial Intelligence, pp. 2023.
Liu, H., Yang. Y., and Wang. X. Overcoming catastrophic forgetting in graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, 2021.
pp.
Liu, M., Li, S., Chen, X., and Song, L.. Graph condensation via receptive field distribution matching. arXiv preprint arXiv:2206. 1rr6..1360 13697, 2022.
Qiao, Z., Xiao, M., Guo, Luo, X., and Xiong, H. Infor- mation filtering and interpolating for semi-supervised graph domain adaptation. Pattern Recognition, 153: 1 110498, 2024.
Liu, S., Li, T., Feng, Y., Tran, N., Zhao, H., Qiu, Q., and Li, P. Structural re- re-weighting improves graph domain adap- tation. In International Conference on Machine Learning, pp. 21778-21793. PMLR, 2023a.
Qiao, 7.,, Xiao, J., Sun, Q., Xiao, M., Luo, X., and Xiong, H. Towards continuous reuse of graph models via holistic memory diversification. In The Thirteenth International Conference on Learning Representations, 2025.
Liu, Y., Qiu, R., and Huang, Z. Cat: Balanced continual graph leamnngg with graph condensation. In 2023 IEEE International Conference on Data Mining (ICDM), pp. 1157-1162. IEEE, 2023b.
Rozemberczki, B., Allen, C., and Sarkar, R. Multi-scale at- tributed node embedding. Journal of Complex Networks, 9(2)cnabb014 2021.
Liu, Y., Qiu, R., Tang, Y., Yin, H., and Huang, Z. Puma: Ef- ficient continual graph learning with graph condensation. arXiv preprint ar3iv:2312 2.. 14439, 2023c.
Sener, O. and Savarese, Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv: 00489,,0049000000 2017.
Ma, X., Zhang, T., and Xu, C. Gcan: Graph convolutional adversarial network for unsupervised domain adaptation.
Song, J., Lee, J., Kweon, S., and Choi, S. Ecotta: Memory- efficient continual test-time adaptation via self-distilled
10

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
regularization. In Proceedings of the IEEE/CVF Confer- ence Computer Vision and Pattern Recognition, pp. 11920-11929, 2023.
Wang, Q.. Fink, O., Van Gool, L.,, and Dai, D. Continual test- time domain adaptation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 7201-7211, 2022.
Sun, L., Ye, J., Peng. H., Wang, F., and Philip, S. Self- supervised continual graph learning in adaptive rieman- nian spaces. In Proceedings of the AAAI Conference on Artificial Intelligenee, volume 37, pp. 4633-4642, 2023a.
Wang, Q. Sun, X., and Cheng, H. Does graph prompt work? a data operation perspective with theoretical analysis. arXiv preprint arXiv:2410001635, 2024b.
Sun, Q. Chen, Z., Yang, B., Ji, C.,, Fu, X., Zhou, S., Peng, H., Li, J., and Philip, S. Y. Ge-bench: An open and unified benchmark for graph condensation. In The Thihty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
Wang, X., Duan, M., Li, J., Ma, A., Xin, G., Xu, D., Z., Liu, B., and Ma, Q. Marsgt: Multi-omics analysis for rare population inference using single-cell graph transformer. Nature Communications, 15(1):338, 2024c.
Sun, Q., Li, J., Peng, H., Wu, J., Fu, X., C., and Philip, S. Y. Graph structure learning with variational informa- tion bottleneck. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 4165-4174 2022.
Wu, M., Pan, S., Zhou, C., Chang, X., and Zhu, X. Unsu- pervised domain adaptive graph convolutional networks. In Proceedings of The Web Conference 2020, pp. 1457- 1467, 2020a.
Wu, M., Pan, S., and Zhu, X. Attraction and repulsion: Unsupervised domain adaptive graph contrastive learn- ing network. IEEE Transactions on Emerging Topics in Computational Intelligence, 2022a.
Sun, Q.. Chen, C., Qiao, Z., Zheng, X., and Wang, K. Single- view graph contrastive learning with soft neighborhood awareness. In AAAI 2025, 2024.
Sun, X., Cheng, H., Li, J., Liu, B., and Guan, J. All in one: Multi-task prompting for graph neural networks. In Proceedings of the 29th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 2120-2131, 2023b.
Wu, M., Zheng, X., Zhang, Q. Shen, X., Luo, X., Zhu, X., and Pan. S. Graph learning under distribution shifts: A comprehensive survey on domain adaptation, out- of offdistribution, and continual learning. arXiv preprint arXiv:2402.16377 2024a.
Sun, X., Zhang, J.. Wu, X., Cheng, H.. Xiong, Y., and Li, J. Graph prompt learning: A comprehensive survey and beyond. arXiv preprint arXiv:2311.16534 2023c.
Wu, Q., Zhang, H., Yan, J., and Wipf, D. Handling distribu- tion shifts on graphs: An invariance perspective. Interna- tional Conference on Learning Representations, 2022b.
Traud, A. L., Mucha, P. J., and Porter, M. A. Social structure of facebook networks. Physica A: Statistical Mechanics and its Applications, (91(16):4165 4165-444165-18888 2012.
Wu, T., Ren, H., Li, P., and Leskovec, J. Graph information bottleneck. Advances in Neural Information Processing Systems, 33:20437-20444, 2020b.
Tzeng, E., Hoffman, J., Saenko, K., and Darrell, T. Adver- sarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7167-7176, 2017.
Wu, W., Wang, Shen, D., Qin, C., Chen, L., and Xiong, H. Afdgef: Adaptive feature de-correlation graph collab- orative filtering for recommendations. In Proceedings of the 47th International ACM SIGIR Conference on Re- search and Development in Information Retrieval, pp. 1242-1252, 2024b.
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. Graph attention networks. arXiv preprint arXiv:/710 10903, 2017.
Wang. D., Shelhamer, F.,, Liu, S., Olshausen, B., and Darrell, T. Tent: Fully test-time adaptation by entropy minimiza- tion. In International Conference on Learning Represen- tations, 2021. URL https:/ / openrevie ew. net/ prrummid=ux13bb220
Xiao, J., Dai, Q., Xie, X., Dou, Q., Kwok, K.-W,, and Lam, J. Domain adaptive graph infomax via conditional adversarial networks. IEEE Transactions on Network Science and Engineering, 2022.
Xu, Y., Zhang, Y., Guo, W., Guo, H., Tang, R., and Coates, M. Graphsail: Graph structure aware incremental learn- ing for recommender systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 2861-2863 2020.
Wang, L., Zhang, X., Su, H., and Zhu, J. A comprehen- sive survey of continual learning: theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024a.

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
Zhang, P., Yan, Y., Li, C., Wang, S., Xie, X., Song, G., and Kim, S. Continual learning on dynamic graphs via param- eter isolation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 601-611, 2023a.
Zhang, W., Ouyang, W., Li, W., and Xu, D. Collaborative and adversarial network for unsupervised domain adapta- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3801-3809, 2018.
Zhang, X.,
Song, D., and Tao. D. Hierarchical prototype net- works for continual graph representation learning. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 45(4))4622-4633 2022a.
Zhang, X., Song, D., and Tao, D. Sparsified subgraph memory for continual graph representation learning. In 2022 IEEE International Conference on Data Mining (ICDM), pp. 1335-1340. IEEE, 2022b.
Zhang, X., Song, D., and Tao, D. Ricci curvature-based graph sparsification for continual graph representation learning. IEEE Transactions on Neural Networks and Learning Systems, 2023b.
Zhang. X., Song, D., Chen, Y., and Tao, D. Topology-aware embedding memory for learning expanding graphs. arXiv preprint arXiv:2401.13200 2024.
Zhao, H., Chen, A., Sun, X., Cheng. H., and Li, All one and one for all: A simple yet effective method towards cross-domain graph pretraining. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4443 4454, 2024.
Zheng, X., Zhang, M., Chen, C., Nguyen, Q V. H., Zhu, X., and Pan, S. Structure-free graph condensation: From large-scale graphs to condensed graph-free data. Ad- vances in Neural Information Processing Systems, 36, 2024.
Zhou, F. and Cao, C. Overcoming catastrophic forgetting in graph neural networks with experience replay. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 4714-4722 2021.
12

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
A. Proof of Theorem 3.1.
In the section Variational Memory Graph Generation, we give Theorem to define lower bound of the information bottleneck for generating memory graphs G, from the original graph G₁:
Theorem A.I. Let Gt be generated graph conditioned the lalent representation Ze of the original graph Gz. Suppose Q(G)) is variational approximation of the true posterior P(G.), Then, the following lower bound the optimization objective for & holds:
(
Elog Pу ATKKI 44) QI PE[log (Eallog(Pa(Ciie (g Z.)l
(16)
Here, KL(- || indicates the Kullback Leibler divergence. Pj is considered as the classifier f(O)).
Here provide the proof of Theorem 3.1:
Proof. We start by decomposing the objective function ((Φ) in 4 and using variational approximations and known inequalities to make the optimization tractable.
First, the mutual information term I(G;;; Y2) quantifies the information shared between G. and Ye, which is defined as:
P(V]C..)
I( V.) = E, log
(17)
P(
Expanding and simplifying this using the definitions of expectation and entropy, we get the following:
T(Grs P,) Fa [lg G(V.).)) Ey, log P(Y))
Ba log G())) (())
(18)
R. llog Py (Y,(.)
The inequality holds because H (Ý.)), the entropy of Y, is always non non-negative.
Second, the mutual information I(Gr: G1, Z+) measures the amount information gained about G, by observing G, and Z. defined as:
P(G 2z) P(G.)
T(G,; G.. Zi) = G..2.
log
(19)
We introduce variational approximation Q(G.) to the true posterior P(Gx). Then, the negative mutual information is defined as:
P(G. 2x)
-I(G,; G., Z,) = = =- G.,
log Q(G.)
+ |
P(G.(G,,
(20)
2 2-G log Q(G.)
= EEK, 2,) || Q(G.))
Third, the conditional mutual information I(G;; Gt) quantifies the additional information about G, obtained from Z, given Gr. It is defined by the equation:
P(G,,ZIG.)
(GG; = Ea.2....
log P(Ê, G.).G.))
log P(,,
Ea.z..
P(G./G.)
(21)
-EG,2,0
log P(,, Gi) + +H(G.G)..)
log P(,, 2. G.)
2 EE,.z.G.
13

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
The inequality holds because the entropy H[,, (G,) is always non-negative.
By combining these results, we can derive the lower bound for the objective [()):
[ 2 Elog -EIKL:))-PE 2,) Q(Ê.))) + pEllog(P,(G.IGe Z))]
(22)
This completes the proof.
B. Detialed Experimental Setup
B.1. Datasets
Our paper involves two primary categories of graph datasets, Regional Shifts and Temporal Shifts, utilizing continual learning principles to effectively manage adaptations across different regions and temporal variations. The datasets used include Facebook-100(TI
et al., 2021), OGB-Arxiv(Hu et al., 2020), and EllipticcParrea et al., 2020). The statistical details of the datasets are shown in Table
et al., 2012). Wrilcc-ExxpleiiRRoembeerre
Regional Shifts: The Facebook- dataset comprises 100 snapshots of Facehook friendship networks from 2005, each representing users from a specific American university. These networks vary greatly in size, density, and degree distribution. Additionally, the Exitch-Explicit dataset includes seven networks, where nodes are Twitch users and edges denote mutual friendships. Each network originates from one of the following regions: DE, ENGB, ES, FR, PTBR, RU, and TW.
The OGB-Arxiv dataset contains 169,343 Arxiv CS papers from 40 subject areas, detailing their citation relationships, and suitable for analyzing the evolution of scientific collaboration networks. The Elliptic dataset includes 49 sequential graph snapshots of Bitcoin transaction networks, where nodes represent transactions and edges represent payment flows. Around 20% of transactions are labeled licit illicit, with the objective of detecting future illegal transactions.
Temporal Shifts:
B.2. Baselines
We evaluate the performance of our continual adaptive learning framework against diverse set of baseline methods. Test employs a pretrained graph model to perform direct inference on the target dataset without any adaptation, serving the lower bound. DANN (Ganin et al., 2016), traditional domain adaptation method, utilizes the source graph and adversarial training to minimize domain discrepancies at each step. The category "No Rehearsal Based Test- Time Adaptation" " comprises one-step test-time adaptation methods. Tent (Wang et al., 2021), which minimizes entropy of test samples; BN Stats Adapt (Li et al., 2016), which adjusts network weights and Batch Normalization statistics based on current input data for prediction; EERM (Wu et al., 2022b), tailored for graph datasets, maximizing risk variance to manage domain shifts and out-of-distribution challenges; and GTRANS (Jin al., 2022), enhancing test-time performance by refining graph structure and node features through contrastive surrogate loss. The category "Continual Test Time Adaptation" is continual test-time training methods, including CoTTA and EATA. CoTTA (Wang et al., 2022) combines weight-averaged predictions with partial neuron restoration to mitigate error accumulation and catastrophic forgetting. EATA (Niu et al., 2022) enhances adaptation efficiency through entropy minimization and employs Fisher-based regularizer to maintain performance across domain shifts. For baselines not originally designed for graphs, their architectures have been adapted to GCNs to ensure consistency evaluation.
B.3. Experimental Setting
To effectively evaluate our model's adaptability across varying domains and over time, we have introduced a continual adaptive learning framework across evolving graph data. For academic and social networks, initial training is conducted on three university graphs from the Facebook- 100 dataset: Amherst41, Caltech36, and Johns Hopkins55. We continually extend the domain adaptation the remaining eleven university graphs without labels. This data selection sequence requires the model to handle different graph structures from training/validation to testing data(Wu et al., 2022b). For Twitch-explicit, the model is initially trained on the DE network and later tested for adaptability across regional networks, including ENGB, ES, FR, PTBR, RU, and TW, also without labels. In addressing temporal shifts, the OGB-Arxiv dataset is employed, using data prior to 2011 to pretrain the model, while data from 2011 and later is used for continual adaptation, exposing the model to evolving scientific collaborations. For the Elliptic dataset, we utilize snapshots 7 through 9 for initial training, avoiding the first six due to their extreme class imbalance, with the remaining data employed for continual adaptation. This approach
14

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
Spet
(a) Twitch-ENGB
(b) Twitch-ES
(c) Twitch-FF
(d) Twitch-PTBR
(e) Twitch-Ri -RU
(f) Twitch-TW
Figure 6. Visualized comparison of the original graphs (the first line) and generative graphs (the second line) of GCAL in the Twitch dataset.
reflects the dynamic and challenging nature of financial transactions. Our training strategy begins by pretraining the model on selected graphs from each dataset, then continually adapting to the remaining unlabeled graph datasets in an online manner.
We use a 2-layer GCN as the backbone for three datasets, except for OGB-Arxiv, where we use GraphSAGE (Hamilton et al., 2017). The training, validation, and test rates for pretrain datasets are set at 60%. 20%, and 20% respectively. During the train and adapt periods, the learning rates and weight decays are set as follows: training, and Ir 0.001 and wd x 10-4 for adaptation. The number of epochs for pre-training set between 100 and 200, while the adaptation phase involves relatively smaller number of epochs, ranging from 1 to 10, for these four datasets. settings are provided in the accompanying code. For the evaluation metric,
0.0001 and wd x 10 for
The detailed dyper-parameee accuracy matrix Mace E T which is a lower triangular matrix where Macc.á.j (for 2 j) represents the accuracy on the domain after training on the domain i. Specifically, similar to (Jin et al., 2022; Wu et al., 2022b), for the Twitch-Explieit and Facebook- 100 datasets, the results are measured using ROC-AUC and Accuracy, respectively. For the Elliptic dataset, the metric used the F1 Score, while for the OGB-Arxiv dataset, Accuracy is used. To compute single numeric value upon completing all domains, we calculate the Average Performance (AP) as Σi= Mff,, primarily assessing adaptation ability, and the Average Forgetting (AF) as 7ー1 E I (Mrs M. primarily evaluating the ability to avoid forgetting. Fach experiment is repeated five times, with results reported as the mean and standard deviation.
present the
C. Additional Experiments
C.1. Visualization of Generated Memory Graph.
This experiment aims to answer: How do the structure of memory graphs generated by GCAL look like compared with the original ones? To demonstrate the effectiivnnes of the memory graphs we generated, we conducted an experiment to visualize the graph structures. The experimental results are shown in Figure 6, where we used the networks python library as tool on the Twitch dataset to display the generated effects on six continuous domain graphs. The top row shows the original graphs, while the bottom row displays the generated graphs. From this, we can observe that (1) the generated memory graphs significantly reduce the number and density of the graphs, making them more lightweight, and (2) the structure of the generated graphs is not random but shows a coherent structure, verifying the reliability and authenticity of the generated graphs. This visualization confirms the capability of GCAL to produce streamlined yet structurally meaningful graphs.
C.2. Performance Matrices.
To present a more fine-gran mne-grained demonstration of the model's performance in continual adaptive learning on graphs, we analyzed the average performance across all previously encountered domains each time a new domain was learned. We have visualized the performance matrix of the Twitch and Elliptic datasets in Figure. Here, we further report the results
15

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
Domains
coc
xen
12
Domains
(a) Facebook-100-CoTT7
(b) Facebook-100-GCAM.
ogbn-arxiv-CoTTA
(d) ogbn-arxiv-GCAM -CAL
Figure 7. Performance matrices of GCAL and CoTTA in different datasets.
Austage
GCN
65A22
307
6h.
623
GSAGE
GIN
6Gv
GG4G:
GAT
GIM
60.
SCAGE
6.7
SN
(a) explicit
(b) Facebook-
Elliptic
(d) OGB-Arxiv
Figure 8. The model performance with different GNN backhones.
of the Facebook and Ogbn-arxiv datasets Figure 7. In these matrices, each row represents the performance across all domains upon learning a new one, while each column captures the evolving performance specific domain as domains learned sequentially. In the visual representation, darker shades signify better performance, while lighter hues indicate inferior outcomes. GCAL predominantly displays lighter shades across the majority of blocks compared to CoTTA in Figure which has similar experimental phenomena before, further proving the effectiveness of GCAL in the continual graph model adaptation problem.
C.3. Backbone Analysis.
This experiment aims to answer: How do different GNN backbones compare within GCrL continual adaptation? We select representative GNN models, including GCN, GSAGE, GAT, and GIN, These GNN models serve the foundational backbones for integrating with GCAL. The results are presented in Figure 8. We compare the results of incorporating these GNN backbones within our framework versus utilizing them individually as standalone models. Using our framework demonstrates remarkable enhancements, showing the effectiveness of our proposed techniques. Lastly, important to note that the consistent use of different backbones significantly enhances results, thereby demonstrating the robustness and adaptability of our proposed method across various datasets. This is in contrast to the Test method, which directly infers from subsequent data without continuous domain adaptation. For Facebook- -00, the GAT model encounters Out Of Memory (OOM) issue and is therefore not included in the backbone comparison for this dataset.
D. Related Work
D.1. Graph Continual Learning.
Existing graph continual learning (GCL) methodologies are typically divided into three main categories: regularization, parameter isolation, and memory replay approaches. Regularization based methods primarily aim preserve parameters crucial to previous tasks, thereby minimizing disruptions (Cai et al., 2022; Liu al., 2021; Sun et al., 2023a; Xu al., 2020). Examples include topology-awan aware weight preserving (TWP) (Liu al., 2021) and RieGrace (Sun et al., 2023a), which focus on maintaining essential parameters and structural topologies. Parameter isolation techniques allocate distinct parameters for new tasks to maintain those relevant to prior tasks (Niu et al.; Zhang et al., 2023a; 2022a), as seen in HPNs (Zhang et al., 2022a). contrast, memory replay strategies(Li et al., 2024; Qiao et al., 2025) archive and revisit representative data from past tasks to alleviate the critical issue catastrophic forgetting, as exemplified by ER-GNN (Zhou & Cao, 2021), SSM (Zhang et al., 2022b), SEM-curvature (Zhang et al., 2023b), PDGNNs (Zhang et al., 2024), and CaT (Liu et al., 2023b).
16

---

GCAL: Adapting Graph Models to Evolving Domain Shifts
GCL has garnered increasing interest due to its practical applications, with each approach offering distinct strategies for handling task progression in graph-based models(Wu al., 2024a). Our approach, which belongs to the memory replay category, uniquely preserves critical topological structures while minimizing memory usage, Notably, while existing GCL methods are confined to supervised learning settings, our work introduces an unsupervised approach to graph continual learning, marking a pioneering step in this direction.
D.2. Graph Domain Adaptation.
Unlike traditional domain adaptation, which typically assumes static target domain, Continual Domain Adaptation addresses evolving target data. Traditional methods, including Maximum Mean Discrepancy (MMD) (Dziugaite et al., 2015) and adversarial techniques (Dan et al., 2024; Qiao et al., 2023; Tzeng et al., 2017; Zhang et al., 2018), form the basis of this field. graph-based domain adaptation, a variety of methods have been proposed (Ding et al., 2018; Jin et al., 2022; Liu al., 2023a; Ma al., 2019; Qiao et al., 2024; Wu al., 2022a; Xiao al., 2022). For instance, UDA-GCN (Wu et al., 2020a) and AdaGCN (Dai et al., 2022) leverage graph topology improve adaptability, reducing discrepancies between source and target graphs via local and global consistencies and graph domain discriminated loss, respectively. Continual Test-Time Adaptation (CTTA), a critical facet of Continual Domain Adaptation, addresses the unique demands of non non-static domains. Unlike traditional Time Adaptation (TTA), CTTA incorporates advanced strategies such as bi-average pseudo labels and stochastic weight resets, as implemented in CoTTA (Wang et al., 2022). Innovations like VDP (Gan et al., 2023), with visual domain prompts to counter error accumulation, and RMT (Döbler et al., 2023), which employs symmetric cross-entropy for enhanced robustness, further refine CTTA. Additional strategies, including entropy minimization by Tent and EATA (Niu et al., 2022) and meta-networks in ECoTTA (Song et al., 2023), contribute improved model normalization and adaptability. Despite these advancements, challenges such noisy pseudo-labels and calibration issues persist, and notable gap remains in unsupervised graph continual domain adaptation research.
D.3. Graph Condensation
Graph condensation(Sun et al.) has become increasingly prominent for its ability to create compact synthetic datasets that closely approximate the performance of full datasets (Hashemi et al., 2024). Classical techniques like GCond (Jin et al., 2021) and MCond (Gao et al., 2024) utilize gradient alignment to synthesize representative samples that maintain the statistical properties of the original data, drawing from principles of traditional sampling (Sener & Savarese, 2017). Among recent innovations, GCDM (Liu et al., 2022), introduce graph-specific distribution alignment to enhance condensation effectiveness. SFGC (Zheng et al., 2024) further refines this by distilling large graphs into structure-free node sets using meta-matching and dynamic feature scoring, resulting compact, highly generalizable data. Additionally, techniques such as CaT (Liu et al., 2023b) and PUMA (Liu et al., 2023c) extend graph condensation continual learning, demonstrating these methods' adaptability for dynamic graph-based applications. In parallel, Graph prompt learning based mSth et al., 2023b;c; Wang et al., 2024b; Zhao et al., 2024) also condenses knowledge into a compact graph-base based prompt, which is typically used to fine-tune pre-trained graph models on specific downstream tasks. However, most of these method rely on supervised signals for knowledge condensation. In contrast, our work addresses a more challenging task by condensing graphs into memory an unsupervised manner.
17


Reviews:
[{"review_id": "fgilJaw5uA", "reviewer": "Reviewer_GL4u", "summary": "This paper proposes Graph Adaptive Continual Learning (GCAL), extending the graph domain adaptation from single-step adaptation to continuous adaptation over a sequence of multiple domains. The proposed GCAL adopts a bi-level optimization strategy and consists of two phases. The adapt phase fine-tunes the given graph model on new graph domains based on information maximization, and the generate memory phase condenses the original graphs into memories, which will be used in future adapt phases to avoid forgetting.\n\nThe proposed method is evaluated on 4 public datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **W1: For task construction, it is unclear how the adopted datasets are constructed into different tasks with different distributions.**  \n**Q1: How are the datasets constructed into different domains, and how to ensure that the different domains have different distribution?**\n\nWe appreciate the reviewer’s valuable feedback on our task construction. In our study, we selected datasets from established benchmarks widely used in research on graph out-of-distribution generalization [1,2]. **Each dataset comprises multiple real-world graphs, with each graph considered an independent domain**. We organize these graphs sequentially, based on regional differences and temporal shifts, to construct a continual adaptation setting.\n\nAppendix B.1 and B.3 provide a comprehensive explanation of dataset construction and partitioning. Specifically, Facebook-100 dataset consists of 100 separate Facebook friendship networks, each representing a distinct American university. Twitch-Explicit dataset includes seven networks on Twitch, sourced from different regions, such as France, Germany, and Russia. In these two datasets, nodes represent users, and edges denote friendships. OGB-Arxiv dataset encompasses 169,343 Arxiv CS papers across 40 subject areas to construct their citation networks, with the graphs divided by publication years. Elliptic dataset features 49 sequential graph snapshots of Bitcoin transaction networks, which are evenly spaced with an interval of about two weeks, where nodes represent individual transactions and edges indicate the flow of payments. For each dataset, we selected multiple former source domains for pre-training and selected the remaining graphs as sequential target domains to facilitate continual adaptation under varying distributions.\n\nThese datasets are widely recognized for exhibiting distribution discrepancies between their graphs in the literature. From Figure 1, we also have **empirical evidence** supporting that the graphs indeed have different distributions. \n\nThe links for downloading the datasets and the code for dataset processing are provided in the anonymous link in the Abstract to ensure the reproducibility of our study.\n\n[1] Wu Q, et al. Handling Distribution Shifts on Graphs: An Invariance Perspective, ICLR 2022\n[2] Jin W, et al. Empowering Graph Representation Learning with Test-Time Graph Transformation, ICLR 2023\n\n\n> **W2: In Table 2, most methods have a similar performance with Test, which is the lower bound; this is weird.**\n\nWe appreciate the reviewer’s insightful comment. To clarify this point, the metric we used, \"Average Performance\" (AP) and \"Average Forgetting\" (AF), assesses model performance across all previously encountered domains. The phenomenon that many baseline methods exhibit similar performance to the \"Test\" occurs because common adaptation methods do not fully address the continual adaptation problem. They tend to experience substantial forgetting when adapting to new tasks. As these models continually update their parameters, they easily forget previously learned tasks, significantly reducing overall performance. Conversely, the \"Test\" method does not update or fine-tune its model parameters. This lack of adaptation makes stable, though still low, performance across all datasets. \n\nOur proposed method, GCAL, distinguishes itself by specifically addressing these challenges through memory replay and variational memory graph generation, which effectively retain and reuse previously learned information, enabling improved performance.\n\nThis phenomenon is **not unique** to our study and indeed exists in the broader field of continuous learning. For example, in a different continual learning setting on graphs—class-incremental graph learning[3], certain baselines perform similarly or even worse than the lower bound. This is because baseline methods often collapse quickly due to catastrophic forgetting when continuously learning new tasks.\n\n[3] Zhang, Xikun, Dongjin Song, and Dacheng Tao. \"Cglb: Benchmark tasks for continual graph learning.\" Advances in Neural Information Processing Systems 35 (2022): 13006-13021.\n\n> **Q2: What does #Nodes and #Egdes mean in Table 1? Why is it a range?**\n\nThanks for the valuable questions. \\#Nodes represents the number of nodes, and \\#Edges means the number of edges in each graph. Since each dataset contains multiple graphs, each graph exhibiting different distributions, the number of nodes and edges varies across these graphs. Therefore, we report the range (minimum and maximum values) of nodes and edges present within the graphs of each dataset. We will make it clear by introducing this explicitly in the caption of Table 1 in the final version.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "For task construction, it is unclear how the adopted datasets are constructed into different tasks with different distributions. \n\nIn Table 2, actually most methods have a similar performance with Test, which is the lower bound, this is weird.", "relation_to_broader_scientific_literature": "Continual graph domain adaptation learning is broadly related to different application scenarios involving evolving graph data.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n\nThe proposed method resolves a weakness of the existing domain adaptation works, which is the limitation to single-step adaptation. The targeted continual multi-domain adaptation is more practical in real-world applications.\n\nThe proposed generate memory phase is supported by theoretical analysis.\n\nThe proposed method outperforms the baselines on all datasets.\n\nWeakness:\n\nFor task construction, it is unclear how the adopted datasets are constructed into different tasks with different distributions. \n\nIn Table 2, most methods have a similar performance with Test, which is the lower bound; this is weird.", "other_comments_or_suggestions": "See above.", "questions_for_authors": "How are the datasets constructed into different domains, and how to ensure that the different domains have different distribution?\n\nWhat does #Nodes and #Egdes mean in Table 1? Why is it a range?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "D6m9TN4D7G", "reviewer": "Reviewer_izUY", "summary": "This paper proposes GCAL, a continual graph domain adaptation framework that mitigates catastrophic forgetting through bilevel optimization, integrating information maximization for adaptation and variational memory graph generation for knowledge replay. The approach is theoretically grounded in information bottleneck theory. Extensive experiments on multiple datasets demonstrate that GCAL outperforms baselines.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **W1: The memory replay framework is commonly used in continual learning research. This paper does not introduce a fundamentally new framework in this regard.**\n\nWe acknowledge that memory replay is indeed a well-known approach to continual learning. We would like to clarify that **our novelty specifically lies in how the replay mechanism is integrated within the context of evolving out-of-distribution graphs**. Unlike existing methods that rely on stored raw data or labeled samples for replay, GCAL introduces a variational information bottleneck-based graph generator that creates synthetic memory graphs. This component is theoretically grounded (Eq. 3–5 in Sec. 3.2.1) and uniquely capable of generating compact, informative, and generalizable memory graphs in an unsupervised manner—a critical advancement where labeled data is not available. \n\nFurthermore, existing continual learning replay techniques primarily target Euclidean data formats. Our design includes graph condensation, Gumbel-softmax-based differentiable edge sampling, and gradient-matching-based memory optimization, which are specifically tailored for graph topologies, accounting for both structural and feature-level preservation (Sec. 3.2.2–3.2.5).\n\n> **W2: This paper uses an adaptation learning objective to condense the graphs into memory graphs. The adaptation loss may not adequately capture the necessary structural or semantic information required for high-quality condensation.**\n\nWe appreciate this valuable observation regarding the sufficiency of the adaptation loss for graph condensation. We would like to clarify that the adaptation loss is only one component of a multi-objective memory graph learning strategy in GCAL. In particular, **the quality and informativeness of the memory graphs are ensured combination of three specialized losses with theoretical grounding**. We leverage the information bottleneck principle to derive three loss functions, each designed to explicitly preserve structural and semantic characteristics of the original graphs.\n\nAs detailed in Sections 3.2.3–3.2.5 of the paper, the generation of memory graphs is not solely guided by the adaptation loss. The condensation loss $L_{MGL}$ leverages gradient matching to ensure the generated memory graphs induce similar optimization trajectories (gradients) as the original graphs. Rooted in variational inference (Eq. 12–13), this KL divergence-based term controls the latent distribution and promotes stability and informativeness in node and edge generation, avoiding overfitting to spurious patterns. The generation loss (Eq. 14) minimizes the distributional discrepancy between the memory graph and the original graph in the model’s latent space.\n\n\n> **W3: The multiple learning objectives add to the complexity of this method. The time complexity of this method should be provided.**\n\nFor time complexity, we use GCNs as the backbone. The propagation cost is $O(L N_t d h + L N_t h^2)$, where $L$ is the number of layers, $N_t$ is the number of nodes, $d$ is the average degree, and $h$ is the hidden dimension. For the memory graph generation part, the TopKselector involving a sorting costs $O(N_t\\log K+  N_t h)$, the construction and reparameterization in Eq.7,8 costs $O(K h)$ and $K^2 h$. The loss computations in Eq. 10, 13, and 14 cost $O(L h^2)$, $O(Kh +K^2)$, and $O(N_th + Kh)$. Because $K<<N_t$, the computation cost of the memory graph generator and subsequent losses is **significantly less** than the propagation of the GNN backbone, indicating the efficiency of our method.\n\n\n> **W4: Some of the latest literature for graph domain adaptation is not included.**\n\nThank you for the valuable observation. We will update our related work section to incorporate the latest literature, ensuring a more comprehensive overview of graph domain adaptation methods.", "claims_and_evidence": "Overall, the paper presents well-supported claims with theoretical support and superior experimental performance.", "methods_and_evaluation_criteria": "The proposed GCAL framework and evaluation criteria are well-aligned with the problem of continual graph domain adaptation", "theoretical_claims": "The paper presents a theoretical lower bound based on the information bottleneck theory to support memory graph generation.", "experimental_designs_or_analyses": "The experimental designs appear sound. But a time complexity analysis would be helpful.", "relation_to_broader_scientific_literature": "GCAL contributes a novel framework for continual graph domain adaptation, related to the research topics of continual graph learning, domain adaptation, and graph condensation.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths\n\n1. The studied problem is new and practical. \n\n2. The proposed model is supported by a sound theoretical foundation based on information bottleneck theory.\n\n2. The paper introduces a novel variational memory graph generation method for graph continual domain adaptation.\n\n\nWeaknesses\n\n1. The memory replay framework is commonly used in continual learning research. This paper does not introduce a fundamentally new framework in this regard.\n\n2. This paper uses an adaptation learning objective to condense the graphs into memory graphs. The adaptation loss may not adequately capture the necessary structural or semantic information required for high-quality condensation.\n\n3. The multiple learning objectives add to the complexity of this method. The time complexity of this method should be provided.\n\n4. Some of the latest literature for graph domain adaptation is not included.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "See the weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "U0xhmmAL70", "reviewer": "Reviewer_YBfY", "summary": "This paper introduces **Graph Continual Adaptive Learning (GCAL)**, a novel framework for continual domain adaptation in graph models, specifically addressing challenges in adapting to multiple out-of-distribution (OOD) graph shifts. The method employs a bilevel optimization strategy with two phases: (1) **Adaptation**, using information maximization for self-supervised adaptation while mitigating catastrophic forgetting via memory replay, and (2) **Memory Generation**, utilizing a variational memory graph generation module based on an information bottleneck lower bound. The paper demonstrates through extensive experiments that GCAL outperforms existing state-of-the-art methods in continual graph adaptation.\n\n## update after rebuttal\n\nThanks the authors for their rebuttal, I keep my score unchanged.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **Computational Efficiency**\n\nThank you for your valuable feedback. Our approach leverages a variational-based generation strategy, which is inherently designed to be efficient and scalable. This strategy allows for effective memory graph generation without significantly increasing the computational complexity. Thus, as the **time complexity analysis** in the Response to Reviewer `izUY` shows, the computation cost of the memory graph generator and subsequent losses is significantly less than the propagation of the GNN backbone, indicating the efficiency of our method.\n\nFollowing your suggestion, we have conducted extensive experiments to compare the running times of GCAL with CoTTA and GTrans training across four datasets on four 4090 GPUs. The results of these experiments are summarized in the following table. \n\n| Time / Seconds | CoTTA | GTrans | GCAL |\n| --- | --- | --- | --- |\n| Twitch | 41.6537 | 34.2958 | 26.842674 |\n| FB100 | 156.3651 | OOM | 41.446378 |\n| Elliptic | 26.5828 | 32.1564 | 22.967516 |\n| OGB-Arxiv | 44.4698 | 38.3925 | 40.146278 |\n\nThese results demonstrate that GCAL not only operates within a competitive time frame but also significantly outperforms the baseline methods on multiple datasets. This evidence supports GCAL's effectiveness in this task.\n\nRegarding the scale of the datasets, the choice of these four datasets is aligned with the **established benchmarks** in studies related to out-of-distribution graph generalization [1,2]. We would like to clarify that the numbers of nodes and edges, as indicated in Table 1, represent the range within each dataset, where each dataset comprises multiple graphs. The overall number of nodes and edges across these datasets is indeed **substantial**. For example, Elliptic comprises a total of $189,033$ nodes and $217,223$ edges, and Facebook-100 consists of $157,921$ nodes and $13,197,698$ edges. Thus, these datasets meet the large-scale criteria to some extent. We hope this response adequately addresses your concerns.\n\n[1] Wu Q, et al. Handling Distribution Shifts on Graphs: An Invariance Perspective, ICLR 2022\n\n[2] Jin W, et al. Empowering Graph Representation Learning with Test-Time Graph Transformation, ICLR 2023\n\n> **Comparison to heuristic-based memory selection**\n\nOur approach in the unsupervised and out-of-distribution setting **diverges** from traditional continual learning frameworks, which typically assume that labels are provided and focus on adapting models to incremental classes or tasks. Consequently, many memory selection methods that rely on label information in standard continual learning settings are not applicable to our context. \n\nFollowing your suggestion, we have used a widely recognized heuristic-based memory selection method, K-Center[3], into our framework for comparative evaluation. This method selects K-representative data points as centers without the need for labels. The comparative results are in the table below:\n\n| Method | Twitch | FB100 | Elliptic | OGB-Arxiv |\n|--------|--------|-------|----------|-----------|\n| K-Center | 54.74±0.25 | 51.88±0.28 | 54.37±0.26 | 43.18±0.38 |\n| GCAL | 55.65±0.09 | 52.72±0.36 | 56.57±0.14 | 45.22±0.17 |\n\nThe results show that our variational memory graph generation method overall outperforms K-Center in our framework, demonstrating its effectiveness in learning meaningful graph memory. \n\n[3] Nguyen, Cuong V., et al. \"Variational Continual Learning.\"  ICLR 2018.\n\n> **Evaluation of Synthetic Feature or Structural Distribution Shifts**\n\nThank you for your thoughtful feedback. The datasets used in our experiments inherently **involve both feature and structural shifts** across domains. For instance, social networks from different universities Facebook-100 exhibit variations in node attributes (e.g., user demographics) and structural patterns (e.g., friendship density). Citation networks OGB-Arxiv evolve over time, with node features (e.g., paper topics) and citation structures changing as research trends progress. These **real-world shifts** align with the challenges GCAL aims to address, validating its ability to adapt to combined feature and structural distribution shifts.\n\nSynthetic perturbations (e.g., Gaussian noise, edge rewiring) usually are more valuable for testing the robustness of graph neural networks, like against adversarial attacks. Our focus is on addressing practical, real-world OOD challenges within the continual adaptation framework using established benchmarks.\n\nThank you again for your constructive feedback. Due to the time limitation, we will explore this direction in future research.\n\n\n\n\n> **Additional References**\n\nThank you for your valuable feedback. As suggested, we will include a discussion of graph prompting techniques in Related Works, incorporating your highlighted references to provide a more comprehensive literature review.", "claims_and_evidence": "Overall, the submission provides substantial empirical and theoretical support for its claims. However, there are a few areas where the claims could be better substantiated or require additional clarification. \n\nSome weakly supported claims:  \n1. **GCAL is efficient for continual adaptation in large-scale graphs.**  \n   - The paper does **not** provide computational complexity analysis or runtime benchmarks comparing GCAL to existing approaches.  \n   - Since bilevel optimization and variational memory graph generation introduce **additional computational overhead**, the authors should provide evidence on **training/inference time**, particularly for large-scale graphs.  \n   - Suggested improvement: Include runtime comparisons against CoTTA, EERM, or GTrans to demonstrate computational feasibility.  \n\n2. **Variational memory graph generation leads to significantly better knowledge retention.**  \n   - While the **ablation study** confirms that removing memory generation reduces performance, it is unclear **how much variational memory generation improves over simpler alternatives** (e.g., naive replay of stored subgraphs).  \n   - Suggested improvement: Compare GCAL’s memory generation against a **simpler heuristic-based memory selection** to isolate the exact benefits of the variational approach.  \n\n3. **GCAL can generalize well across different types of OOD shifts.**  \n   - The paper only evaluates **two types of shifts** (regional and temporal), which, while useful, do not fully represent all real-world graph distribution shifts (e.g., feature shifts, structural perturbations).  \n   - Suggested improvement: Add experiments on **synthetically perturbed graphs** to test robustness against **node feature corruption, edge rewiring, or adversarial attacks**.  \n\n\nMost claims in the paper are well-supported with empirical results and theoretical justification. Addressing the above gaps would make the claims more robust and convincing.", "methods_and_evaluation_criteria": "The Methods and Evaluation Criteria Largely Make Sense. Some Areas Need More Justification or Alternative Evaluations\n\n1. **Lack of Large-Scale Graph Evaluation**  \n   - The datasets used (Twitch, Facebook-100, OGB-Arxiv, Elliptic) have **relatively moderate-scale graphs** (up to hundreds of thousands of edges).  \n   - **Real-world continual graph adaptation problems (e.g., social media networks, citation networks, e-commerce graphs) often involve millions of nodes and edges.**  \n   - **Suggestion:** Evaluate GCAL on **larger-scale dynamic graphs** such as:  \n     - **Reddit (social interactions, time-evolving)**\n     - **MAG-Scholar (large citation network)**\n     - **Amazon/Alibaba (e-commerce graphs, evolving product-user interactions)**  \n\n2. **Computational Efficiency Not Evaluated**  \n   - **Bilevel optimization and variational memory graph generation** add complexity.  \n   - The paper does **not** provide a **runtime analysis** or **memory usage comparison** against baselines.  \n   - **Suggestion:** Report **training time, inference time, and memory footprint** compared to simpler adaptation methods (e.g., CoTTA, GTrans).  \n\n3. **No Evaluation on Feature or Structural Distribution Shifts**  \n   - The datasets primarily evaluate **temporal and regional shifts**, but in real-world applications, **feature shifts** (e.g., node attribute changes) and **structural shifts** (e.g., edge rewiring, node insertion/deletion) are common.  \n   -  **Suggestion:** Test GCAL on **synthetic or adversarial perturbations** to evaluate robustness under feature and structure shifts.  \n     - **Perturb node features (e.g., Gaussian noise, dropout).**  \n     - **Rewire graph structures (e.g., edge deletion/addition, graph sparsification).**", "theoretical_claims": "I did not check the correctness of their theoretical proofs and I assume they are all correct.", "experimental_designs_or_analyses": "I carefully read their experimental analysis and I think their analysis is reasonable", "relation_to_broader_scientific_literature": "Related Prior Work like graph prompt learning [1-4] explores prompting techniques for cross-task generalization in GNNs. These methods suggest that prompt-based approaches can enable few-shot adaptation to new graph distributions. While GCAL does not explicitly use graph prompting, its memory replay approach serves a similar function—storing distilled graph representations for future adaptation. It would be more solid if they could include such a discussion in their related work section [2]. and give a summary of future work like exploring combining GCAL’s memory generation with graph prompting techniques to improve adaptability [1,3,4].  \n- [1] All in One: Multi-task Prompting for Graph Neural Networks. KDD 2023. \n- [2] All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining. KDD2024. \n- [3] Graph Prompt Learning: A Comprehensive Survey and Beyond. https://arxiv.org/abs/2410.01635 \n- [4] Does Graph Prompt Work? A Data Operation Perspective with Theoretical Analysis. https://arxiv.org/abs/2410.01635", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2ejTzWHDMT", "reviewer": "Reviewer_Ufiz", "summary": "This paper introduces GCAL, a novel framework designed to address the challenge of continual domain adaptation in graph models, particularly in scenarios involving evolving, out-of-distribution graphs. GCAL employs a bilevel optimization strategy: the \"adapt\" phase fine-tunes the model on new graph domains while mitigating catastrophic forgetting through memory replay, and the \"generate memory\" phase condenses original graphs into smaller, informative memory graphs using a variational memory generator guided by information bottleneck theory. Extensive experiments on regional and temporal graph datasets demonstrate that GCAL outperforms state-of-the-art methods in adaptability and knowledge retention.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **W1. It appears that the label is not given in the adaptation process, however, the label Y is explicitly referenced in the theoretical analysis. More explanation about how the labels are eliminated in this process should be added.**\n\nWe appreciate the reviewer’s observation. Indeed, in our setting, true labels are not available during the adaptation process, as we focus on unsupervised continual adaptation to out-of-distribution graph domains. Thus, label variable $ \\hat{Y}_t$ appears in the theoretical formulation in Equation. 3–5 is addressed via soft pseudo-labels derived from the model‘s predictions. \n\nSpecifically, In Section 3.2.1 (Eq. 3–5), we derive a variational lower bound based on the graph information bottleneck:\n\n $L(\\Phi) =  \\max_{\\Phi}  \\left[I(\\widehat{G}_t;\\widehat{Y}_t) - \\beta I(\\widehat{G}_t;G_t,Z_t) + \\beta I(\\widehat{G}_t; Z_t|G_t)\\right].$\n\nHere in the first term, $ \\hat{Y}_t$ refers to the training signal associated with the memory graph $\\widehat{G}_t$, and is used to theoretically quantify task-relevant information retained during memory graph generation. Since true labels are not accessible during adaptation, we set $ \\hat{Y}_t$ with soft pseudo-labels generated via self-supervised information maximization in Equation 1. We have clarified this in Section 3.2.3, where the condensation loss is minimized using pseudo-label-based adaptation objectives.\n\nThis practice of using pseudo-labels has strong **precedents** in unsupervised domain adaptation and test-time training literature, such as Tent [1] and CoTTA [2]. We follow the same line of reasoning, adapting it to the graph domain.\n\n[1] Wang, Dequan, et al. \"Tent: Fully Test-Time Adaptation by Entropy Minimization.\" ICLR 2021\n\n[2] Wang, Qin, et al. \"Continual test-time domain adaptation.\" CVPR 2022.\n\n\n> **W2. It is not clear how the variational memory graph generator is superior to the traditional graph generation method.**\n\nThank you for raising this important point. We would like to clarify that the variational memory graph generator in GCAL is specifically designed to address the **unique requirements of continual domain adaptation in graphs**, which traditional graph generation methods do not fully support. \n\nConventional generators typically focus on reconstructing entire input graphs or producing realistic samples based on learned distributions. However, these methods are not optimized for the goals of continual learning—particularly the need to retain and condense task-relevant knowledge for replay across evolving domains. In contrast, our variational memory graph generator is grounded in the information bottleneck principle, which explicitly balances compression and relevance. It learns a variational latent representation of the input graph and selectively generates a small, condensed graph that captures only the most informative features and structural signals necessary for downstream prediction tasks. It can also generate diversified representations of memory graphs via variational reparameterization for generalized training.\n\nThe empirical results in Table 2 and the ablation study in Table 3 demonstrate that memory graphs generated by our variational method enable superior adaptation and retention performance compared to baseline approaches, even with a significantly reduced memory graph. \n\n> **W3. A deeper analysis of how the weights of losses $L_{Reg}$ and $L_{Gen}$ influence the continual adaptation process would be beneficial.**\n\nResponse: \nWe conduct a hyperparameter study on the loss weights $\\lambda_1$ and $\\lambda_2$ (initially set as 1,1), which control the two auxiliary losses---$L_{Reg}$ and $L_{Gen}$. The results are summarized in the table below:\n\n| L_Reg | L_Gen | Twitch | FB100 | Elliptic | OGB-Arxiv |\n| --- | --- | --- | --- | --- | --- |\n| 1 | 2 | 55.62±0.23 | 52.33±0.87 | 56.21±0.41 | 45.18±0.24 |\n| 1 | 3 | 55.56±0.15 | 52.48±0.11 | 56.17±0.38 | 45.12±0.23 |\n| 2 | 1 | 55.59±0.19 | 52.43±0.64 | 56.23±0.30 | 45.14±0.22 |\n| 3 | 1 | 55.55±0.18 | 52.69±0.38 | 56.27±0.23 | 45.15±0.26 |\n\nAcross all datasets and configurations, the performance remains relatively stable, with only minor variations (generally within ±0.1%–0.3%). This indicates that GCAL is robust to moderate fluctuations in the weighting of these auxiliary objectives.\n\n> **Q1. Why are the AF results of some baseline N/A in Table 2?**\n\nThank you for your inquiry. These results are marked as N/A because certain methods like EERM and GTrans operate by training a new set of parameters for each graph independently, without updating a model across multiple domains. Consequently, there is no shared memory or parameter set across tasks, thus eliminating the concept of \"forgetting\" as typically measured in continual learning scenarios. Thus, there is no way to measure their average forgetting.", "claims_and_evidence": "The claims made in the submission are generally well-supported by the theoretical grounding and experimental results.", "methods_and_evaluation_criteria": "Yes, the proposed methods and evaluation criteria in the paper are well-suited for the problem.", "theoretical_claims": "The proof of Theorem 3.1 is generally sound and applies information bottleneck theory and variational inference techniques to derive the lower bound.", "experimental_designs_or_analyses": "The experimental designs and analyses in the paper are generally sound and valid. However, the clarifications of why the AF results of some baselines are N/A in Table 2 should be provided.", "relation_to_broader_scientific_literature": "This paper is related to graph machine learning, domain adaptation, and continual learning. The paper’s approach makes contributions by addressing the gaps in handling evolving graph domains and mitigating catastrophic forgetting.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "S1. The paper is well-written, with clear problem formulation and illustrations. The motivation is well-justified and compelling.\n\nS2. The paper is grounded in a solid theoretical framework, leveraging information bottleneck theory to derive a lower bound for memory graph generation.\n\nS3: The experiments are extensive, showing the advanced performance of the proposed method compared to baselines.\n\n\nW1. It appears that the label is not given in the adaptation process, however, the label Y is explicitly referenced in the theoretical analysis. More explanation about how the labels are eliminated in this process should be added.\n\nW2. It is not clear how the variational memory graph generator is superior to the traditional graph generation method.\n\nW3. A deeper analysis of how the weights of losses $L_{Reg}$ and $L_{Gen}$ influence the continual adaptation process would be beneficial.", "other_comments_or_suggestions": "None.", "questions_for_authors": "Q1. Why are the AF results of some baseline N/A in Table 2?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: **Summary:** This paper proposes GCAL, a bilevel optimization framework for continual graph domain adaptation under evolving out-of-distribution (OOD) shifts. GCAL alternates between two phases: (1) an Adapt phase based on information maximization for updating model parameters across new domains while mitigating forgetting, and (2) a Generate Memory phase that compresses source graphs into compact memory graphs via variational generation guided by an information bottleneck objective. The method is extensively evaluated across four datasets exhibiting regional and temporal shifts, and shows consistent improvements over baselines in terms of adaptability and knowledge retention.

**Decision:** This paper received uniformly positive evaluations from all four reviewers, with two clear accept scores and two weak accepts. While some concerns were raised—such as the absence of runtime comparisons, limited novelty given existing continual learning techniques, etc.—the authors addressed these in detail. Given the overall positive reception, the clear technical contributions, and the comprehensive rebuttal, I recommend acceptance.
Decision: Accept (poster)