Can LLMs Alleviate Catastrophic Forgetting in Graph Continual Learning? A Systematic Study

## Abstract

Abstract Nowadays, real-world data, including graph-structured data, often arrives in a streaming manner, which means that learning systems need to continuously acquire new knowledge without forgetting previously learned information. Although substantial existing works attempt to address catastrophic forgetting in graph machine learning, they are all based on training from scratch with streaming data. With the rise of pretrained models, an increasing number of studies have leveraged their strong generalization ability for continual learning. Therefore, in this work, we attempt to answer whether large language models (LLMs) can mitigate catastrophic forgetting in Graph Continual Learning (GCL) . We first point out that current experimental setups for GCL have significant flaws, as the evaluation stage may lead to task ID leakage. Then, we evaluate the performance of LLMs in more realistic scenarios and find that even minor modifications can lead to outstanding results. Finally, based on extensive experiments, we propose a simple-yet-effective method, Sim ple G raph C ontinual L earning (SimGCL), that surpasses the previous state-of-the-art GNN-based baseline by around 20% under the rehearsal-free constraint. To facilitate reproducibility, we have developed an easy-to-use benchmark LLM4GCL for training and evaluating existing GCL methods. The code is available at https://github.com/ZhixunLEE/LLM4GCL .

## 1 Introduction

Graph Machine Learning (GML) has become a cornerstone of modern data science, enabling intelligent systems to analyze complex relational structures across domains such as social networks (Fan et al., 2019), financial networks (Han et al., 2025; Li et al., 2022; Yang et al., 2025), recommendation systems (Li et al., 2025b; Wu et al., 2022), and biological structures.(Chen et al., 2023; Wang et al., 2022). Currently, with the remarkable capability to model diverse graph structures and node attributes, Graph Neural Networks (GNNs) have emerged as a prevalent approach for GML tasks. However, most existing approaches assume a static setting where the entire dataset and label space are known in advance. In contrast, real-world graphs are dynamic in nature, as new nodes, edges, and even previously unseen classes may appear over time. This evolving structure introduces the need for Graph Continual Learning (GCL), where models must incrementally integrate new knowledge while retaining previously learned information. Although significant progress has been made in GCL, with numerous studies (Liu et al., 2021; 2023b; Niu et al., 2024a; Zhang et al., 2023a; 2022b; 2022c; b; Zhou & Cao, 2021), substantial challenges still remain unresolved:

- •
A Lack of investigation into the rationality of experimental setups in GCL. This paper focuses specifically on the more mainstream and challenging setting of Node-level Class-Incremental Learning (NCIL). Based on the scope of each task’s test set, we divide NCIL into two categories: *local testing* and *global testing*. Although numerous works have been proposed within each category, there is currently no study that analyzes the rationality of the setups.
- •
Existing methods are all trained from scratch on streaming graph-structure data. The great success of pretrained models like GPT (Achiam et al., 2023) and CLIP (Radford et al., 2021) in fields such as natural language processing and computer vision has provided new opportunities for continual learning. However, due to a lack of powerful foundation models in GML, existing baselines are all trained from scratch.
- •
No existing studies that evaluate the performance of LLM-based methods in GCL. Currently, LLM-based methods have achieved state-of-the-art performance in node classification tasks (Li et al., 2024b; Wu et al., 2025), significantly outperforming traditional GNN models. However, there is a lack of benchmark evaluations assessing the performance of these models in the GCL setting.

To fill these gaps, we systematically analyze the NCIL task and explore the opportunities for mitigating catastrophic forgetting in GCL via LLMs. Specifically, we first identify a flaw in the local testing setting of NCIL, where task ID leakage occurs. This causes class-incremental learning to degrade into task-incremental learning, significantly reducing the task’s difficulty. We employ an extremely simple method to predict the task ID with 100% accuracy, achieving a forget-free performance and matching the results of the previous SOTA model (Niu et al., 2024b). Subsequently, we systematically evaluate the performance of LLMs and Graph-enhanced LLMs (GLMs) in global testing, a more realistic and challenging setting. We find that by incorporating common techniques such as Parameter Efficient Fine-Tuning (PEFT) or prototype classifiers, it is possible to leverage the strong generalization ability of pretrained models to surpass existing baseline methods. Finally, based on extensive experiments, we propose a simple yet effective method called SimGCL (Simple Graph Continual Learning for short). Technically, to better incorporate graph structural knowledge, we design ego-graph-derived prompts for each node, which effectively capture textual and structural features of the nodes. Then, in the first session, we applied LoRA (Hu et al., 2022) to efficiently fine-tune the LLMs with instruction tuning, aiming to narrow the distribution gap between the specific dataset and the pretrained dataset. In the incremental sessions, we utilize a training-free prototype classifier for prediction, avoiding parameter updates caused by the arrival of new knowledge, which greatly alleviates catastrophic forgetting. Based on experiments in NCIL and Few-Shot Node-level Class-Incremental Learning (FSNCIL), our proposed SimGCL can significantly surpass the current SOTA models.

In summary, our main contributions are as follows:

- •
Analysis of the rationality of experimental setups. This paper is the first to analyze the flaws in certain experimental setups in GCL and prevent task ID leakage.
- •
Comprehensive benchmark. We propose LLM4GCL, the first comprehensive benchmark for LLMs on GCL. We integrate 9 LLM-based methods and 7 textual-attributed graphs for evaluations.
- •
Model design. We propose a simple yet effective model, SimGCL, which is able to surpass all existing baselines and achieve an absolute increase of nearly 20% on certain datasets.
- •
Easy-to-use platform. To facilitate future work and help researchers quickly use the latest models, we develop an easy-to-use open-source platform. We hope to encourage more researchers to explore the use of pretrained models to mitigate catastrophic forgetting in GCL. The code is available at: [https://github.com/ZhixunLEE/LLM4GCL](https://github.com/ZhixunLEE/LLM4GCL).

## 2 Formulations and Background

Text-attributed Graphs (TAGs).
In this benchmark, we evaluate the continual learning capabilities of LLMs through node classification tasks on Text-Attributed Graphs (TAGs) (Ma & Tang, 2021). Formally, a TAG is represented as $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathcal{R})$, where $\mathcal{V}$ is the set of $N$ nodes, $\mathcal{E}$ is the edge set, and $\mathcal{R}$ is the collection of textual descriptions associated with each node $v_{i}\in\mathcal{V}$. Traditionally, the textual attributes of nodes can be encoded into shallow embeddings as $\textbf{X}=[\mathbf{x}_{1},\mathbf{x}_{2},...,\mathbf{x}_{N}]\in\mathbb{R}^{N\times d}$, where $d$ is the embedding dimension. Such an approach is widely adopted in the graph neural network literature. In contrast, LLM-based approaches directly process the raw text. Given a TAG with labeled nodes $\mathcal{V}^{l}$, the node classification task aims to predict the labels of the unlabeled nodes $\mathcal{V}^{u}$.

Node-Level Class-Incremental Learning (NCIL).
NCIL represents a fundamental paradigm in GCL, addressing incremental node classification tasks in an expanding graph $\mathcal{G}$. As shown in Figure [1](https://arxiv.org/html/2505.18697v2#S2.F1), NCIL consists an ordered sequence of training tasks $\{\mathcal{T}_{s_{1}},...,\mathcal{T}_{s_{n}}\}$ where each task $\mathcal{T}_{s_{i}}=\{\mathcal{G}_{s_{i}},\mathcal{C}_{s_{i}},\mathcal{V}_{s_{i}}^{l}\}$ consists of a task subgraph $\mathcal{G}_{s_{i}}\subset\mathcal{G}$, a class set $\mathcal{C}_{s_{i}}$, and labeled nodes $\mathcal{V}_{s_{i}}^{l}$, with $\mathcal{G}_{s_{i}}\cap\mathcal{G}_{s_{j}}=\mathcal{C}_{s_{i}}\cap\mathcal{C}_{s_{j}}=\varnothing$ and $|\mathcal{C}_{s_{i}}|=|\mathcal{C}_{s_{j}}|$, $|\mathcal{V}^{l}_{s_{i}}|=|\mathcal{V}^{l}_{s_{j}}|$ for all $i\neq j$. In NCIL, the model is trained sequentially from $\mathcal{T}_{s_{1}}$ to $\mathcal{T}_{s_{n}}$. This paradigm well reflects the real-world applications, where new categories (e.g., new products for recommendation systems or novel research fields for citation networks) are continually incorporated into the graph in a streaming manner as the domain evolves.

Few-Shot Node-Level Class-Incremental Learning (FSNCIL).
Newly emerged categories always contain a few samples. Consequently, it is difficult to collect sufficient samples for incremental tasks that match the initial task’s scale, leading to the FSNCIL paradigm. Given an ordered sequence of training tasks $\{\mathcal{T}_{s_{0}},\mathcal{T}_{s_{1}},...,\mathcal{T}_{s_{n-1}}\}$ where $\mathcal{T}_{s_{0}}$ serves as the base session and subsequent $\mathcal{T}_{s_{i}}$ ($i\in\{1,2,\ldots,n-1\}$) represent incremental sessions, each $\mathcal{T}_{s_{i}}=\{\mathcal{G}_{s_{i}},\mathcal{C}_{s_{i}},\mathcal{V}_{s_{i}}^{l}\}$ satisfies $\mathcal{G}_{s_{i}}\cap\mathcal{G}_{s_{j}}=\mathcal{C}_{s_{i}}\cap\mathcal{C}_{s_{j}}=\varnothing$ for all $i\neq j$, and $\mathcal{G}_{s_{i}}\cap\mathcal{G}_{s_{0}}=\mathcal{C}_{s_{i}}\cap\mathcal{C}_{s_{0}}=\varnothing$. Distinct from NCIL, in FSNCIL, the base session contains both more samples and more classes than incremental sessions which follow a strict $m$-way $k$-shot paradigm, i.e., $|\mathcal{C}_{s_{0}}|\gg|\mathcal{C}_{s_{i}}|=|\mathcal{C}_{s_{j}}|=m$ and $|\mathcal{V}^{l}_{s_{0}}|\gg|\mathcal{V}^{l}_{s_{i}}|=|\mathcal{V}^{l}_{s_{j}}|=k$ for $1\leq i\leq j$(^1^11FSNCIL typically features more classes in its base session than NCIL. Conversely, NCIL includes more classes and samples than the incremental sessions of FSNCIL. For a direct comparison of these settings, see Appendix [A.1](https://arxiv.org/html/2505.18697v2#A1.SS1); detailed experimental configurations are provided in Appendix [A.3](https://arxiv.org/html/2505.18697v2#A1.SS3), Table [6](https://arxiv.org/html/2505.18697v2#A1.T6).). During training, models are pre-trained on the base session $\mathcal{T}_{s_{0}}$, followed by fine-tuning sequentially on the incremental session sequence $\mathcal{T}_{s_{1}}-\mathcal{T}_{s_{n-1}}$.

Figure: Figure 1: The settings of LLM4GCL. Upper: The training pipelines for NCIL (left) and FSNCIL (right), where the model is trained exclusively on the graph of the current task to predict its corresponding classes. Lower-left: Two testing methods: global testing, which uses the complete graph with inter-session edges, and local testing, which evaluates the model on each task-specific graph sequentially. Lower-right: The Task ID Leakage issue inherent in local testing, where the model can infer the task ID of a given test graph via pooling and prototype matching.
Refer to caption: x1.png

## 3 LLM4GCL : A Systematic Study of LLMs for GCL

### 3.1 Learning Scenarios

LLM4GCL supports both NCIL and FSNCIL paradigms. Prior to this benchmark, existing GCL methods prevalently adopted the evaluation metrics from CGLB (Zhang et al., 2022a), which we refer to as the *local testing*. Specifically, when the $i$-the task $\mathcal{T}_{s_{i}}$ has been trained and the the model need to be evaluated, this testing manner has an ordered testing task sequence $\{\mathcal{T}_{q_{1}},\ldots,\mathcal{T}_{q_{i}}\}$, where $i\leq n$. The evaluation process consists of $i$ independent evaluations, with each testing task $\mathcal{T}_{q_{j}}=\{\mathcal{G}_{q_{j}},\mathcal{C}_{q_{j}}\}$ ($j\leq i$) leveraging the local subgraph ($\mathcal{G}_{q_{j}}=\mathcal{G}_{s_{j}}$) and global class set ($\mathcal{C}_{q_{j}}=\bigcup_{z=1}^{j}\mathcal{C}_{s_{z}}$), respectively. However, we highlight that this setup has a fundamental flaw: for each task, since testing and training samples are drawn from the same subgraph, their distribution similarity makes it trivial to accurately predict task ID, degrading class-incremental to task-incremental learning. For instance, TPP (Niu et al., 2024b) transductively captures task-specific prototypes utilizing a Laplacian smoothing-based matching approach, achieving 100% task ID prediction accuracy and 0% forgetting ratio. However, as demonstrated in Table [1](https://arxiv.org/html/2505.18697v2#S3.T1), our analysis reveals that even using a basic mean pooling operation to get prototypes achieves flawless task ID prediction. Furthermore, with accurate task identification, the model can also attain substantial performance by discarding the GNN entirely and simply employing task-specific two-layer MLPs for classification(^2^22Smoothing and mean pooling are applied to all nodes of each test graph $\mathcal{G}_{q_{i}}$, as shown in Figure [1](https://arxiv.org/html/2505.18697v2#S2.F1).). Consequently, we argue that this evaluation method is fundamentally flawed, as it fails to accurately assess the model’s genuine continual learning capability. In this paper, we primarily investigate the more challenging and realistic *global testing* setup. In *global testing*, we also have an ordered testing task sequence, but the difference is that for each testing task $\mathcal{T}_{q_{j}}=\{\mathcal{G}_{q_{j}},\mathcal{C}_{q_{j}}\}$ ($j\leq i$), the evaluated graph is the union of the subgraphs from all previous tasks ($\mathcal{G}_{q_{j}}=\bigcup_{z=1}^{j}\mathcal{G}_{s_{z}}$) and the class set is still global ($\mathcal{C}_{q_{j}}=\bigcup_{z=1}^{j}\mathcal{C}_{s_{z}}$). This setup not only better reflects real-world scenarios but also prevents the issue of task ID leakage, enabling authentic assessment of continual learning performance(^3^33For comprehensive discussion on the two test settings, and the details of TPP, please see Appendix [A.2](https://arxiv.org/html/2505.18697v2#A1.SS2).).

In addition to evaluation settings, previous knowledge leakage and label imbalance also constitute significant concerns for both NCIL and FSNCIL approaches. Since GNNs’ message-passing mechanisms inherently aggregate neighborhood information, these edges connecting old and new task nodes potentially expose previous task knowledge during new task training (Zhang et al., 2022a). Given that real-world scenarios often prohibit access to previous task data due to privacy or storage constraints, our benchmark excludes inter-task edges, using only intra-task connections. As for the label imbalance issue, since each session contains only a limited number of classes, while sample sizes vary substantially across classes, the imbalance issue becomes pronounced in continual learning settings. To prevent performance degradation caused by this issue, our benchmark removes the classes with insufficient samples and utilizes a unified sample size for each remaining class.

**Table 1: Performance evaluation on three pipelines: (1) GNN for feature aggregation + TPP (Laplacian smoothing) for prototype generation; (2) GNN aggregation + standard mean pooling (MP) for prototypes; (3) MLP aggregation + MP for prototypes. AA and AF represent average accuracy and forgetting ratio across all sessions.**
|  |  | Cora | Citeseer | WikiCS | Photo | Products | Arxiv-23 | Arxiv |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Backbone | Method | AA | AF | AA | AF | AA | AF | AA | AF | AA | AF | AA | AF | AA | AF |
| GNN | TPP | 95.2 | -0.0 | 87.9 | -0.0 | 93.6 | -0.0 | 91.3 | -0.0 | 89.6 | -0.0 | 91.6 | -0.0 | 85.7 | -0.0 |
| GNN | MP | 95.2 | -0.0 | 87.9 | -0.0 | 93.6 | -0.0 | 91.3 | -0.0 | 89.6 | -0.0 | 91.6 | -0.0 | 85.7 | -0.0 |
| MLP | MP | 90.3 | -0.0 | 86.9 | -0.0 | 89.2 | -0.0 | 79.0 | -0.0 | 83.1 | -0.0 | 88.8 | -0.0 | 81.6 | -0.0 |

### 3.2 Benchmark Settings

Datasets.
To ensure comprehensive evaluation, we gather seven diverse and representative datasets: Cora (He et al., 2023), Citeseer (Chen et al., 2024b), WikiCS (Liu et al., 2023a), Photo (Yan et al., 2023), Products (Hu et al., 2020), Arxiv-23 (He et al., 2023), and Arxiv (Hu et al., 2020), according to the following criteria: (1) Multiple domains. LLM4GCL incorporates datasets spanning multiple domains, including citation networks, web link networks, and e-commerce networks, reflecting diverse real-world applications; (2) Diverse scale and density. LLM4GCL datasets cover a wide range of scales, from thousands to hundreds of thousands of nodes. Moreover, the density of these datasets also varies significantly, supporting a robust evaluation of the model’s effectiveness; (3) Various sessions. LLM4GCL datasets provide varied session settings of both total session number and classes per session, allowing comprehensive analysis of the model’s continual learning performance. The statistics of these datasets and descriptions are provided in Appendix [A.3](https://arxiv.org/html/2505.18697v2#A1.SS3).

Baselines. LLM4GCL evaluates three baseline categories differentiated by backbone integration strategies: (1) GNN-based methods, including Vanilla GCN (Kipf & Welling, 2016), EWC (Kirkpatrick et al., 2017), LwF (Li & Hoiem, 2017), Cosine, and TPP (Niu et al., 2024b) for NCIL, with TEEN (Wang et al., 2023a) added for FSNCIL; (2) LLM-based methods, including encoder-only BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019), decoder-only LLaMA (Touvron et al., 2023), and RoBERTa integrated with SimpleCIL (Zhou et al., 2025); (3) GLM-based methods, including two LLM-as-Enhancer method GCN${}_{\text{Emb}}$ (Wu et al., 2025) and ENGINE (Zhu et al., 2024), and three LLM-as-Predictor approaches GraphPrompter (Liu et al., 2024), GraphGPT (Tang et al., 2024a), and LLaGA (Chen et al., 2024a). Extended descriptions are provided in Appendix [A.4](https://arxiv.org/html/2505.18697v2#A1.SS4) and [B](https://arxiv.org/html/2505.18697v2#A2).

Metrics.
In LLM4GCL, we employ Accuracy as the evaluation metric. Following established continual learning benchmark (Rebuffi et al., 2017), we define $\mathcal{A}_{i}$ as the accuracy after the $i$-th training task. The final metrics for both NCIL and FSNCIL are $\bar{\mathcal{A}}=\frac{1}{n}\sum_{i=1}^{n}\mathcal{A}_{i}$ and $\mathcal{A}_{N}$, which refer to the average accuracy across all tasks and the accuracy after the last task, respectively. These two metrics can well assess the model’s overall performance and ultimate knowledge preservation.

Figure: Figure 2: Proposed SimGCL framework, which executes two stages: (1) fine-tuning with graph prompts and instructions, followed by (2) training-free prototype generation and classification using LLM-generated node embeddings.
Refer to caption: x2.png

### 3.3 SimGCL: A Simple yet Effective GLM-based Approach for GCL

Currently, the application of LLMs and GLMs to GCL tasks remains nascent. To the best of our knowledge, there are no existing LLM- or GLM-based methods specifically designed for GCL tasks. While directly employing GLMs offers a straightforward solution by combining evolving graph structures with LLMs’ generalization capabilities, this approach incurs substantial computational overhead. Furthermore, as new tasks accumulate, LLMs’ general knowledge progressively degrades, resulting in a biased preference toward newer tasks. Consequently, we propose SimGCL, as depicted in Figure [2](https://arxiv.org/html/2505.18697v2#S3.F2), which demonstrates two advantages: (1) Efficiency. SimGCL only requires a single round of instruction tuning in the first session, dramatically reducing training costs. (2) Generalization. SimGCL constructs class prototypes directly from the embeddings produced by its fine-tuned LLM. This training-free approach avoids parameter updates, thereby mitigating catastrophic forgetting.

Instruction Tuning.
The instruction tuning phase facilitates LLM adaptation to graph-structured tasks by bridging the distributional gap between the pretraining corpus and target task datasets. Inspired by (Wang et al., 2025), we employ a neighborhood-aware graph prompt to guide the LLM’s processing. Derived from each node’s ego-graph $\tilde{\mathcal{G}}_{v}$, this prompt template comprises three components: (1) Task description outlining the scenario and dataset; (2) Graph structure incorporating target node attributes and neighborhood structure, with each node represented in the format of [Node ID][Node Text]; and (3) Question that specifies requirements and label text options. Moreover, to ensure training efficiency, we limit LLM tuning to the first session and adopt LoRA (Hu et al., 2022) instead of full-parameter fine-tuning to prevent overfitting.

Prototype Generation and Classification.
Following instruction tuning, SimGCL progressively generates class prototypes across sessions. Formally, let $f_{\theta}:(r_{\text{prompt}},\mathcal{G}_{v})\mapsto\mathbf{h}_{v}$ denote the LLM’s mapping from inputs to final-layer embeddings, the prototype $\mathbf{c}_{i}$ for class $i$ is computed as:

$$ $\displaystyle\mathbf{c}_{i}=\frac{1}{K}\sum_{j=1}^{|\mathcal{Y}_{b}|}\mathbb{I}(y_{j}=i)\mathbf{h}_{j}$ (1) $$

where $K=\sum_{j=1}^{|\mathcal{Y}_{b}|}\mathbb{I}(y_{j}=i)$, with $|\mathcal{Y}_{b}|$ being the number of labeled nodes in session $b$ where class $i$ appears, and $\mathbb{I}$ denotes the indicator function. Then during prediction, for node $v_{i}$ with its embedding $\mathbf{h}_{i}$, its probability of belonging to class $j$ is formulated as follows:

$$ $\displaystyle\hat{y}_{i,j}=\frac{\mathbf{h}_{i}\cdot{\mathbf{c}_{j}}}{\left\|\mathbf{h}_{i}\right\|\cdot\left\|\mathbf{c}_{j}\right\|}\cdot\tau$ (2) $$

where $\tau(\tau>0)$ is the scaling hyperparameter controlling the weight distribution. This training-free approach effectively preserves class-discriminative features across sessions, simultaneously reducing computational cost for new sessions while alleviating catastrophic forgetting in LLMs.

## 4 Experiments and Analysis

In this section, we conduct extensive experiments to analyze the performance of various methods on GCL tasks. Specifically, Tables [2](https://arxiv.org/html/2505.18697v2#S4.T2) and [3](https://arxiv.org/html/2505.18697v2#S4.T3) show the results for the NCIL and FSNCIL scenarios, respectively. Figure [3](https://arxiv.org/html/2505.18697v2#S4.F3) compares the influence of various LLM model sizes, while Figure [4](https://arxiv.org/html/2505.18697v2#S4.T4) shows how the performance changes as the number of sessions increases. Additional experiments, including results on domain-incremental learning, time-efficiency analysis, and visualizations, are available in Appendix [D](https://arxiv.org/html/2505.18697v2#A4). Based on these results, we give the following key observations (Obs.):

**Table 2: Performance comparison of GNN-, LLM-, and GLM-based methods in the NCIL scenario of LLM4GCL. The best and second results are highlighted.**
|  | Cora | Citeseer | WikiCS | Photo | Products | Arxiv-23 | Arxiv |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ |
| GCN | 57.0 | 38.2 | 52.4 | 30.2 | 54.9 | 34.9 | 46.5 | 19.9 | 25.5 | 5.4 | 19.9 | 2.9 | 21.2 | 1.5 |
| EWC | 56.0 | 31.0 | 45.9 | 28.5 | 55.3 | 33.0 | 46.9 | 20.5 | 29.4 | 15.9 | 25.7 | 15.0 | 24.9 | 13.8 |
| LwF | 55.7 | 30.8 | 47.8 | 28.2 | 55.0 | 34.0 | 45.8 | 20.4 | 26.0 | 7.5 | 21.1 | 6.8 | 18.9 | 1.6 |
| Cosine | 65.4 | 45.2 | 50.7 | 31.0 | 66.5 | 53.5 | 63.6 | 49.6 | 36.1 | 16.1 | 36.1 | 24.2 | 37.4 | 27.8 |
| TPP | 45.7 | 13.7 | 45.4 | 9.6 | 35.1 | 9.8 | 36.3 | 5.7 | 15.0 | 0.0 | 12.2 | 0.3 | 19.6 | 3.4 |
| BERT | 56.0 | 29.9 | 53.8 | 28.7 | 58.4 | 30.0 | 43.4 | 18.4 | 26.9 | 4.1 | 27.1 | 5.1 | 26.0 | 9.7 |
| RoBERTa | 54.6 | 29.6 | 54.1 | 28.6 | 55.1 | 30.8 | 43.5 | 19.1 | 27.6 | 3.2 | 23.1 | 0.8 | 24.4 | 1.4 |
| LLaMA | 65.6 | 53.8 | 55.7 | 31.7 | 55.5 | 30.9 | 44.6 | 19.1 | 29.8 | 0.4 | 24.3 | 1.0 | 24.9 | 1.4 |
| SimpleCIL | 70.8 | 58.3 | 66.4 | 49.5 | 71.4 | 57.3 | 62.1 | 52.5 | 66.8 | 52.6 | 52.4 | 38.8 | 50.6 | 36.5 |
| GCN${}_{\text{LLMEmb}}$ | 59.1 | 31.1 | 53.6 | 30.4 | 53.4 | 27.5 | 47.7 | 21.0 | 26.9 | 0.1 | 22.9 | 0.8 | 24.3 | 1.4 |
| ENGINE | 59.2 | 31.3 | 53.5 | 29.8 | 56.4 | 30.1 | 47.9 | 21.0 | 27.2 | 1.1 | 22.5 | 0.7 | 24.6 | 1.3 |
| GraphPrompter | 61.9 | 46.8 | 60.2 | 30.6 | 59.6 | 38.3 | 51.5 | 31.0 | 29.0 | 0.8 | 23.4 | 0.9 | 24.8 | 1.4 |
| GraphGPT | 55.5 | 31.6 | 60.0 | 30.1 | 62.0 | 49.2 | 50.8 | 30.2 | 35.5 | 3.2 | 30.7 | 5.8 | 32.2 | 3.9 |
| LLaGA | 58.2 | 30.2 | 51.3 | 27.8 | 53.7 | 27.6 | 47.2 | 20.7 | 25.7 | 0.2 | 26.9 | 4.1 | 26.1 | 1.3 |
| SimGCL (Ours) | 84.6 | 80.0 | 77.1 | 66.3 | 73.5 | 61.9 | 82.1 | 72.6 | 71.1 | 60.2 | 38.7 | 13.6 | 59.9 | 33.8 |

Obs. ❶ GNN-based methods exhibit persistent limitations in both NCIL and FSNCIL scenarios. As shown in Table [2](https://arxiv.org/html/2505.18697v2#S4.T2) and Table [3](https://arxiv.org/html/2505.18697v2#S4.T3), GNN-based approaches consistently underperform across all seven datasets. Notably, even on the small-scale Cora and Citeseer datasets, their final incremental accuracy ($\mathcal{A}_{N}$) fails to exceed 50%. This challenge even escalates significantly in larger-scale datasets like Arxiv-23 and Arxiv, where GCN and LwF achieve merely 1.5% and 1.6%, 2.0% and 3.8% accuracy after the final session of NCIL and FSNCIL, respectively, underscoring severe catastrophic forgetting. Moreover, classic CL methods (e.g., EWC and LwF) demonstrate inferior performance to vanilla GCN on shorter-session datasets and yield only marginal improvements in longer-session scenarios, while TPP fails to accurately identify task IDs, resulting in severely degraded performance under *global testing* settings. These limitations underscore the critical need for novel approaches to address GCIL and GFSCIL challenges effectively.

Obs. ❷ LLMs demonstrate higher performance in both NCIL and FSNCIL tasks, even without explicit graph structure utilization.
As shown in Table [2](https://arxiv.org/html/2505.18697v2#S4.T2) and Table [3](https://arxiv.org/html/2505.18697v2#S4.T3), while the encoder-only BERT and RoBERTa achieve performance only comparable to or marginally better than GCN, the decoder-only LLaMA attains a 16.9% improvement on Citeseer in the FSNCIL scenario, while SimpleCIL surpasses all the GNN-based models in most datasets and metrics (24 out of 28), with average improvements of 12.93% in NCIL and 9.7% in FSNCIL, respectively. We attribute this superior performance to the LLM backbone’s exceptional generalization capability for processing textual features and capturing class-specific semantics. The results also demonstrate the substantial potential of LLMs in addressing NCIL and FSNCIL challenges.

**Table 3: Performance comparison of GNN-, LLM-, and GLM-based methods in the FSNCIL scenario of LLM4GCL. The best and second results are highlighted.**
|  | Cora | Citeseer | WikiCS | Photo | Products | Arxiv-23 | Arxiv |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ |
| GCN | 68.0 | 38.1 | 39.5 | 17.4 | 62.4 | 47.4 | 58.5 | 32.3 | 36.0 | 14.1 | 22.2 | 5.4 | 25.3 | 2.0 |
| EWC | 59.0 | 36.3 | 49.2 | 21.2 | 58.4 | 40.4 | 62.0 | 28.4 | 45.7 | 31.5 | 36.4 | 29.5 | 31.5 | 18.3 |
| LwF | 63.3 | 43.5 | 45.1 | 20.7 | 59.4 | 41.0 | 60.1 | 29.4 | 50.3 | 38.7 | 30.8 | 15.1 | 27.8 | 3.8 |
| Cosine | 72.6 | 57.8 | 49.1 | 25.7 | 68.0 | 50.7 | 67.9 | 50.5 | 50.9 | 33.6 | 40.1 | 27.9 | 27.2 | 17.2 |
| TEEN | 60.9 | 40.3 | 59.0 | 39.5 | 59.3 | 42.4 | 59.3 | 35.5 | 49.6 | 28.4 | 39.2 | 27.1 | 31.8 | 18.6 |
| TPP | 39.0 | 9.1 | 37.3 | 12.6 | 37.3 | 12.4 | 40.0 | 14.0 | 14.0 | 0.0 | 7.3 | 0.0 | 11.1 | 6.0 |
| BERT | 56.4 | 34.7 | 61.1 | 38.8 | 61.7 | 33.0 | 47.7 | 25.8 | 22.4 | 3.8 | 15.2 | 3.4 | 14.9 | 6.0 |
| RoBERTa | 59.6 | 41.9 | 54.0 | 29.2 | 67.2 | 42.3 | 58.2 | 29.6 | 38.8 | 6.9 | 22.6 | 1.4 | 25.7 | 1.3 |
| LLaMA | 72.6 | 55.6 | 75.9 | 55.5 | 65.2 | 43.9 | 61.4 | 32.3 | 43.5 | 13.7 | 23.5 | 1.5 | 22.7 | 1.4 |
| SimpleCIL | 69.6 | 53.6 | 64.1 | 49.9 | 73.2 | 63.1 | 66.3 | 53.0 | 65.6 | 53.6 | 49.8 | 40.0 | 46.4 | 36.6 |
| GCN${}_{\text{LLMEmb}}$ | 68.2 | 40.1 | 54.3 | 28.7 | 54.7 | 31.0 | 66.0 | 34.2 | 30.6 | 0.2 | 21.4 | 1.0 | 22.1 | 0.9 |
| ENGINE | 52.2 | 28.3 | 47.6 | 25.5 | 46.8 | 23.7 | 48.0 | 21.5 | 20.9 | 0.1 | 17.4 | 0.5 | 17.3 | 1.4 |
| GraphPrompter | 63.2 | 37.3 | 65.3 | 34.5 | 69.5 | 51.6 | 69.7 | 51.0 | 37.9 | 2.2 | 27.4 | 2.3 | 27.3 | 1.0 |
| GraphGPT | 62.4 | 39.6 | 65.0 | 41.7 | 71.2 | 61.6 | 62.2 | 38.9 | 43.2 | 16.2 | 25.4 | 1.7 | 24.3 | 2.0 |
| LLaGA | 62.0 | 39.6 | 52.2 | 28.0 | 49.4 | 30.4 | 48.8 | 18.0 | 28.0 | 0.2 | 18.2 | 0.9 | 16.6 | 1.6 |
| SimGCL (Ours) | 78.0 | 67.6 | 78.0 | 63.8 | 68.8 | 64.1 | 81.2 | 71.3 | 69.7 | 62.7 | 31.8 | 10.3 | 36.3 | 6.8 |

Obs. ❸ Current GLM-based methods demonstrate unsatisfactory performance on both NCIL and FSNCIL scenarios.
Contrary to expectations, deliberately designed GLMs fail to achieve competitive performance, indicating that current GLM approaches exhibit greater limitations in GCL compared to pure LLM-based methods. We subsequently analyze this phenomenon through two aspects: ① LLM-as-Enhancer. GCN${}_{\text{Emb}}$ and ENGINE employ LLMs to augment graph features, yielding minimal but consistent NCIL improvements (0.2%–3.4% over vanilla GCN). However, in FSNCIL settings, these methods exhibit significant performance degradation across WikiCS, Products, Arxiv-23, and Arxiv datasets, underperforming vanilla GCN. We attribute this phenomenon to these models’ tendency to overfit training samples, which is further exacerbated by FSNCIL’s few-shot learning scenario, ultimately amplifying catastrophic forgetting. These methods still fundamentally rely on GNN architectures for final predictions, inheriting their limitations in continual learning scenarios. Given that the GNN’s limited generalization capacity serves as the performance bottleneck, we argue that LLM-as-Enhancer GLMs are suboptimal for GCL scenarios.
② LLM-as-Predictor. Among GraphPrompter, GraphGPT, and LLaGA, GraphPrompter demonstrates superior performance, consistently exceeding GCN${}_{\text{Emb}}$ across most datasets and metrics. Nevertheless, its advantages over LLaMA remain limited to specific datasets. We attribute this to two key factors: On the one hand, the architectural gap between shallow GNNs and deep LLMs leads to divergent parameter updates during continual learning, worsening inter-modal misalignment, and degrading GLM performance. On the other hand, GLMs like LLaGA possess strong fitting capabilities, which may lead to over-adaptation to recent tasks, consequently compromising their generalization and transfer abilities.

Obs. ❹ Dense graph structures may enhance GLM effectiveness in GCL.
Table [2](https://arxiv.org/html/2505.18697v2#S4.T2) and Table [3](https://arxiv.org/html/2505.18697v2#S4.T3) reveal that LLM-as-Predictor GLMs achieve notable performance gains on the WikiCS and Photo datasets, with GraphPrompter and GraphGPT ranking as runner-up or third-best models. This improvement likely stems from these datasets’ higher graph density than datasets like Cora or Products, providing richer structural information for LLM to better understand the task scenarios and make predictions. However, all GLMs exhibit suboptimal performance on Arxiv despite their comparable edge density (6.89 edges/node), suggesting that extended session ranges may disproportionately compromise LLMs’ generalization capacity, outweighing potential structural benefits.

Obs. ❺ Prototype-based learning improves cross-task generalization for GNNs and LLMs.
Table [2](https://arxiv.org/html/2505.18697v2#S4.T2) and Table [3](https://arxiv.org/html/2505.18697v2#S4.T3) demonstrate that prototype-based methods achieve superior performance, with Cosine and SimpleCIL emerging as optimal approaches for GNN- and LLM-based methods, respectively. In NCIL settings with abundant per-session samples, these methods outperform other baselines by significant margins (maximum improvements of 29.7% and 39.2% versus GCN and RoBERTa). This advantage stems from an optimal balance between generalization and plasticity: Initial session tuning bridges the distribution gap between the initialized or pretrained representations and target tasks, while subsequent session prototype generation preserves model parameters, maintaining generalization without catastrophic interference. Moreover, the prototype framework requires only first-session training, offering significant computational efficiency benefits.

Obs. ❻ SimGCL consistently overperform GNN-, LLM- and GLM-based baselines. From Table [2](https://arxiv.org/html/2505.18697v2#S4.T2) and Table [3](https://arxiv.org/html/2505.18697v2#S4.T3), it is obvious that SimGCL consistently overperform other baselines (23 out of 28), with a maximum 21.7% and 18.0% improvement in NCIL and FSNCIL, respectively. The effectiveness of SimGCL stems from two key factors: (1) its graph-structured instruction tuning and prompting framework enhances LLMs’ comprehension of graph topology, yielding higher-quality prototypes enriched with task-specific structural information for more discriminative representations; (2) unlike GraphPrompter and GraphGPT that rely on GNN-derived representations, SimGCL encodes graph information through textual prompts that better align with LLMs’ native input space, eliminating cross-architecture representation mismatches. However, SimGCL demonstrates relatively inferior performance on the arxiv-23 dataset and in FSNCIL compared to NCIL, which we attribute to two factors: (1) The sparse graph structure of Arxiv-23 provides limited topological information, potentially compromising LLMs’ structural comprehension (Zhu et al., 2024); (2) SimGCL’s expanded tuning set (12 classes vs. 4 classes in NCIL) in FSNCIL may promote overfitting to the initial session data, thereby diminishing generalization capability for subsequent incremental sessions.

Figure: Figure 3: The performance of SimpleCIL and SimGCL across varying LLM backbone sizes on Arxiv. The ’B’ and ’Ro’ refer to BERT and RoBERTa.
Refer to caption: x3.png

Obs. ❼ Scaling LLM parameters enhances generalization. Figure [3](https://arxiv.org/html/2505.18697v2#S4.F3) compares the performance of SimpleCIL and SimGCL across LLM backbones of varying parameter scales. The results indicate that increased parameter counts consistently enhance generalization capacity across both encoder-only and decoder-only architectures, consequently boosting GCL performance throughout all task sessions.

Obs. ❽ LLMs and GLMs are better at dealing with tasks with short-range sessions. Table [4](https://arxiv.org/html/2505.18697v2#S4.T4) evaluates model performance on the Arxiv dataset across varying session and class configurations(^2^22Detailed results for BERT, GraphGPT, and LLaGA are provided in Appendix [D](https://arxiv.org/html/2505.18697v2#A4).). In the 8-way 5-session setting, most LLM- and GLM-based methods achieve performance comparable to or exceeding top GNN-based approaches. However, this advantage diminishes with increasing session numbers, where performance differences between LLM/GLM and GNN methods become negligible. We attribute this convergence to progressive overfitting during incremental sessions, which increasingly weakens the pretrained models’ generalization capacity, ultimately reducing their performance advantage. Furthermore, prototype-based methods (i.e., Cosine, SimpleCIL, and SimGCL) demonstrate consistent performance stability across all experimental configurations, as their training-free prototype generation mechanism remains unaffected by session variations. This property makes them particularly suitable for long-session GCL applications.

**Table 4: Performance comparison of LLM4GCL baselines in NCIL scenario across different class sizes (W) and session numbers (S) on Arxiv. The best and second results are highlighted.**
|  | 8W5S | 5W8S | 4W10S | 2W20S |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ |
| GCN | 24.5 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 2.0 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 20.5 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 1.1 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 21.2 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 1.5 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 13.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 0.7 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ |
| EWC | 31.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 7.1}$ | 19.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 17.6}$ | 26.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 5.7}$ | 13.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 12.7}$ | 24.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.7}$ | 13.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 12.3}$ | 15.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.0}$ | 4.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.5}$ |
| LwF | 29.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 4.7}$ | 8.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.6}$ | 22.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.1}$ | 2.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.6}$ | 18.9 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 2.3}$ | 1.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.1}$ | 14.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.8}$ | 1.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.3}$ |
| Cosine | 35.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 11.1}$ | 23.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 21.2}$ | 38.3 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 17.8}$ | 27.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 26.8}$ | 37.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 16.2}$ | 27.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 26.3}$ | 41.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 28.2}$ | 31.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 30.7}$ |
| RoBERTa | 33.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 9.4}$ | 3.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.9}$ | 26.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.2}$ | 2.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.0}$ | 24.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.2}$ | 1.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.1}$ | 15.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.2}$ | 0.3 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.4}$ |
| LLaMA | 35.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 10.5}$ | 3.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.8}$ | 27.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 7.1}$ | 1.5 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.4}$ | 24.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.7}$ | 1.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.1}$ | 15.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.5}$ | 0.3 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.4}$ |
| SimpleCIL | 46.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 21.6}$ | 31.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 29.4}$ | 49.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 29.2}$ | 35.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 34.8}$ | 50.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 29.4}$ | 36.5 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 35.0}$ | 52.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 39.2}$ | 39.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 38.4}$ |
| GCN${}_{\text{LLMEmb}}$ | 33.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 9.2}$ | 3.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.8}$ | 26.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.1}$ | 2.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.9}$ | 24.3 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.1}$ | 1.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.1}$ | 15.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.2}$ | 0.3 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.4}$ |
| ENGINE | 34.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 9.6}$ | 3.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.7}$ | 26.5 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.0}$ | 2.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.9}$ | 24.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.4}$ | 1.3 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.2}$ | 15.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.3}$ | 0.2 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.5}$ |
| GraphPrompter | 35.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 10.6}$ | 2.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.9}$ | 27.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 7.3}$ | 2.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.9}$ | 24.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.6}$ | 1.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.1}$ | 16.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.4}$ | 0.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.3}$ |
| SimGCL (Ours) | 51.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 27.1}$ | 28.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 26.7}$ | 53.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 32.5}$ | 30.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 29.3}$ | 59.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 38.7}$ | 33.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 32.3}$ | 57.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 44.0}$ | 17.5 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 16.8}$ |

## 5 Related Works

Continual Learning with Pre-trained Models (PTMs).
The application of PTMs to continual learning was initially explored in Computer Vision (CV) (Zhou et al., 2025; 2024) and Natural Language Processing (NLP) (Wang et al., 2023b; Zhao et al., 2024b), with Vision Transformer (ViT) (Dosovitskiy et al., 2020) and BERT (Devlin et al., 2019) serving as representative implementations in each domain respectively. Pre-trained on extensive corpora (Brown et al., 2020) rather than trained from scratch, PTMs exhibit inherent advantages in generalizability (Cao et al., 2023), which supports flexible algorithm design, including both frozen-backbone approaches in a non-continual manner (e.g., training-free prototypes (Zhou et al., 2025)) and parameter-efficient tuning (PEFT) methods that preserve pre-trained weights (e.g., prompt tuning (Jia et al., 2022) and adapter learning (Chen et al., 2022)). Furthermore, leveraging the lightweight nature of PEFT blocks, there are also methods that employ orthogonal optimization via auxiliary losses (Wang et al., 2023b) or null-space projection (Lu et al., 2024) to minimize cross-task interference.

Graph Continual Learning (GCL).
While continual learning in CV and NLP focuses on preventing catastrophic forgetting of semantic representations, GCL must additionally address the forgetting of learned topological patterns induced by evolving graph structures (Zhang et al., 2024). Typically, GCL methods can be categorized based on their approach to mitigating catastrophic forgetting, including (1) regularizing parameter updates through explicit constraints, such as topology weight (Liu et al., 2021) or knowledge distillation (Hoang et al., 2023; Li & Hoiem, 2017); (2) isolating task-critical parameters, typically implemented through expanding parameters or model components to accommodate new knowledge (Zhang et al., 2023a; 2022b); and (3) replaying representative historical data, including key nodes (Zhou & Cao, 2021), sparsified computational subgraphs (Zhang et al., 2022c; 2023b), and condensed task-specific graphs (Liu et al., 2023b; Niu et al., 2024a).
However, practical constraints like privacy concerns and storage limitations often prevent access to historical task data, thereby restricting the applicability of replay-based methods. Therefore, our benchmark primarily focuses on replay-free approaches.

Graph-enhanced Language Models (GLMs).
Recently, with the rapid development of LLMs (e.g., RoBERTa (Conneau et al., 2019), LLaMA (Touvron et al., 2023), GPTs (Achiam et al., 2023)), research interest in leveraging LLMs for graph tasks has grown substantially, resulting in numerous GLM approaches. Generally, these methods can be categorized into three main approaches based on how LLMs interact with graph structures or GNNs (Jin et al., 2024; Li et al., 2023): (1) LLM-as-Enhancer (He et al., 2023; Liu et al., 2023a; Zhu et al., 2024), which leverages LLMs to retrieve node-relevant semantic knowledge, enhancing GNNs’ initial embedding quality; (2) LLM-as-Predictor (Chen et al., 2024a; Huang et al., 2024; Li et al., 2024a), which employs LLMs to autoregressively make predictions using graph-structured information, including structural prompts and GNN-encoded representations; and (3) GNN-LLM Alignment (Jin et al., 2023; Zhao et al., 2022), which aligns LLMs with GNNs in the same vector space, enabling GNNs to be more semantically aware. Considering the generalization and semantic understanding capabilities of LLMs, combined with the graph structure modeling capacity of GNNs and other integrations, GLMs are anticipated to provide a novel approach to more effectively addressing GCL tasks.

## 6 Conclusion

This paper introduces LLM4GCL, the first comprehensive benchmark for evaluating GCL performance of LLMs and graph-enhanced LLMs. Our work makes three key contributions: Firstly, we systematically analyze the current GCL evaluation paradigm, identifying its critical flaws that may cause task ID leakage and compromise performance assessment. (2) We evaluate 9 representative LLM-based methods across 7 textual-attributed graphs, revealing that existing GLM-based approaches underperform in both NCIL and FSNCIL scenarios due to overfitting tendencies and LLM-GNN representation misalignment. However, prototype-based modifications enable these models to achieve state-of-the-art performance, with additional gains attainable through model scaling. (3) We propose SimGCL, an effective GLM-based approach combining graph-prompted instruction tuning with prototype networks, which enhances topological understanding while preserving LLMs’ generalization capacity. We believe the proposal of LLM4GCL will establish a foundation for future research by delineating effective pathways for PTM-graph integration. Moreover, we have made our code publicly available and welcome further contributions of new datasets and methods.

## References

- Achiam et al. (2023)
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.
Gpt-4 technical report.
*arXiv preprint arXiv:2303.08774*, 2023.
- Bhargava et al. (2021)
Prajjwal Bhargava, Aleksandr Drozd, and Anna Rogers.
Generalization in nli: Ways (not) to go beyond simple heuristics, 2021.
- Brown et al. (2020)
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.
Language models are few-shot learners.
*Advances in neural information processing systems*, 33:1877–1901, 2020.
- Cao et al. (2023)
Boxi Cao, Qiaoyu Tang, Hongyu Lin, Shanshan Jiang, Bin Dong, Xianpei Han, Jiawei Chen, Tianshu Wang, and Le Sun.
Retentive or forgetful? diving into the knowledge memorizing mechanism of language models.
*arXiv preprint arXiv:2305.09144*, 2023.
- Chen et al. (2023)
Dingshuo Chen, Yanqiao Zhu, Jieyu Zhang, Yuanqi Du, Zhixun Li, Qiang Liu, Shu Wu, and Liang Wang.
Uncovering neural scaling laws in molecular representation learning.
*Advances in Neural Information Processing Systems*, 36:1452–1475, 2023.
- Chen et al. (2024a)
Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang.
Llaga: Large language and graph assistant.
*arXiv preprint arXiv:2402.08170*, 2024a.
- Chen et al. (2022)
Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo.
Adaptformer: Adapting vision transformers for scalable visual recognition.
*Advances in Neural Information Processing Systems*, 35:16664–16678, 2022.
- Chen et al. (2024b)
Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, et al.
Exploring the potential of large language models (llms) in learning on graphs.
*ACM SIGKDD Explorations Newsletter*, 25(2):42–61, 2024b.
- Conneau et al. (2019)
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov.
Unsupervised cross-lingual representation learning at scale.
*arXiv preprint arXiv:1911.02116*, 2019.
- Devlin et al. (2019)
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.
Bert: Pre-training of deep bidirectional transformers for language understanding.
In *Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers)*, pp. 4171–4186, 2019.
- Dosovitskiy et al. (2020)
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.
An image is worth 16x16 words: Transformers for image recognition at scale.
*arXiv preprint arXiv:2010.11929*, 2020.
- Fan et al. (2019)
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin.
Graph neural networks for social recommendation.
In *The world wide web conference*, pp. 417–426, 2019.
- Han et al. (2025)
Li Han, Longxun Wang, Ziyang Cheng, Bo Wang, Guang Yang, Dawei Cheng, and Xuemin Lin.
Mitigating the tail effect in fraud detection by community enhanced multi-relation graph neural networks.
*IEEE Transactions on Knowledge and Data Engineering*, 2025.
- He et al. (2020)
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.
Momentum contrast for unsupervised visual representation learning.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 9729–9738, 2020.
- He et al. (2023)
Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi.
Harnessing explanations: Llm-to-lm interpreter for enhanced text-attributed graph representation learning.
*arXiv preprint arXiv:2305.19523*, 2023.
- He et al. (2024)
Yufei He, Yuan Sui, Xiaoxin He, and Bryan Hooi.
Unigraph: Learning a unified cross-domain foundation model for text-attributed graphs.
*arXiv preprint arXiv:2402.13630*, 2024.
- Hoang et al. (2023)
Thanh Duc Hoang, Do Viet Tung, Duy-Hung Nguyen, Bao-Sinh Nguyen, Huy Hoang Nguyen, and Hung Le.
Universal graph continual learning.
*arXiv preprint arXiv:2308.13982*, 2023.
- Hu et al. (2022)
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al.
Lora: Low-rank adaptation of large language models.
*ICLR*, 1(2):3, 2022.
- Hu et al. (2020)
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec.
Open graph benchmark: Datasets for machine learning on graphs.
*Advances in neural information processing systems*, 33:22118–22133, 2020.
- Huang et al. (2024)
Xuanwen Huang, Kaiqiao Han, Yang Yang, Dezheng Bao, Quanjin Tao, Ziwei Chai, and Qi Zhu.
Can gnn be good adapter for llms?
In *Proceedings of the ACM Web Conference 2024*, pp. 893–904, 2024.
- Jia et al. (2022)
Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim.
Visual prompt tuning.
In *European conference on computer vision*, pp. 709–727. Springer, 2022.
- Jin et al. (2023)
Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Xinyang Zhang, Qi Zhu, and Jiawei Han.
Patton: Language model pretraining on text-rich networks.
*arXiv preprint arXiv:2305.12268*, 2023.
- Jin et al. (2024)
Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, and Jiawei Han.
Large language models on graphs: A comprehensive survey.
*IEEE Transactions on Knowledge and Data Engineering*, 2024.
- Kipf & Welling (2016)
Thomas N Kipf and Max Welling.
Semi-supervised classification with graph convolutional networks.
*arXiv preprint arXiv:1609.02907*, 2016.
- Kirkpatrick et al. (2017)
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al.
Overcoming catastrophic forgetting in neural networks.
*Proceedings of the national academy of sciences*, 114(13):3521–3526, 2017.
- Li et al. (2025a)
Yayong Li, Peyman Moghadam, Can Peng, Nan Ye, and Piotr Koniusz.
Inductive graph few-shot class incremental learning.
In *Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining*, pp. 466–474, 2025a.
- Li et al. (2023)
Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, and Jeffrey Xu Yu.
A survey of graph meets large language model: Progress and future directions.
*arXiv preprint arXiv:2311.12399*, 2023.
- Li et al. (2024a)
Yuhan Li, Peisong Wang, Zhixun Li, Jeffrey Xu Yu, and Jia Li.
Zerog: Investigating cross-dataset zero-shot transferability in graphs.
In *Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining*, pp. 1725–1735, 2024a.
- Li et al. (2024b)
Yuhan Li, Peisong Wang, Xiao Zhu, Aochuan Chen, Haiyun Jiang, Deng Cai, Victor W Chan, and Jia Li.
Glbench: A comprehensive benchmark for graph with large language models.
*Advances in Neural Information Processing Systems*, 37:42349–42368, 2024b.
- Li et al. (2025b)
Yuhan Li, Xinni Zhang, Linhao Luo, Heng Chang, Yuxiang Ren, Irwin King, and Jia Li.
G-refer: Graph retrieval-augmented large language model for explainable recommendation.
In *Proceedings of the ACM on Web Conference 2025*, pp. 240–251, 2025b.
- Li et al. (2022)
Zhixun Li, Dingshuo Chen, Qiang Liu, and Shu Wu.
The devil is in the conflict: Disentangled information graph neural networks for fraud detection.
In *2022 IEEE International Conference on Data Mining (ICDM)*, pp. 1059–1064. IEEE, 2022.
- Li & Hoiem (2017)
Zhizhong Li and Derek Hoiem.
Learning without forgetting.
*IEEE transactions on pattern analysis and machine intelligence*, 40(12):2935–2947, 2017.
- Liu et al. (2023a)
Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang.
One for all: Towards training one graph model for all classification tasks.
*arXiv preprint arXiv:2310.00149*, 2023a.
- Liu et al. (2021)
Huihui Liu, Yiding Yang, and Xinchao Wang.
Overcoming catastrophic forgetting in graph neural networks.
In *Proceedings of the AAAI conference on artificial intelligence*, volume 35, pp. 8653–8661, 2021.
- Liu et al. (2023b)
Yilun Liu, Ruihong Qiu, and Zi Huang.
Cat: Balanced continual graph learning with graph condensation.
In *2023 IEEE International Conference on Data Mining (ICDM)*, pp. 1157–1162. IEEE, 2023b.
- Liu et al. (2019)
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.
Roberta: A robustly optimized bert pretraining approach.
*arXiv preprint arXiv:1907.11692*, 2019.
- Liu et al. (2024)
Zheyuan Liu, Xiaoxin He, Yijun Tian, and Nitesh V Chawla.
Can we soft prompt llms for graph learning tasks?
In *Companion Proceedings of the ACM Web Conference 2024*, pp. 481–484, 2024.
- Lu et al. (2024)
Yue Lu, Shizhou Zhang, De Cheng, Yinghui Xing, Nannan Wang, Peng Wang, and Yanning Zhang.
Visual prompt tuning in null space for continual learning.
*arXiv preprint arXiv:2406.05658*, 2024.
- Ma & Tang (2021)
Yao Ma and Jiliang Tang.
*Deep learning on graphs*.
Cambridge University Press, 2021.
- Mao et al. (2024)
Haitao Mao, Zhikai Chen, Wenzhuo Tang, Jianan Zhao, Yao Ma, Tong Zhao, Neil Shah, Mikhail Galkin, and Jiliang Tang.
Graph foundation models.
*CoRR*, 2024.
- Ni et al. (2019)
Jianmo Ni, Jiacheng Li, and Julian McAuley.
Justifying recommendations using distantly-labeled reviews and fine-grained aspects.
In *Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP)*, pp. 188–197, 2019.
- Niu et al. (2024a)
Chaoxi Niu, Guansong Pang, and Ling Chen.
Graph continual learning with debiased lossless memory replay.
*arXiv preprint arXiv:2404.10984*, 2024a.
- Niu et al. (2024b)
Chaoxi Niu, Guansong Pang, Ling Chen, and Bing Liu.
Replay-and-forget-free graph class-incremental learning: A task profiling and prompting approach.
*arXiv preprint arXiv:2410.10341*, 2024b.
- Radford et al. (2021)
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.
Learning transferable visual models from natural language supervision.
In *International conference on machine learning*, pp. 8748–8763. PmLR, 2021.
- Rebuffi et al. (2017)
Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi.
Learning multiple visual domains with residual adapters.
*Advances in neural information processing systems*, 30, 2017.
- Tang et al. (2024a)
Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang.
Graphgpt: Graph instruction tuning for large language models.
In *Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval*, pp. 491–500, 2024a.
- Tang et al. (2024b)
Wenzhuo Tang, Haitao Mao, Danial Dervovic, Ivan Brugere, Saumitra Mishra, Yuying Xie, and Jiliang Tang.
Cross-domain graph data scaling: A showcase with diffusion models.
*arXiv preprint arXiv:2406.01899*, 2024b.
- Touvron et al. (2023)
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al.
Llama 2: Open foundation and fine-tuned chat models.
*arXiv preprint arXiv:2307.09288*, 2023.
- Wang et al. (2023a)
Qi-Wei Wang, Da-Wei Zhou, Yi-Kai Zhang, De-Chuan Zhan, and Han-Jia Ye.
Few-shot class-incremental learning via training-free prototype calibration.
*Advances in Neural Information Processing Systems*, 36:15060–15076, 2023a.
- Wang et al. (2023b)
Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang.
Orthogonal subspace learning for language model continual learning.
*arXiv preprint arXiv:2310.14152*, 2023b.
- Wang et al. (2025)
Yuxiang Wang, Xinnan Dai, Wenqi Fan, and Yao Ma.
Exploring graph tasks with pure llms: A comprehensive benchmark and investigation.
*arXiv preprint arXiv:2502.18771*, 2025.
- Wang et al. (2022)
Yuyang Wang, Jianren Wang, Zhonglin Cao, and Amir Barati Farimani.
Molecular contrastive learning of representations via graph neural networks.
*Nature Machine Intelligence*, 4(3):279–287, 2022.
- Wu et al. (2019)
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger.
Simplifying graph convolutional networks.
In *International conference on machine learning*, pp. 6861–6871. Pmlr, 2019.
- Wu et al. (2022)
Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui.
Graph neural networks in recommender systems: a survey.
*ACM Computing Surveys*, 55(5):1–37, 2022.
- Wu et al. (2025)
Xixi Wu, Yifei Shen, Fangzhou Ge, Caihua Shan, Yizhu Jiao, Xiangguo Sun, and Hong Cheng.
A comprehensive analysis on llm-based node classification algorithms.
*arXiv preprint arXiv:2502.00829*, 2025.
- Yan et al. (2023)
Hao Yan, Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao, Wenwen Zhuang, Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, et al.
A comprehensive study on text-attributed graphs: Benchmarking and rethinking.
*Advances in Neural Information Processing Systems*, 36:17238–17264, 2023.
- Yang et al. (2025)
Jie Yang, Rui Zhang, Ziyang Cheng, Dawei Cheng, Guang Yang, and Bo Wang.
Grad: Guided relation diffusion generation for graph augmentation in graph fraud detection.
In *Proceedings of the ACM on Web Conference 2025*, pp. 5308–5319, 2025.
- Yu et al. (2025)
Xingtong Yu, Zechuan Gong, Chang Zhou, Yuan Fang, and Hui Zhang.
Samgpt: Text-free graph foundation model for multi-domain pre-training and cross-domain adaptation.
In *Proceedings of the ACM on Web Conference 2025*, pp. 1142–1153, 2025.
- Zhang et al. (2023a)
Peiyan Zhang, Yuchen Yan, Chaozhuo Li, Senzhang Wang, Xing Xie, Guojie Song, and Sunghun Kim.
Continual learning on dynamic graphs via parameter isolation.
In *Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval*, pp. 601–611, 2023a.
- Zhang et al. (2022a)
Xikun Zhang, Dongjin Song, and Dacheng Tao.
Cglb: Benchmark tasks for continual graph learning.
*Advances in Neural Information Processing Systems*, 35:13006–13021, 2022a.
- Zhang et al. (2022b)
Xikun Zhang, Dongjin Song, and Dacheng Tao.
Hierarchical prototype networks for continual graph representation learning.
*IEEE Transactions on Pattern Analysis and Machine Intelligence*, 45(4):4622–4636, 2022b.
- Zhang et al. (2022c)
Xikun Zhang, Dongjin Song, and Dacheng Tao.
Sparsified subgraph memory for continual graph representation learning.
In *2022 IEEE International Conference on Data Mining (ICDM)*, pp. 1335–1340. IEEE, 2022c.
- Zhang et al. (2023b)
Xikun Zhang, Dongjin Song, and Dacheng Tao.
Ricci curvature-based graph sparsification for continual graph representation learning.
*IEEE Transactions on Neural Networks and Learning Systems*, 2023b.
- Zhang et al. (2024)
Xikun Zhang, Dongjin Song, and Dacheng Tao.
Continual learning on graphs: Challenges, solutions, and opportunities.
*arXiv preprint arXiv:2402.11565*, 2024.
- Zhao et al. (2024a)
Haihong Zhao, Aochuan Chen, Xiangguo Sun, Hong Cheng, and Jia Li.
All in one and one for all: A simple yet effective method towards cross-domain graph pretraining.
In *Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining*, pp. 4443–4454, 2024a.
- Zhao et al. (2022)
Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang.
Learning on large-scale text-attributed graphs via variational inference.
*arXiv preprint arXiv:2210.14709*, 2022.
- Zhao et al. (2024b)
Weixiang Zhao, Shilong Wang, Yulin Hu, Yanyan Zhao, Bing Qin, Xuanyu Zhang, Qing Yang, Dongliang Xu, and Wanxiang Che.
Sapt: A shared attention framework for parameter-efficient continual learning of large language models.
*arXiv preprint arXiv:2401.08295*, 2024b.
- Zhou et al. (2024)
Da-Wei Zhou, Hai-Long Sun, Jingyi Ning, Han-Jia Ye, and De-Chuan Zhan.
Continual learning with pre-trained models: A survey.
*arXiv preprint arXiv:2401.16386*, 2024.
- Zhou et al. (2025)
Da-Wei Zhou, Zi-Wen Cai, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu.
Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need.
*International Journal of Computer Vision*, 133(3):1012–1032, 2025.
- Zhou & Cao (2021)
Fan Zhou and Chengtai Cao.
Overcoming catastrophic forgetting in graph neural networks with experience replay.
In *Proceedings of the AAAI conference on artificial intelligence*, volume 35, pp. 4714–4722, 2021.
- Zhu et al. (2024)
Yun Zhu, Yaoke Wang, Haizhou Shi, and Siliang Tang.
Efficient tuning and inference for large language models on textual graphs.
*arXiv preprint arXiv:2401.15569*, 2024.

## Appendix A Additional Details on LLM4GCL

### A.1 Comparison between NCIL and FSNCIL

In this section, we will give a direct comparison between the two in Table [5](https://arxiv.org/html/2505.18697v2#A1.T5).

**Table 5: Comparison of NCIL and FSNCIL settings.**
| Setting | Session Type | Samples per Class | Classes per Session |
| --- | --- | --- | --- |
| NCIL | - | Sufficient (e.g., 200 samples) | Small (e.g., 3 classes) |
| FSNCIL | Base | Sufficient (e.g., 200 samples) | Large (e.g., 1/3 total classes) |
| Incremental | Few-shot (e.g., 10 shots) | Small ($\ll$ base, e.g., 3 classes) |  |

### A.2 Discussion on Local Testing and Global Testing

In this section, we will first introduce the pipeline of TPP (Niu et al., 2024b), followed by a comprehensive analysis of the two evaluation paradigms: *local testing* and *global testing*, supported by empirical results.

#### A.2.1 Approaches

TPP(^4^44In this work, we focus solely on formalizing the pipeline of TPP; for rigorous mathematical derivations and proofs, we direct readers to the original literature.).
TPP follows the *local testing* manner and proposes to use a Laplacian smoothing approach to generate a prototypical embedding for each graph task for task ID prediction. As for the $i$-th task with graph data $\mathcal{G}_{s_{i}}=\mathcal{G}_{q_{i}}=(\mathbf{A}_{s_{i}},\mathbf{X}_{s_{i}},\mathcal{V}_{s_{i}}^{l},\mathcal{V}_{s_{i}}^{u})$, TPP constructed a task prototype $\mathbf{p}_{s_{i}}$ based on the train set $\{\mathbf{x}_{j}\mid v_{j}\in\mathcal{V}_{s_{i}}^{l}\}$. Specifically, given training graph $\mathcal{G}_{s_{i}}$, the Laplacian smoothing is first applied to obtain the node embeddings $\mathbf{Z}_{s_{i}}$:

$$ $\displaystyle\mathbf{Z}_{s_{i}}=(\mathbf{I}-(\mathbf{\hat{D}}_{s_{i}})^{-\frac{1}{2}}\mathbf{\hat{L}}_{s_{i}}(\mathbf{\hat{D}}_{s_{i}})^{-\frac{1}{2}})^{k}\mathbf{X}_{s_{i}}$ (3) $$

where $k$ is the smoothing steps, $\mathbf{I}$ is an identity matrix, $\mathbf{\hat{L}}_{s_{i}}$ is the graph Laplacian matrix of $\mathbf{\hat{A}}_{s_{i}}$, and $\mathbf{\hat{D}}_{s_{i}}$ is the diagonal degree matrix. Consequently, the training task prototype $\mathbf{p}_{s_{i}}$ can be generated as:

$$ $\displaystyle\mathbf{p}_{s_{i}}=\frac{1}{|\mathcal{V}_{s_{i}}^{l}|}\sum_{v_{j}\in\mathcal{V}_{s_{i}}^{l}}\mathbf{z}_{s_{i},j}(\mathbf{\hat{D}}_{s_{i},jj})^{-\frac{1}{2}}$ (4) $$

Thus, all the task prototypes can be separately constructed and stored as $\mathcal{P}=\{\mathbf{p}_{s_{1}},\mathbf{p}_{s_{2}},...,\mathbf{p}_{s_{n}}\}$. Similarly, the task prototype of the local testing on $\mathcal{G}_{q_{i}}=\mathcal{G}_{s_{i}}$ can be denoted as:

$$ $\displaystyle\mathbf{p}_{q_{i}}=\frac{1}{|\mathcal{V}_{q_{i}}^{u}|}\sum_{v_{j}\in\mathcal{V}_{q_{i}}^{u}}\mathbf{z}_{q_{i},j}(\mathbf{\hat{D}}_{q_{i},jj})^{-\frac{1}{2}}$ (5) $$

Then, during local testing, the task ID prediction is conducted between the train and test prototypes:

$$ $\displaystyle t_{q_{i}}=\arg\min(d(\mathbf{p}_{q_{i}},\mathbf{p}_{s_{1}}),d(\mathbf{p}_{q_{i}},\mathbf{p}_{s_{2}}),...,d(\mathbf{p}_{q_{i}},\mathbf{p}_{s_{n}}))$ (6) $$

After precisely identifying the task ID, TPP subsequently incorporates a graph prompt learning framework that assigns distinct prompt vectors to adapt a pretrained GNN for diverse tasks:

$$ $\displaystyle\overline{\mathbf{x}}_{q_{i},j}=\mathbf{x}_{q_{i},j}+\sum_{m}^{M}\alpha_{m}\phi_{i,m},\alpha_{m}=\frac{e^{(\mathbf{w}_{m})^{T}\mathbf{x}_{q_{i},j}}}{\sum_{l}^{M}e^{(\mathbf{w}_{l})^{T}\mathbf{x}_{q_{i},j}}}$ (7) $$

where $\Phi_{i}=[\phi_{i,1},\phi_{i,2},...,\phi_{i,M}]^{T}\in\mathbb{R}^{M\times F}$ represents the learnable graph prompt for the $i$-th task, with $M$ denoting the number of vector-based tokens $\phi_{i}$. $\alpha_{m}$ is the importance score of token $\phi_{i,m}$ in the prompt, while $\mathbf{w}_{j}$ corresponds to a learnable projection weight. The final prompted graph representation $\mathcal{G}_{q_{i}}=\{\mathbf{A}_{q_{i}},\mathbf{X}_{q_{i}}+\Phi_{i}\}$ is then processed by a frozen pre-trained GNN model $f(\cdot,\cdot)$ to generate predictions:

$$ $\displaystyle\hat{\mathcal{Y}}_{q_{i}}=\varphi_{i}(f(\mathbf{A}_{q_{i}},\mathbf{X}_{q_{i}}+\Phi_{i}))$ (8) $$

where $\varphi_{i}$ denotes the task-specific MLP layer for the $i$-th task. Crucially, TPP effectively transforms the class-incremental learning problem into a task-incremental learning paradigm, decomposing the overall continual learning objective into multiple independent sub-tasks, thereby significantly mitigating the learning difficulty.

Mean Pooling.
TPP proposes that when applying a sufficiently large number of Laplacian smoothing iterations $k$, the distance between training and test prototypes from the same task approaches zero, while maintaining substantial separation between prototypes belonging to different CIL tasks. This discriminative property ensures reliable prediction accuracy. To evaluate the effectiveness of this technique, we implement an ablation model called Mean Pooling, where the Laplacian smoothing operation is replaced by a basic mean pooling aggregation, formally defined as:

$$ $\displaystyle\mathbf{p}_{s_{i}}=\frac{1}{|\mathcal{V}_{s_{i}}^{l}|}\sum_{v_{j}\in\mathcal{V}_{s_{i}}^{l}}\mathbf{x}_{s_{i},j},\ \ \mathbf{p}_{q_{i}}=\frac{1}{|\mathcal{V}_{q_{i}}^{u}|}\sum_{v_{j}\in\mathcal{V}_{q_{i}}^{u}}\mathbf{x}_{q_{i},j}$ (9) $$

Under this formulation, the distance between training and test prototypes depends solely on the feature distributions of the training and testing nodes.

MLP. By decomposing the CIL task into independent subtasks, TPP eliminates the need for session-wide classification across all accumulated classes. Instead, the model can focus exclusively on individual subtasks, significantly reducing task complexity. Building upon this observation, we propose an additional ablation model employing a basic two-layer MLP for classification:

$$ $\displaystyle\hat{\mathcal{Y}}_{q_{i}}=\varphi_{i}(\mathbf{X}_{q_{i}})$ (10) $$

#### A.2.2 Experimental Results

To investigate the impact of Laplacian smoothing and assess subtask complexity, we evaluate the performance of TPP against two variants, Mean Pooling and MLP, with results presented in Table [1](https://arxiv.org/html/2505.18697v2#S3.T1). We employ the *local testing* paradigm, where model performance is quantified using two key metrics: Average Accuracy (AA) and Average Forgetting (AF). These metrics are defined as follows:

$$ $\displaystyle\text{AA}=\frac{1}{n}\sum_{j=1}^{n}(\mathcal{A}_{n,j}),\ \ \text{AF}=\frac{1}{n}\sum_{j=1}^{n-1}(\mathcal{A}_{n,j}-\mathcal{A}_{j,j})$ (11) $$

where $n$ denotes the total number of sessions, and $\mathcal{A}_{i,j}$ represents the model’s classification accuracy for task $\mathcal{T}_{q_{j}}$ after training on task $\mathcal{T}_{s_{i}}$. The experimental results yield two key conclusions:

♣Laplacian smoothing demonstrates limited necessity for accurate task identification.
Comparative analysis between TPP and Mean Pooling demonstrates that both approaches achieve accurate task ID prediction. This phenomenon can be explained by the *local testing* setup, where training and testing nodes are drawn from the same graph ($\mathcal{G}_{s_{i}}=\mathcal{G}_{q_{i}}$). In this configuration, the feature distribution similarity between training and testing nodes within each class provides sufficient discriminative information for reliable task identification. These results indicate that the *local testing* paradigm is inherently susceptible to task ID leakage.

♠Decomposition into subtasks significantly reduces task complexity.
As evidenced by the experimental results, the MLP baseline achieves comparable performance to TPP across Citeseer, WikiCS, Arxiv-23, and Arxiv datasets, with merely a 3.08% average performance gap. This finding demonstrates that task decomposition effectively reduces learning difficulty, enabling simpler architectures to attain competitive performance. However, this characteristic may compromise the evaluation of models’ true continual learning capabilities, as the simplified subtasks fail to fully capture the challenges inherent in the original CIL problem.

### A.3 Details of Datasets

**Table 6: Statistics and experimental settings for all datasets in LLM4GCL. The symbol ’^∗’ indicates adjusted statistics that differ from the original dataset.**
| Dataset | Cora | Citeseer | WikiCS | Photo | Products | Arxiv-23 | Arxiv |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Attributes | # Nodes | 2,708 | 3,186 | 11,701 | 48,362 | 53,994^∗ | 46,196^∗ | 169,343 |
| # Edges | 5,429 | 4,277 | 215,863 | 500,928 | 72,242^∗ | 78,542^∗ | 1,166,243 |  |
| # Features | 1,433 | 3,703 | 300 | 768 | 100 | 300 | 128 |  |
| # Classes | 7 | 6 | 10 | 12 | 31^∗ | 37^∗ | 40 |  |
| Avg. # Token | 183.4 | 210.0 | 629.9 | 201.5 | 152.9^∗ | 237.7^∗ | 239.8 |  |
| Domain | Citation | Citation | Web link | E-Commerce | E-Commerce | Citation | Citation |  |
| NCIL | # Classes per session | 2 | 2 | 3 | 3 | 4 | 4 | 4 |
| # Sessions | 3 | 3 | 3 | 4 | 8 | 9 | 10 |  |
| # Shots per class | 100 | 100 | 200 | 400 | 400 | 400 | 800 |  |
| FSNCIL | # Base Classes | 3 | 2 | 4 | 4 | 11 | 13 | 12 |
| # Novel Classes | 4 | 4 | 6 | 8 | 20 | 24 | 28 |  |
| # Ways per session | 2 | 2 | 3 | 4 | 4 | 4 | 4 |  |
| # Sessions | 3 | 3 | 3 | 3 | 6 | 7 | 8 |  |
| # Shots per base class | 100 | 100 | 200 | 400 | 400 | 400 | 800 |  |
| # Shots per novel class | 5 | 5 | 5 | 5 | 5 | 5 | 5 |  |

All of the public datasets used in LLM4GCL were previously published, covering a multitude of domains. For each dataset, we store the graph-type data in the .pt format using PyTorch. This includes shallow embeddings, raw text of nodes, edge indices, node labels, and label names. The statistics of these datasets are presented in Table [6](https://arxiv.org/html/2505.18697v2#A1.T6), and the detailed descriptions are listed in the following:

- •
Cora (Chen et al., 2024b).
The Cora dataset is a citation network comprising research papers and their citation relationships within the computer science domain. The raw text data for the Cora dataset was sourced from the GitHub repository provided in Chen et al.(^5^55[https://github.com/CurryTang/Graph-LLM](https://github.com/CurryTang/Graph-LLM)). In this dataset, each node represents a research paper, and the raw text feature associated with each node includes the title and abstract of the respective paper. An edge in the Cora dataset signifies a citation relationship between two papers. The label assigned to each node corresponds to the category of the paper.
- •
Citeseer (Chen et al., 2024b).
The Citeseer dataset is a citation network comprising research papers and their citation relationships within the computer science domain. The TAG version of the dataset contains text attributes for 3,186 nodes, and the raw text data for the Citeseer dataset was sourced from the GitHub repository provided in Chen et al.. Each node represents a research paper, and each edge signifies a citation relationship between two papers.
- •
WikiCS (Liu et al., 2023a).
The WikiCS dataset is an internet link network where each node represents a Wikipedia page, and each edge represents a reference link between pages. The raw text data for the WikiCS dataset was collected from OFA(^6^66[https://github.com/LechengKong/OneForAll](https://github.com/LechengKong/OneForAll)). The raw text associated with each node includes the name and content of a Wikipedia entry. Each node’s label corresponds to the category of the entry.
- •
Ele-Photo (Photo) (Yan et al., 2023).
The Ele-Photo dataset(^7^77[https://github.com/sktsherlock/TAG-Benchmark](https://github.com/sktsherlock/TAG-Benchmark)) is derived from the Amazon Electronics dataset (Ni et al., 2019). The nodes in the dataset represent electronics-related products, and edges between two products indicate frequent co-purchases or co-views. The label for each dataset corresponds to the three-level label of the electronics products. User reviews on the item serve as its text attribute. In cases where items have multiple reviews, the review with the highest number of votes is utilized. For items lacking highly-voted reviews, a user review is randomly chosen as the text attribute.
- •
Ogbn-products (Products) (He et al., 2023).
The OGBN-Products dataset is characterized by its large scale, originally containing approximately 2 million nodes and 61 million edges. Following the node sampling strategy proposed in TAPE(^8^88[https://github.com/XiaoxinHe/TAPE](https://github.com/XiaoxinHe/TAPE)), we utilize a subset consisting of about 54k nodes and 72k edges for experimental use. In this dataset, each node represents a product sold on Amazon, and edges between two products indicate frequent co-purchasing relationships. The raw text data for each node includes product titles, descriptions, and/or reviews.
- •
Arxiv-23 (He et al., 2023).
The Arxiv-23 dataset is proposed in TAPE, representing a directed citation network comprising computer science arXiv papers published in 2023 or later. Similar to Ogbn-arxiv, each node in this dataset corresponds to an arXiv paper, and directed edges indicate citation relationships between papers. The raw text data for each node includes the title and abstract of the respective paper. The label assigned to each node represents one of the 40 subject areas of arXiv CS papers, such as cs.AI, cs.LG, and cs.OS. These subject areas are manually annotated by the paper’s authors and arXiv moderators.
- •
Ogbn-arxiv (Arxiv) (Liu et al., 2023a).
The Ogbn-arxiv dataset is a citation network comprising papers and their citation relationships, collected from the arXiv platform. The raw text data for the dataset was sourced from the GitHub repository provided in OFA. The original raw texts are available here(^9^99[https://snap.stanford.edu/ogb/data/misc/ogbn_arxiv](https://snap.stanford.edu/ogb/data/misc/ogbn_arxiv)). Each node represents a paper, and each edge denotes a citation relationship.

### A.4 Details of Baseline Models

In LLM4GCL, we integrate 15 baseline methods for NCIL and FSNCIL tasks, including 6 GNN-based methods, 4 LLM-based methods, and 5 GLM-based methods. We provide detailed descriptions of these methods used in our benchmark as follows.

- •
GNN-based methods.
–
GCN (Kipf & Welling, 2016). The first deep learning model that leverages graph convolutional layers. In LLM4GCL, we employ a two-layer GCN followed by one MLP layer. We use the code available at [https://github.com/pyg-team/pytorch_geometric](https://github.com/pyg-team/pytorch_geometric).
–
EWC (Kirkpatrick et al., 2017). A regularization-based CL method that prevents catastrophic forgetting through quadratic penalties on key parameter deviations. Parameter importance is quantified via the Fisher information matrix, maintaining performance on learned tasks during new task acquisition. We use the code available at [https://github.com/QueuQ/CGLB](https://github.com/QueuQ/CGLB).
–
LwF (Li & Hoiem, 2017). LwF minimizes the discrepancy between the logits of the old model and the new model through knowledge distillation to preserve knowledge from the old tasks. We use the code available at [https://github.com/QueuQ/CGLB](https://github.com/QueuQ/CGLB).
–
Cosine. Cosine first trains a GNN on the initial task session, subsequently freezes the network parameters, and leverages a training-free prototype mechanism to generate task-specific prototypes, ultimately employing cosine similarity for classification.
–
TPP (Niu et al., 2024b). A replay-and-forget-free GCL approach that employs Laplacian smoothing-based task identification to precisely predict task IDs and incorporates graph prompt engineering to adaptively transform the GNN into a series of task-specific classification modules. We use the code available at [https://github.com/mala-lab/TPP](https://github.com/mala-lab/TPP).
–
TEEN (Wang et al., 2023a). TEEN is a training-free prototype calibration strategy that addresses the issue of new classes being misclassified into base classes by leveraging the semantic similarity between base and new classes, which is captured by a feature extractor pre-trained on base classes. We use the code available at [https://github.com/wangkiw/TEEN](https://github.com/wangkiw/TEEN).
- •
LLM-based methods.
–
BERT (Bhargava et al., 2021; Devlin et al., 2019). A Transformer-based model family that employs masked language modeling and next sentence prediction pretraining to learn bidirectional contextual representations. It achieves state-of-the-art performance across multiple NLP tasks with minimal task-specific modifications. We use the code available at [https://github.com/google-research/bert](https://github.com/google-research/bert) and [https://github.com/prajjwal1/generalize_lm_nli](https://github.com/prajjwal1/generalize_lm_nli).
–
RoBERTa (Liu et al., 2019). An optimized BERT variant family that enhances pretraining through dynamic masking, larger batches, and extended data. It removes the next sentence prediction objective while maintaining MLM, achieving superior performance across NLP benchmarks with more efficient training. We use the code available at [https://github.com/facebookresearch/fairseq/tree/main/examples/xlmr](https://github.com/facebookresearch/fairseq/tree/main/examples/xlmr).
–
LLaMA (Touvron et al., 2023). A foundational large language model family featuring architectural optimizations like RMSNorm and SwiGLU activations. Trained on trillions of tokens from diverse public corpora, it achieves remarkable few-shot performance while maintaining efficient inference compared to similarly-sized models. We use the code available at [https://github.com/meta-llama/llama-cookbook](https://github.com/meta-llama/llama-cookbook).
–
SimpleCIL (Zhou et al., 2025). A computationally efficient approach that freezes the PTM’s embedding function and derives class prototypes by averaging embeddings for classification, which harnesses the model’s inherent generalizability for effective knowledge transfer without fine-tuning. We use the code available at [https://github.com/LAMDA-CL/RevisitingCIL](https://github.com/LAMDA-CL/RevisitingCIL).
- •
GLM-based methods.
–
GCN${}_{\text{Emb}}$. A simple method enhances GNNs by replacing their shallow embeddings with deep LLM-generated embeddings, where node text descriptions are encoded using LLaMA-8B to improve representation quality.
–
ENGINE (Zhu et al., 2024). A framework adds a lightweight and tunable G-Ladder module to each layer of the LLM, which uses a message-passing mechanism to integrate structural information. This enables the output of each LLM layer (i.e., token-level representations) to be passed to the corresponding G-Ladder, where the node representations are enhanced and then used for node classification. We use the code available at [https://github.com/ZhuYun97/ENGINE](https://github.com/ZhuYun97/ENGINE).
–
GraphPropmter (Liu et al., 2024). A framework integrating graph structures and LLMs via adaptive prompts, projecting GNN-based structural embeddings through MLP to align with the LLM’s semantic space for joint topology-text processing. We use the code available at [https://github.com/franciscoliu/graphprompter](https://github.com/franciscoliu/graphprompter).
–
GraphGPT (Tang et al., 2024a). A framework that initially aligns the graph encoder with natural language semantics through text-graph grounding, and then combines the trained graph encoder with the LLM using a projector, through which the model can directly complete graph tasks with natural language, thus performing zero-shot transferability. We use the code available at [https://github.com/WxxShirley/LLMNodeBed](https://github.com/WxxShirley/LLMNodeBed).
–
LLaGA (Chen et al., 2024a). LLaGA uses node-level templates to transform graph data into structured sequences, then maps them into token embeddings, enabling LLMs to process graph data with improved generalizability.
We use the code available at [https://github.com/WxxShirley/LLMNodeBed](https://github.com/WxxShirley/LLMNodeBed).

## Appendix B Details on Implementations

### B.1 Hyper-parameters

- •
For GCN, we grid-search the hyperparameters
lr in [1e-5, 1e-4, 1e-3], hidden_dim in [64, 128, 256]
layer_num = 2, dropout = 0.5
- •
For EWC, we grid-search the hyperparameters
lr in [1e-5, 1e-4, 1e-3], hidden_dim in [64, 128, 256],
strength in [1, 100, 10000], layer_num = 2
dropout = 0.5
- •
For LwF, we grid-search the hyperparameters
lr in [1e-5, 1e-4, 1e-3], hidden_dim in [64, 128, 256],
strength in [1, 100, 10000], lambda in [0.1, 1.0]
T in [0.2, 2.0], layer_num = 2, dropout = 0.5
- •
For Cosine, we grid-search the hyperparameters
lr in [1e-5, 1e-4, 1e-3], hidden_dim in [64, 128, 256]
layer_num = 2, dropout = 0.5, T = 1.0, sample_num = 100
- •
For TPP, we grid-search the hyperparameters
lr in [1e-5, 1e-4, 1e-3], hidden_dim in [64, 128, 256],
pe in [0.2, 0.3], pf in [0.2, 0.3], layer_num = 2
dropout = 0.5, pretrain_batch_size = 500
pretrain_lr = 1e-3
- •
For TEEN, we grid-search the hyperparameters
lr in [1e-5, 1e-4, 1e-3], hidden_dim in [64, 128, 256]
layer_num = 2, dropout = 0.5, T = 1.0, sample_num = 100
softmax_T = 16, shift_weight = 0.5
- •
For BERT and RoBERTa, we grid-search the hyperparameters
lr in [1e-4, 2e-4, 5e-4], batch_size in [5, 10, 20]
min_lr = 5e-6, weight_decay = 5e-2, dropout = 0.1
att_drouput = 0.1, max_length = 256, lora_r = 5
lora_alpha = 16, lora_dropout = 0.05
- •
For LLaMA, we grid-search the hyperparameters
lr in [1e-4, 2e-4, 5e-4], batch_size in [5, 10, 20]
min_lr = 5e-6, weight_decay = 5e-2, dropout = 0.1
att_drouput = 0.1, max_length = 512, lora_r = 5
lora_alpha = 16, lora_dropout = 0.05
- •
For SimpleCIL, we grid-search the hyperparameters
lr in [1e-4, 2e-4, 5e-4], batch_size in [5, 10, 20]
min_lr = 5e-6, weight_decay = 5e-2, dropout = 0.1
att_drouput = 0.1, max_length = 256, lora_r = 5
lora_alpha = 16, lora_dropout = 0.05, T = 1.0
sample_num = 20
- •
For GCN${}_{\text{Emb}}$, we grid-search the hyperparameters
lr in [1e-5, 1e-4, 1e-3], hidden_dim in [64, 128, 256]
layer_num = 2, dropout = 0.5
- •
For ENGINE, we grid-search the hyperparameters
lr in [1e-5, 1e-4, 1e-3], hidden_dim in [64, 128, 256]
r in [1, 32], layer_num = 1, dropout = 0.5, T = 0.1
layer_select = [0, 6, 12, 18, 24, -1]
- •
For GraphPrompter, we follow the instructions from the paper (Liu et al., 2024), use
lr = 2e-4, batch_size = 10, min_lr = 5e-6
weight_decay = 5e-2, dropout = 0.1, att_drouput = 0.1
max_length = 512, lora_r = 5, lora_alpha = 16
lora_dropout = 0.05, gnn = ’GCN’, layer_num = 4
hidden_dim = 1024, output_dim = 1024,
dropout = 0.5, proj_hidden_dim = 1024
- •
For GraphGPT, we follow the instructions from the paper (Wu et al., 2025), use
dropout = 0.1, att_drouput = 0.1, max_length = 512
s1_k_hop = 2, s1_num_neighbors = 5
s1_max_txt_length = 512, s1_epoch = 2, s1_batch_size = 5
s1_lr = 1e-4, s2_num_neighbors = 4, max_txt_length = 512
s2_epoch = 10, s2_batch_size = 5, s2_lr = 1e-4
lora_r = 5, lora_alpha = 16, lora_dropout = 0.05
- •
For LLaGA, we follow the instructions from the paper (Chen et al., 2024a), use
lr = 2e-4, batch_size = 10, min_lr = 5e-6
weight_decay = 5e-2, dropout = 0.1, att_drouput = 0.1
max_length = 512, llm_freeze = ’True’
neighbor_template = ’ND’, nd_mean = True, k_hop = 2
sample_size = 10, hop_field = 4, proj_layer = 2
- •
For SimGCL, we follow the instructions from the paper (Wang et al., 2025), use
lr = 2e-4, batch_size = 10, min_lr = 5e-6
weight_decay = 5e-2, lora_r = 5, lora_alpha = 16
lora_dropout = 0.05, T = 1.0, sample_num = 50
hop = [20, 20], include_label = False, max_node_text_len = 128

**Table 7: Selection of GNN and LLM Backbones in LLM4GCL, $\mathbf{A},\mathbf{X},\mathcal{R}$ refers to the adjacent matrix, feature matrix, and raw text attributes of the input TAG, respectively.**
| Type | Method | Predictor | Input | Output Format | GNN Backbone | LLM Backbone |
| --- | --- | --- | --- | --- | --- | --- |
| GNN-based | GCN (Kipf & Welling, 2016) | GNN | $(\mathbf{A,\mathbf{X}})$ | Logits | GCN | - |
| EWC (Kirkpatrick et al., 2017) | GNN | $(\mathbf{A,\mathbf{X}})$ | Logits | GCN | - |  |
| LwF (Li & Hoiem, 2017) | GNN | $(\mathbf{A,\mathbf{X}})$ | Logits | GCN | - |  |
| Cosine | GNN | $(\mathbf{A,\mathbf{X}})$ | Logits | GCN | - |  |
| TPP (Niu et al., 2024b) | GNN | $(\mathbf{A,\mathbf{X}})$ | Logits | SGC, GCN | - |  |
| TEEN (Wang et al., 2023a) | GNN | $(\mathbf{A,\mathbf{X}})$ | Logits | GCN | - |  |
| LLM-based | BERT (Devlin et al., 2019) | LLM | $\mathcal{R}$ | Logits | - | BERT-base (110M) |
| RoBERTa (Liu et al., 2019) | LLM | $\mathcal{R}$ | Logits | - | RoBERTa-large (355M) |  |
| LLaMA (Touvron et al., 2023) | LLM | $\mathcal{R}$ | Prediction Texts | - | LLaMA3-8B |  |
| SimpleCIL (Zhou et al., 2025) | LLM | $\mathcal{R}$ | Logits | - | RoBERTa-large (355M) |  |
| GLM-based | GCN${}_{\text{Emb}}$ | GNN | $(\mathbf{A},\mathcal{R})$ | Logits | GCN | LLaMA3-8B |
| ENGINE (Zhu et al., 2024) | GNN | $(\mathbf{A},\mathcal{R})$ | Logits | GCN | LLaMA3-8B |  |
| GraphPrompter (Huang et al., 2024) | LLM | $(\mathbf{A},\mathbf{X},\mathcal{R})$ | Prediction Texts | GCN | LLaMA3-8B |  |
| GraphGPT (Tang et al., 2024a) | LLM | $(\mathbf{A},\mathbf{X},\mathcal{R})$ | Prediction Texts | - | LLaMA3-8B |  |
| LLaGA (Chen et al., 2024a) | LLM | $(\mathbf{A},\mathbf{X},\mathcal{R})$ | Prediction Texts | - | LLaMA3-8B |  |

### B.2 Backbone Selection

To compare the various baseline methods in LLM4GCL as fairly as possible, we try to utilize the same GNN and LLM backbones in our implementations. Table [7](https://arxiv.org/html/2505.18697v2#A2.T7) shows the GNN and LLM backbones used in the original implementations, as well as those implemented in LLM4GCL.

## Appendix C Text Prompt Design

For GraphPrompter (Liu et al., 2024), GraphGPT (Tang et al., 2024a), and LLaGA (Chen et al., 2024a), we utilize the prompt templates provided in their original papers for several datasets, including Cora and arXiv. For datasets not originally addressed, such as Photo, we adapt their prompt designs to create similarly formatted prompts. For SimGCL, we adopt the graph prompt template established by Wang et al. (Wang et al., 2025). Below is a summary of these prompt templates using Cora as an example, where $\langle$labels$\rangle$ denotes the dataset-specific label space, $\langle$graph$\rangle$ represents the tokenized graph context, $\langle$raw_text$\rangle$ and $\langle$paper_titles$\rangle$ refer to the node’s original raw text, and $\langle$target node$\rangle$ is the node’s id within the graph. It is important to emphasize that during the continual learning process, as new classes emerge sequentially, the content of $\langle$labels$\rangle$ dynamically expands to incorporate these newly introduced class labels.

## Appendix D Additional Experimental Results

### D.1 Session Numbers

**Table 8: Performance comparison of LLM4GCL baselines in NCIL scenario across different class sizes (W) and session numbers (S) on Arxiv. The best and second results are highlighted.**
|  | 8W5S | 5W8S | 4W10S | 2W20S |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ |
| GCN | 24.5 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 2.0 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 20.5 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 1.1 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 21.2 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 1.5 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 13.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ | 0.7 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.0}$ |
| EWC | 31.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 7.1}$ | 19.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 17.6}$ | 26.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 5.7}$ | 13.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 12.7}$ | 24.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.7}$ | 13.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 12.3}$ | 15.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.0}$ | 4.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.5}$ |
| LwF | 29.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 4.7}$ | 8.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.6}$ | 22.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.1}$ | 2.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.6}$ | 18.9 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 2.3}$ | 1.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.1}$ | 14.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.8}$ | 1.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.3}$ |
| Cosine | 35.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 11.1}$ | 23.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 21.2}$ | 38.3 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 17.8}$ | 27.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 26.8}$ | 37.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 16.2}$ | 27.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 26.3}$ | 41.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 28.2}$ | 31.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 30.7}$ |
| BERT | 36.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 12.2}$ | 15.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 13.0}$ | 29.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 9.2}$ | 10.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 9.1}$ | 26.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 4.8}$ | 9.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 8.2}$ | 16.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.2}$ | 5.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 4.3}$ |
| RoBERTa | 33.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 9.4}$ | 3.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.9}$ | 26.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.2}$ | 2.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.0}$ | 24.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.2}$ | 1.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.1}$ | 15.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.2}$ | 0.3 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.4}$ |
| LLaMA | 35.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 10.5}$ | 3.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.8}$ | 27.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 7.1}$ | 1.5 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.4}$ | 24.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.7}$ | 1.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.1}$ | 15.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.5}$ | 0.3 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.4}$ |
| SimpleCIL | 46.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 21.6}$ | 31.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 29.4}$ | 49.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 29.2}$ | 35.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 34.8}$ | 50.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 29.4}$ | 36.5 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 35.0}$ | 52.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 39.2}$ | 39.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 38.4}$ |
| GCN${}_{\text{LLMEmb}}$ | 33.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 9.2}$ | 3.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.8}$ | 26.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.1}$ | 2.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.9}$ | 24.3 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.1}$ | 1.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.1}$ | 15.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.2}$ | 0.3 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.4}$ |
| ENGINE | 34.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 9.6}$ | 3.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.7}$ | 26.5 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.0}$ | 2.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.9}$ | 24.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.4}$ | 1.3 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.2}$ | 15.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.3}$ | 0.2 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.5}$ |
| GraphPrompter | 35.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 10.6}$ | 2.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.9}$ | 27.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 7.3}$ | 2.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.9}$ | 24.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.6}$ | 1.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.1}$ | 16.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.4}$ | 0.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.3}$ |
| GraphGPT | 38.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 14.4}$ | 7.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 5.4}$ | 35.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 14.5}$ | 6.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 5.1}$ | 32.2 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 11.0}$ | 3.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.4}$ | 23.5 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 10.1}$ | 3.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.4}$ |
| LLaGA | 34.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 10.3}$ | 8.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.6}$ | 27.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 7.4}$ | 5.3 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 4.2}$ | 26.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 4.9}$ | 1.3 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.2}$ | 16.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.2}$ | 0.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.2}$ |
| SimGCL (Ours) | 51.6 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 27.1}$ | 28.7 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 26.7}$ | 53.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 32.5}$ | 30.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 29.3}$ | 59.9 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 38.7}$ | 33.8 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 32.3}$ | 57.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 44.0}$ | 17.5 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 16.8}$ |

Table [8](https://arxiv.org/html/2505.18697v2#A4.T8) presents the performance of baseline models and SimGCL on the Arxiv dataset under different session-class configurations. Consistent with Obs. ❽, as session numbers increase, the performance advantages of BERT, GraphGPT, and LLaGA over GNN-based approaches diminish significantly, demonstrating pronounced catastrophic forgetting in long-session scenarios.

### D.2 Extended Analysis of SimGCL

**Table 9: Performance comparison of GNN-, LLM-, and GLM-based methods in the NCIL scenario of LLM4GCL. The best and second results are highlighted.**
|  | Cora | Citeseer | WikiCS | Photo | Products | Arxiv-23 | Arxiv |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ |
| GCN | 57.0 | 38.2 | 52.4 | 30.2 | 54.9 | 34.9 | 46.5 | 19.9 | 25.5 | 5.4 | 19.9 | 2.9 | 21.2 | 1.5 |
| EWC | 56.0 | 31.0 | 45.9 | 28.5 | 55.3 | 33.0 | 46.9 | 20.5 | 29.4 | 15.9 | 25.7 | 15.0 | 24.9 | 13.8 |
| LwF | 55.7 | 30.8 | 47.8 | 28.2 | 55.0 | 34.0 | 45.8 | 20.4 | 26.0 | 7.5 | 21.1 | 6.8 | 18.9 | 1.6 |
| Cosine | 65.4 | 45.2 | 50.7 | 31.0 | 66.5 | 53.5 | 63.6 | 49.6 | 36.1 | 16.1 | 36.1 | 24.2 | 37.4 | 27.8 |
| TPP | 45.7 | 13.7 | 45.4 | 9.6 | 35.1 | 9.8 | 36.3 | 5.7 | 15.0 | 0.0 | 12.2 | 0.3 | 19.6 | 3.4 |
| BERT | 56.0 | 29.9 | 53.8 | 28.7 | 58.4 | 30.0 | 43.4 | 18.4 | 26.9 | 4.1 | 27.1 | 5.1 | 26.0 | 9.7 |
| RoBERTa | 54.6 | 29.6 | 54.1 | 28.6 | 55.1 | 30.8 | 43.5 | 19.1 | 27.6 | 3.2 | 23.1 | 0.8 | 24.4 | 1.4 |
| LLaMA | 65.6 | 53.8 | 55.7 | 31.7 | 55.5 | 30.9 | 44.6 | 19.1 | 29.8 | 0.4 | 24.3 | 1.0 | 24.9 | 1.4 |
| SimpleCIL | 70.8 | 58.3 | 66.4 | 49.5 | 71.4 | 57.3 | 62.1 | 52.5 | 66.8 | 52.6 | 52.4 | 38.8 | 50.6 | 36.5 |
| GCN${}_{\text{LLMEmb}}$ | 59.1 | 31.1 | 53.6 | 30.4 | 53.4 | 27.5 | 47.7 | 21.0 | 26.9 | 0.1 | 22.9 | 0.8 | 24.3 | 1.4 |
| ENGINE | 59.2 | 31.3 | 53.5 | 29.8 | 56.4 | 30.1 | 47.9 | 21.0 | 27.2 | 1.1 | 22.5 | 0.7 | 24.6 | 1.3 |
| GraphPrompter | 61.9 | 46.8 | 60.2 | 30.6 | 59.6 | 38.3 | 51.5 | 31.0 | 29.0 | 0.8 | 23.4 | 0.9 | 24.8 | 1.4 |
| GraphGPT | 55.5 | 31.6 | 60.0 | 30.1 | 62.0 | 49.2 | 50.8 | 30.2 | 35.5 | 3.2 | 30.7 | 5.8 | 32.2 | 3.9 |
| LLaGA | 58.2 | 30.2 | 51.3 | 27.8 | 53.7 | 27.6 | 47.2 | 20.7 | 25.7 | 0.2 | 26.9 | 4.1 | 26.1 | 1.3 |
| SimGCL-1 Hops | 89.4 | 83.9 | 76.4 | 64.2 | 32.4 | 20.3 | 44.7 | 33.5 | 70.9 | 43.7 | 28.2 | 6.1 | 38.2 | 10.7 |
| SimGCL-2 Hops | 84.6 | 80.0 | 77.1 | 66.3 | 73.5 | 61.9 | 82.1 | 72.6 | 71.1 | 60.2 | 38.7 | 13.6 | 59.9 | 33.8 |

**Table 10: Performance comparison of GNN-, LLM-, and GLM-based methods in the FSNCIL scenario of LLM4GCL. The best and second results are highlighted.**
|  | Cora | Citeseer | WikiCS | Photo | Products | Arxiv-23 | Arxiv |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ |
| GCN | 68.0 | 38.1 | 39.5 | 17.4 | 62.4 | 47.4 | 58.5 | 32.3 | 36.0 | 14.1 | 22.2 | 5.4 | 25.3 | 2.0 |
| EWC | 59.0 | 36.3 | 49.2 | 21.2 | 58.4 | 40.4 | 62.0 | 28.4 | 45.7 | 31.5 | 36.4 | 29.5 | 31.5 | 18.3 |
| LwF | 63.3 | 43.5 | 45.1 | 20.7 | 59.4 | 41.0 | 60.1 | 29.4 | 50.3 | 38.7 | 30.8 | 15.1 | 27.8 | 3.8 |
| Cosine | 72.6 | 57.8 | 49.1 | 25.7 | 68.0 | 50.7 | 67.9 | 50.5 | 50.9 | 33.6 | 40.1 | 27.9 | 27.2 | 17.2 |
| TEEN | 60.9 | 40.3 | 59.0 | 39.5 | 59.3 | 42.4 | 59.3 | 35.5 | 49.6 | 28.4 | 39.2 | 27.1 | 31.8 | 18.6 |
| TPP | 39.0 | 9.1 | 37.3 | 12.6 | 37.3 | 12.4 | 40.0 | 14.0 | 14.0 | 0.0 | 7.3 | 0.0 | 11.1 | 6.0 |
| BERT | 56.4 | 34.7 | 61.1 | 38.8 | 61.7 | 33.0 | 47.7 | 25.8 | 22.4 | 3.8 | 15.2 | 3.4 | 14.9 | 6.0 |
| RoBERTa | 59.6 | 41.9 | 54.0 | 29.2 | 67.2 | 42.3 | 58.2 | 29.6 | 38.8 | 6.9 | 22.6 | 1.4 | 25.7 | 1.3 |
| LLaMA | 72.6 | 55.6 | 75.9 | 55.5 | 65.2 | 43.9 | 61.4 | 32.3 | 43.5 | 13.7 | 23.5 | 1.5 | 22.7 | 1.4 |
| SimpleCIL | 69.6 | 53.6 | 64.1 | 49.9 | 73.2 | 63.1 | 66.3 | 53.0 | 65.6 | 53.6 | 49.8 | 40.0 | 46.4 | 36.6 |
| GCN${}_{\text{LLMEmb}}$ | 68.2 | 40.1 | 54.3 | 28.7 | 54.7 | 31.0 | 66.0 | 34.2 | 30.6 | 0.2 | 21.4 | 1.0 | 22.1 | 0.9 |
| ENGINE | 52.2 | 28.3 | 47.6 | 25.5 | 46.8 | 23.7 | 48.0 | 21.5 | 20.9 | 0.1 | 17.4 | 0.5 | 17.3 | 1.4 |
| GraphPrompter | 63.2 | 37.3 | 65.3 | 34.5 | 69.5 | 51.6 | 69.7 | 51.0 | 37.9 | 2.2 | 27.4 | 2.3 | 27.3 | 1.0 |
| GraphGPT | 62.4 | 39.6 | 65.0 | 41.7 | 71.2 | 61.6 | 62.2 | 38.9 | 43.2 | 16.2 | 25.4 | 1.7 | 24.3 | 2.0 |
| LLaGA | 62.0 | 39.6 | 52.2 | 28.0 | 49.4 | 30.4 | 48.8 | 18.0 | 28.0 | 0.2 | 18.2 | 0.9 | 16.6 | 1.6 |
| SimGCL-1 Hop | 85.5 | 77.9 | 77.3 | 63.5 | 30.4 | 23.2 | 50.6 | 44.8 | 57.2 | 30.1 | 19.9 | 4.2 | 25.9 | 3.9 |
| SimGCL-2 Hops | 78.0 | 67.6 | 78.0 | 63.8 | 73.5 | 61.9 | 81.2 | 71.3 | 69.7 | 62.7 | 38.7 | 13.6 | 59.9 | 33.8 |

To comprehensively assess SimGCL’s performance, we conducted an evaluation using 1-hop neighborhood aggregation, with detailed results presented in Table [9](https://arxiv.org/html/2505.18697v2#A4.T9) and Table [10](https://arxiv.org/html/2505.18697v2#A4.T10).

Observation D.❶ Rich structural information better enhances task comprehension.
In Table [9](https://arxiv.org/html/2505.18697v2#A4.T9) and Table [10](https://arxiv.org/html/2505.18697v2#A4.T10), SimGCL exhibits performance degradation when limited to 1-hop neighborhood information, particularly on densely-connected datasets, WikiCS and Photo, where it underperforms even the vanilla GCN baseline. We hypothesize this stems from an inadequate structural context, misleading the model’s representation learning. Notably, modest improvements occur on sparser graph datasets, Cora and Citeseer, we attribute this to their smaller scale and lower connectivity makes 1-hop information sufficient for effective prediction.

### D.3 Domain-incremental Learning Setting

**Table 11: Performance comparison of GNN-, LLM-, and GLM-based methods in the DIL setting. Domain 1, Domain 2, and Domain 3 refer to the model continually learn on the PubMed, CiteSeer, and Cora datasets, respectively. The best and second results are highlighted.**
|  | Domain 1 | Domain 2 | Domain 3 |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | $\mathcal{A}$ | $\mathcal{F}$ | $\mathcal{A}$ | $\mathcal{F}$ | $\mathcal{A}$ | $\mathcal{F}$ | $\mathcal{\bar{A}}$ |
| GCN | 86.75 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.00}$ | - | 28.26 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.00}$ | -58.49 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.00}$ | 20.46 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.00}$ | -66.29 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.00}$ | 45.16 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.00}$ |
| EWC | 86.00 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.75}$ | - | 27.38 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.88}$ | -59.37 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.88}$ | 20.16 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.30}$ | -66.59 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.30}$ | 44.52 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.64}$ |
| Cosine | 85.50 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 1.25}$ | - | 34.91 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.65}$ | -51.84 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.65}$ | 27.41 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.95}$ | -59.34 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.95}$ | 49.28 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 4.12}$ |
| BERT | 89.67 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.92}$ | - | 29.63 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.37}$ | -60.04 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 1.55}$ | 18.17 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 2.29}$ | -71.58 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 5.29}$ | 45.82 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.66}$ |
| RoBERTa | 93.33 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 6.58}$ | - | 30.27 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.01}$ | -63.06 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 4.57}$ | 17.31 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 3.15}$ | -76.02 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 9.73}$ | 46.97 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.81}$ |
| LLaMA | 89.50 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 2.75}$ | - | 64.27 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 36.01}$ | -22.48 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 36.01}$ | 27.74 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 7.28}$ | -61.72 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 4.57}$ | 60.50 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 15.34}$ |
| SimpleCIL | 90.00 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.25}$ | - | 66.06 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 37.80}$ | -20.69 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 37.80}$ | 53.79 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 33.33}$ | -32.97 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 33.33}$ | 69.95 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 24.79}$ |
| GCN${}_{\text{LLMEmb}}$ | 81.83 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 4.92}$ | - | 39.31 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 11.05}$ | -42.44 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 11.05}$ | 25.58 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 5.12}$ | -61.17 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 5.12}$ | 48.91 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.75}$ |
| ENGINE | 91.10 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 4.35}$ | - | 34.17 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 5.91}$ | -56.93 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 1.56}$ | 21.73 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 1.27}$ | -69.37 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 3.08}$ | 49.00 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.84}$ |
| GraphPrompter | 83.67 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 3.08}$ | - | 52.22 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 23.96}$ | -31.45 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 23.96}$ | 32.25 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 11.79}$ | -51.42 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 14.87}$ | 56.08 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 10.92}$ |
| GraphGPT | 91.17 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 4.42}$ | - | 51.76 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 23.50}$ | -34.99 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 23.50}$ | 28.86 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 8.40}$ | -62.31 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 3.98}$ | 57.26 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 12.10}$ |
| LLaGA | 85.50 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 1.25}$ | - | 49.04 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 20.78}$ | -36.71 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 20.78}$ | 30.39 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 9.93}$ | -55.36 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 10.93}$ | 54.97 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 9.81}$ |
| SimGCL(Ours) | 86.67 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.08}$ | - | 74.53 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 46.27}$ | -12.21 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 46.27}$ | 71.40 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 50.94}$ | -15.06 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 50.94}$ | 77.54 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 32.38}$ |

To extend the evaluation scope of the proposed LLM4GCL framework to the domain-incremental learning (DIL) scenario, we conducted comprehensive experiments comparing various baseline methods with SimGCL under domain-sequential learning conditions. Specifically, each model was trained incrementally on the PubMed, CiteSeer, and Cora datasets in sequence. The experimental results are presented in Table [11](https://arxiv.org/html/2505.18697v2#A4.T11), yielding the following key observations:

Observation D.❷. LLM- and GLM-based methods demonstrate stronger domain-incremental learning capabilities compared to GNN-based approaches. In the domain-incremental learning (DIL) setting, most LLM-based and GLM-based methods consistently outperform training-from-scratch GNNs. This underscores the strong generalization ability of pre-trained LLMs and highlights their potential in addressing DIL challenges within graph-based learning.
However, we note that the performance margin of GLM-based models is narrower than that of LLaMA and SimpleCIL. We attribute this limitation to the architectural discrepancy between shallow GNNs and deep LLMs (e.g., GraphPrompter), as well as the misalignment between textual and graph structural representations (e.g., GraphGPT), which collectively impair the generative capability of these models.

Observation D.❸ SimpleCIL and SimGCL maintain substantial performance advantages over other baselines in the DIL setting. Experimental results show that SimpleCIL and SimGCL reach the two highest performances, exhibiting significantly less forgetting and greater performance gain compared to GCN, in contrast with all other baselines. These outcomes confirm that class prototypes effectively preserve the generalization capabilities of LLMs across diverse domains, while graph structural prompts enhance their ability to comprehend and adapt to unseen graph topologies.

### D.4 Time Efficiency

**Table 12: Time efficiency comparison of GNN-, LLM-, and GLM-based methods on Cora dataset. The asterisk (*) refers to the parameters that are only trainable during the initial session and remain fixed throughout subsequent sessions.**
| Method | Param. (MB) | Training (s) | Inference (s) | $\mathcal{\bar{A}}$ | $\mathcal{A}_{N}$ |
| --- | --- | --- | --- | --- | --- |
| GCN | 0.19 | 4.76 | 0.02 | 57.0 | 38.2 |
| EWC | 0.19 | 5.66 | 0.02 | 56.0 | 31.0 |
| LwF | 0.19 | 7.75 | 0.03 | 55.7 | 30.8 |
| Cosine | 0.19* | 1.94 | 0.04 | 65.4 | 45.2 |
| TPP | 0.20 | 57.70 | 0.07 | 45.7 | 13.7 |
| BERT | 0.29 | 56.67 | 1.30 | 56.0 | 29.9 |
| RoBERTa | 1.76 | 172.85 | 4.16 | 54.6 | 29.6 |
| LLaMA | 3.25 | 454.47 | 72.65 | 65.6 | 53.8 |
| SimpleCIL | 1.76* | 55.90 | 4.04 | 70.8 | 58.3 |
| GCN${}_{\text{LLMEmb}}$ | 0.53 | 5.18 | 0.03 | 59.1 | 31.1 |
| ENGINE | 3.19 | 89.70 | 0.11 | 59.2 | 31.3 |
| GraphPrompter | 10.65 | 805.15 | 99.34 | 61.9 | 46.8 |
| GraphGPT | 8.85 | 1247.46 | 584.72 | 55.5 | 31.6 |
| LLaGA | 22.04 | 688.78 | 425.59 | 58.2 | 30.2 |
| SimGCL(Ours) | 3.27* | 420.00 | 52.67 | 84.6 | 80.0 |

To more comprehensively assess the computational overhead associated with LLMs and GLMs and to better analyze the trade-off between time efficiency and performance, we compared the computational costs of SimGCL against all baseline methods on the Cora dataset. Results are provided in Table [12](https://arxiv.org/html/2505.18697v2#A4.T12).

The results demonstrate that SimGCL is more efficient than other LLaMA-based methods (LLaMA, GraphPrompter, GraphGPT, LLaGA) in terms of model size, training time, and inference time. While its computational cost exceeds traditional GNN methods, this is offset by its superior performance. Moreover, SimpleCIL also achieves competitive efficiency, presenting running-up performance with computational costs similar to the GNN-based method TPP. These results suggest that well-designed LLM approaches can balance performance and computational cost effectively.

### D.5 Visualization

To further investigate the effectiveness of SimGCL, we performed a visualization experiment comparing the embeddings and prototypes generated by three prototype-based methods: Cosine, SimpleCIL, and SimGCL. The results are presented in Figure [4](https://arxiv.org/html/2505.18697v2#A4.F4).

The results indicate that embeddings produced by SimGCL exhibit larger inter-class distances and smaller intra-class distances. Samples belonging to the same class are closely clustered around their respective prototypes, while distinct classes are separated by clear decision boundaries. In contrast, the embeddings generated by Cosine and SimpleCIL are noticeably more dispersed. In some cases, prototypes of different classes appear in close proximity or even overlap (as shown in the middle-bottom and top-left subfigures), suggesting that these methods may struggle to differentiate samples from different classes. These findings underscore the superior embedding capability of SimGCL, reflecting its enhanced ability to comprehend both graph structures and textual information.

Figure: Figure 4: Visualization of the learnt embeddings and prototypes across three prototype-based methods.
Refer to caption: x4.png

### D.6 Performance Evolution

To have deeper insights into the comparative learning dynamics, we present performance evolution curves in Figure [5](https://arxiv.org/html/2505.18697v2#A4.F5). The visualization demonstrates SimGCL’s consistent superiority over baseline models across multiple training sessions, underscoring its robust capability to maintain previously acquired knowledge while effectively assimilating new information.

Figure: Figure 5: Performance evolution of all models on Citeseer, Photo, Products, and Arxiv datasets.
Refer to caption: x5.png

### D.7 EWC with Various Backbones

**Table 13: Performance comparison of different backbones integrated with EWC on Cora, WikiCS, and Arxiv datasets. "w/o" and "w/" represent learning without and with EWC, respectively.**
|  |  | GCN | BERT | RoBERTa | LLaMA |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | Metric | w/o | w/ | w/o | w/ | w/o | w/ | w/o | w/ | SimGCL |
| Cora | $\mathcal{\bar{A}}$ | 57.0 | 56.0 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 1.00}$ | 56.0 | 55.5 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.50}$ | 54.6 | 55.1 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.50}$ | 65.6 | 62.1 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 3.50}$ | 84.6 |
| $\mathcal{A}_{N}$ | 38.2 | 31.0 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 7.20}$ | 29.9 | 30.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.10}$ | 29.6 | 30.0 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.40}$ | 53.8 | 49.4 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 4.40}$ | 80.0 |  |
| WikiCS | $\mathcal{\bar{A}}$ | 52.4 | 45.9 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 6.50}$ | 58.4 | 54.9 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 3.50}$ | 55.1 | 54.0 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 1.10}$ | 55.5 | 54.5 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 1.00}$ | 73.5 |
| $\mathcal{A}_{N}$ | 30.2 | 28.5 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 1.70}$ | 30.0 | 30.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.40}$ | 30.8 | 30.6 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.20}$ | 30.9 | 30.8 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.10}$ | 61.9 |  |
| Arxiv | $\mathcal{\bar{A}}$ | 54.9 | 55.3 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.40}$ | 26.0 | 24.1 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 1.90}$ | 24.4 | 24.3 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.10}$ | 24.9 | 23.8 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 1.10}$ | 59.9 |
| $\mathcal{A}_{N}$ | 34.9 | 33.0 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 1.90}$ | 9.7 | 1.3 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 8.40}$ | 1.4 | 1.4 ${\color[rgb]{1.0,0.5,0.5}\definecolor[named]{pgfstrokecolor}{rgb}{1.0,0.5,0.5}\uparrow 0.00}$ | 1.4 | 1.2 ${\color[rgb]{0.3,0.8,0.8}\definecolor[named]{pgfstrokecolor}{rgb}{0.3,0.8,0.8}\downarrow 0.20}$ | 33.8 |  |

To further investigate the rationale and efficacy of integrating LLMs with traditional continual learning methods, we conducted experiments combining GCN and three LLM backbones—BERT, RoBERTa, and LLaMA—with EWC. Detailed results are provided in Table [13](https://arxiv.org/html/2505.18697v2#A4.T13).

The results indicate that only the RoBERTa+EWC configuration on the Cora dataset yields a marginal performance improvement, suggesting the limited effectiveness of EWC when applied to LLMs. We attribute this to the extensive parameter space characteristic of LLMs: EWC relies on regularizing important parameters via the Fisher information matrix, which becomes computationally intractable at such a scale. Consequently, it is challenging to effectively constrain all critical parameters during continual learning, thereby reducing the method’s practical utility.

## Appendix E Broader Discussion

### E.1 Limitations

Restricted to node-level tasks.
Currently, LLM4GCL is limited to GCL for node classification tasks, as most GLM-based methods are designed for single-task scenarios, and node classification remains the predominant task in GML. However, GML encompasses other critical tasks such as link prediction and graph classification. Moreover, in practical applications, GCL has significant value in these task domains, like predicting interactions between existing and new users (edge-level class incremental learning, ECIL) and identifying novel protein categories (graph-level class incremental learning, GCIL). Notably, recent advances in Graph Foundation Models (Mao et al., 2024) have enabled Graph-enhanced LLMs like OFA (Liu et al., 2023a), UniGraph (He et al., 2024), and UniAug (Tang et al., 2024b) to support edge-level or graph-level tasks. Therefore, a comprehensive evaluation of GLM-based methods in ECIL and GCIL scenarios is imperative, and we designate this as one of the key directions for our future research.

Restricted to TAGs.
In LLM4GCL, all seven benchmark datasets are TAGs, as the evaluated GLM-based methods inherently depend on textual node attributes. However, this requirement limits applicability to real-world non-textual graphs (e.g., transportation networks or flight routes). Currently, emerging solutions like GCOPE (Zhao et al., 2024a) and SAMGPT (Yu et al., 2025) demonstrate promising approaches for text-free graph foundation models. Consequently, extending GLM-based methods to such non-textual graph scenarios remains a critical direction for future research.

### E.2 Further Discussion

Apply SimGCL to extreme scenarios. We observe that SimGCL exhibits relatively inferior performance on extremely sparse graphs, under very few-shot conditions, and in long session scenarios. We attribute this to its reliance on informative ego-graph prompts and extensive pre-training during the initial session. Sparse graphs provide limited structural context, which may result in less informative prompts and hinder the model’s graph comprehension. Furthermore, in FSNCIL settings, the substantial data disparity between the base session and incremental sessions introduces a bias toward base classes, increasing the risk of overfitting.

Nonetheless, SimGCL represents a flexible framework that can be adapted to these challenging settings through targeted improvements: for sparse graphs, integrating lightweight graph encoders such as SGC (Wu et al., 2019) could enhance higher-order structural awareness; for few-shot learning, session-aware fine-tuning via LoRA (Hu et al., 2022) with momentum updates (He et al., 2020) could mitigate overfitting to the base session; for long sessions, prototype calibration methods (Li et al., 2025a) may help maintain stability across old and new classes.

In summary, although SimGCL shows certain limitations in highly challenging environments, it successfully provides a versatile foundation for graph continual learning, as its performance shortcomings can be alleviated through context-aware enhancements.

Generalization of GLM-based Methods. Current GLM-based methods generally demonstrate unsatisfactory performance compared to pure LLM-based methods, facing challenges in GCL tasks, yet their superior generalization and contextual understanding capabilities - surpassing shallow GNNs - show significant potential for complex GCL scenarios when properly implemented. For example, SimpleCIL outperforms most GNN and LLM baselines by leveraging class prototypes, fully utilizing LLMs’ generalization while reducing overfitting; similarly, SimGCL significantly outperforms existing models by being trained with informative graph structural prompts, establishing new SOTA performance across multiple datasets. Consequently, we believe that this paper demonstrates a promising direction for leveraging LLMs/GLMs in GCL, laying the groundwork for unlocking their full potential in this emerging research direction.

### E.3 Potential Impacts

The integration of LLMs with graph tasks represents an emerging and highly promising research direction, demonstrating substantial potential across diverse applications and particularly in GCL scenarios. This paper proposes LLM4GCL to advance research focus on this novel paradigm that leverages graph-enhanced LLMs for improved GCL performance. Through LLM4GCL, we conduct a systematic investigation into graph-enhanced LLM methodologies and establish comprehensive benchmarking across various graph domains. We believe this work will serve as the catalyst and pioneer for accelerated progress in this developing research community.

Moreover, as LLM4GCL demonstrates several efficient LLM- and GLM-based GCL methods, we argue that these advancements hold significant societal value. Specifically, the proposed methods facilitate efficient knowledge updates in dynamic environments, such as social networks and recommendation systems, through seamless integration with LLMs, thereby substantially reducing computational overhead compared to conventional training-from-scratch paradigms. Furthermore, the cross-domain adaptability of these models offers distinct advantages in mission-critical applications, including healthcare diagnostics and financial forecasting. In such domains, continuous model adaptation to emergent knowledge, ranging from novel therapeutic interventions to evolving market trends, can be achieved while effectively mitigating catastrophic forgetting. However, practical deployment necessitates careful consideration of several key challenges, including data privacy implications, the potential for bias propagation in automated decision-making processes, and ensuring robustness against adversarial data injection, particularly in open-domain applications.

In the future, we will keep track of newly emerged techniques in the GCL field and continuously update LLM4GCL with more solid experimental results and detailed analyses.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: Q1. Clarification on the mechanism of forgetting alleviation
You claim that SimGCL “greatly alleviates catastrophic forgetting” through training-free prototype classification. Could you provide concrete evidence (e.g., forgetting curves, representation drift analysis, or embedding similarity metrics) showing that forgetting is truly reduced, rather than simply avoided by freezing parameters?

Q2. Computational efficiency validation
The paper repeatedly highlights SimGCL’s “efficiency,” but no runtime or GPU memory statistics are presented. Could you quantify the actual training and inference cost (in FLOPs, GPU hours, or wall-clock time) compared to other LLM-based and GNN-based baselines?
soundness: 2
strengths: 1. Critical Re-evaluation of GCL Benchmarks:
The paper makes a valuable methodological contribution by identifying and empirically demonstrating a task ID leakage issue in existing Graph Continual Learning (GCL) benchmarks. This flaw—previously overlooked in the community—renders many reported results unreliable. By introducing a corrected global testing setup, the authors establish a fair and realistic evaluation framework for future GCL research.

2. Bridging GCL and Foundation Models:
Conceptually, the paper establishes an important connection between graph continual learning and pretrained foundation models. By demonstrating that LLMs can serve as effective continual learners on graph-structured data when appropriately prompted, the study opens a new research direction for integrating graph reasoning with large-scale language models.

3. Comprehensive and Insightful Experimental Analysis：
The paper provides a detailed and balanced experimental study, including comparisons among GNNs, LLMs, and GLMs across diverse datasets and settings. The analysis offers clear empirical insights—such as why GLMs may underperform and how prototype-based designs and model scaling influence continual learning performance—making the work informative beyond its own method.
summary: This paper presents a systematic and timely investigation into the potential of large language models (LLMs) to mitigate catastrophic forgetting in graph continual learning (GCL). They introduce a more rigorous global testing setup and develop LLM4GCL, a comprehensive benchmark evaluating nine LLM-based and graph-enhanced LLM (GLM) methods across seven text-attributed graph datasets under both node-level class-incremental (NCIL) and few-shot NCIL (FSNCIL) settings. They propose SimGCL, a simple yet effective GLM-based approach that combines graph-structured prompting, LoRA-based instruction tuning, and training-free prototype classification. This method enables LLMs to capture graph structural information while maintaining strong generalization and avoiding parameter updates that induce forgetting. Extensive experiments demonstrate that SimGCL consistently outperforms existing GNN-, LLM-, and GLM-based baselines—achieving up to 20% higher accuracy under rehearsal-free conditions.
weaknesses: 1. Limited Theoretical Justification for SimGCL:
While SimGCL shows strong empirical results, the paper provides little theoretical or analytical grounding for why the combination of graph prompts, LoRA fine-tuning, and prototype-based classification alleviates forgetting. The approach appears largely empirical, and the mechanism behind its robustness (e.g., whether prototype stability or prompt alignment is the key factor) is not formally analyzed. Adding theoretical reasoning or ablation-based evidence would strengthen the scientific depth of the contribution.

2. Overstatement of SimGCL’s generality and simplicity:
The paper claims that SimGCL “greatly alleviates catastrophic forgetting” and “surpasses all existing baselines” under a rehearsal-free constraint. While the reported results do show strong gains (up to 20%), these improvements are dataset-dependent — performance drops significantly on sparse or long-session datasets (e.g., Arxiv-23, FSNCIL). Hence, calling it universally effective may be an overstatement; the method is empirically strong but not universally superior.

3. Assertion of “efficiency” and “low cost”:
The authors emphasize SimGCL’s efficiency due to LoRA-based tuning and prototype inference. However, they provide no runtime, memory, or scaling benchmarks to substantiate this claim. Given that LLM fine-tuning is resource-intensive, this efficiency claim seems qualitative and not empirically validated, representing a mild overstatement.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: -Could the authors clarify and provide more details on the proposed SimCML model and the LLM baselines, particularly how these baselines are applied in the context of GCL?

-Could the authors provide more information on how the hyperparameters were selected and the validation protocol used? Additionally, were the experiments repeated to assess the stability and statistical significance of the results?

-Considering the goal of establishing a benchmark for GCL, could the authors comment on the choice of datasets and whether including large-dimensional datasets or heterophilous/homophilous distinctions might affect the evaluation?

-Could the authors discuss the rationale for not including certain replay-based approaches, such as CaT, PDGNN, and SSM, and how their inclusion might impact the evaluation of SimCML?
soundness: 2
strengths: The paper explores a novel approach that connects LLMs with the graph continual learning framework. Moreover, it analyzes the commonly used GCL scenario, revealing an interesting limitation and proposing a solution to address this issue.
summary: The paper proposes an extension of the Online Graph Learning problem formulation, aiming to address some of the limitations typically found in existing literature. Based on this revised formulation, the authors define two scenarios: a standard setting and a few-shot setting. They evaluate both conventional graph-based approaches and LLM-based methods. Finally, the paper introduces SimCML, a model that leverages a single round of instruction tuning to directly produce class prototypes from the embeddings generated by its fine-tuned LLM.
weaknesses: The paper presents several issues, mainly related to the clarity of presentation and the design of the experimental framework.
Regarding the presentation, the paper does not clearly explain the proposed model or the models used for comparison. As a result, many methodological choices are not properly justified. Going into more detail, in the introduction, the authors claim that there is a lack of investigation into the rationality of the common experimental setup used for GCL. It is worth noting that a recent study has already explored this aspect [1]. Another meaningful reference that should be considered and discussed is [6].
In the background section, the authors correctly highlight one of the main limitations of the commonly adopted GCL setting, which I agree with, and propose the idea of global testing. However, it is not clear how this proposal relates to the subsequent discussion about inter-task edges, which the authors claim not to consider in their tasks. This choice isolates subgraphs, making the problem closer to the original formulation. This point should be discussed and clarified in greater depth. Furthermore, the removal of the limited-representation classes appears to be a strong design choice that requires a more thorough discussion, particularly regarding its advantages and potential drawbacks.
In Section 3.2, the authors claim that the considered datasets cover a wide range of possible sizes. However, in practice, the investigation is limited to relatively small datasets. Large-dimensional datasets, such as OGN-Products and Reddit [2], are not considered. Moreover, most of the selected datasets are citation networks. Considering that the goal of the paper is to establish a benchmark for testing models in GCL, the dataset selection appears rather limited. Another important distinction that can significantly impact the GCL scenario is the difference between heterophilous and homophilous datasets, which is not addressed in this work.
In the paper, both the definitions of the literature methods and that of the proposed SimCML are not presented clearly, which makes it difficult to follow the flow of the discussion and to evaluate the novelty of the proposed approach. For instance, the discussion of the LLM baseline is very limited, the authors only state that it is nascent and that no methods have yet been specifically designed for CGL tasks. What remains unclear is how the LLM baselines are actually applied in the context of CGL, and consequently how the reported results are obtained. Regarding the models considered, it is surprising that the authors do not include several of the most promising replay-based approaches, such as CaT [3], PDGNN [4], SSM [5], among others. Another major issue is the lack of explanation regarding the experimental setting and the validation policy. While the appendix lists the hyperparameters considered, many of them are fixed, and it is not clear how these values were selected. Considering the aim of the paper, this represents a significant limitation. Moreover, the results reported in the tables do not include any measures of variance or standard deviation, raising concerns about whether the experiments were run only once. If this is the case, it is unclear how the authors assess the stability of the results or the statistical significance of the differences between models. This is particularly important for the LLM application, where variance in results is typically more pronounced.


[1] Donghi, G., Pasa, L., Zambon, D., Alippi, C. and Navarin, N., 2025. Online Continual Graph Learning. arXiv preprint arXiv:2508.03283.

[2] Zhang, X., Song, D. and Tao, D., 2024. Continual learning on graphs: Challenges, solutions, and opportunities. arXiv preprint arXiv:2402.11565.

[3] Liu, Y., Qiu, R. and Huang, Z., 2023, December. Cat: Balanced continual graph learning with graph condensation. In 2023 IEEE International Conference on Data Mining (ICDM) (pp. 1157-1162). IEEE.

[4] Zang, X., Song, D., Chen, Y. and Tao, D., 2024, August. Topology-aware embedding memory for continual learning on expanding networks. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 4326-4337).

[5] Zhang, X., Song, D. and Tao, D., 2022, November. Sparsified subgraph memory for continual graph representation learning. In 2022 IEEE International Conference on Data Mining (ICDM) (pp. 1335-1340). IEEE.

[6] Huang, S., Parviz, A., Kondrup, E., Yang, Z., Ding, Z., Bronstein, M., Rabbany, R. and Rabusseau, G., 2025. Are Large Language Models Good Temporal Graph Learners?. arXiv preprint arXiv:2506.05393.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: N/A
soundness: 2
strengths: 1. The paper tackles an underexplored question—whether large language models (LLMs) can mitigate catastrophic forgetting in graph continual learning.
2. A benchmark, LLM4GCL, is developed.
summary: This paper investigates whether large language models can alleviate catastrophic forgetting in graph continual learning (GCL). The authors identify flaws in existing GCL evaluation protocols—particularly task ID leakage—and propose a revised global testing setup. They introduce LLM4GCL, a new benchmark encompassing multiple GNN-, LLM-, and GLM-based methods across seven text-attributed graph datasets, and propose SimGCL that combines graph-structured prompts with prototype-based prediction. Experimental results indicate that SimGCL performs competitively under various GCL settings, suggesting that pretrained LLMs have potential for continual graph learning.
weaknesses: 1. My biggest concern lies in the motivation. The primary goal of continual learning is to enable efficient adaptation under limited resources, whereas LLMs are inherently computationally expensive. It is unclear whether the significant computational cost introduced by LLMs can truly justify the efficiency-driven learning protocol that continual learning aims to achieve.

2. Following this point, I believe the comparison should include GCL methods that allocate additional parameters (expansion-based) or memory (experience-replay-based) to narrow the substantial computational gap between LLMs and traditional GNNs. However, the authors compare their method only with regularization-based approaches, which are the most resource-limited class of GCL methods. Furthermore, the comparisons are arguably unfair since the methods rely on different backbone architectures for graph prediction.

3. The authors acknowledge task ID leakage and address it by introducing a global testing protocol, which is commendable. Nonetheless, all the datasets used (e.g., Cora, Citeseer, WikiCS) are widely used text-attributed benchmarks that may overlap with the pretraining corpora of LLMs. The paper does not verify whether such overlap exists, leaving potential knowledge leakage from pretraining unaddressed.

4. I also find the fairness of the LLM4GCL benchmark setup questionable. The LLM baselines are not fine-tuned on the same datasets and therefore lack domain knowledge of LLM4GCL, making the comparison asymmetric. The proposed method appears less focused on mitigating catastrophic forgetting and more on task-specific instruction tuning, which may even result in overfitting to the given dataset. This is evidenced by the large performance gap between the proposed method and other LLM-based baselines on LLM4GCL, in contrast to the much smaller gap observed on standard public datasets. Overall, the comparisons and evaluations in this paper feel ambiguous, making it difficult to discern the central message or takeaway the authors intend to convey.

---

confidence: 5
contribution: 2
initial_rating: 4
presentation: 3
questions: Please see the weaknesses.
soundness: 3
strengths: 1. The paper is well-written and easy to follow. The code is released.

2. A new benchmark is proposed to evaluate the performance of LLMs and graph-enhanced LLMs.

3. The authors identify the flaw of existing GCL settings and propose global testing. To utilize the generalization capacity of LLMs, a GLM-based approach is proposed.
summary: This paper focuses on the problem of graph continual learning and proposes a method based on LLMs. Different from the previous settings, the authors provide a new setting called global testing and construct a benchmark. Based on the benchmark, a new method based on LLMs and prototype learning is proposed.
weaknesses: 1. The constraint of setting each task to have a unified sample size is unreasonable, and the label imbalance problem is the real challenge that should be addressed.

2. The introduction of global testing is not very clear. More clear figures or visualizations are needed.

3. The used graphs are all text-attribute graphs (TAGs). Can the proposed method be extended to non-TAGs?

4. Experimental details should be included in the main paper to provide readers with a general picture of the experimental settings.

5. There are some typos, and the authors should carefully revise the paper.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Please refer to the weakness section. I would urge the authors to pay special attention to the following concerns first.

- The work is conceptually sound and empirically motivated but not theoretically grounded. The rationale aligns with existing literature, but lacks formal guarantees. Can the authors provide some theoretical justifications to their method?

- Empirical claims are mostly well-supported, though some causal explanations (e.g., why GLMs fail) remain conjectural. Therefore, I request the authors to substantiate such claims.

- Address limited ablation analyses and potential structural oversimplifications when converting graph topology into text prompts.
soundness: 3
strengths: ### 1. Theoretical Soundness
- The work is well motivated as the authors have identified that all prior GCL methods train from scratch and do not exploit pretrained models’ generalization.

- Uses Parameter-Efficient Fine-Tuning (PEFT) via LoRA, a proven method for adapting large models efficiently.

- Employs prototype-based classification, which is widely accepted as a way to mitigate forgetting in continual learning.

- The graph prompt design is conceptually consistent with instruction tuning paradigms in multimodal LLMs.

### 2. Experimental Soundness
- The paper introduces LLM4GCL, with 9 LLM/GLM methods and 7 datasets. This is a substantial empirical foundation for a GCL.

- The identification of task ID leakage in existing setups is a major contribution.

- They release code, datasets, and standardized evaluation. This is a strong signal of methodological transparency.
summary: This paper investigates whether large language models (LLMs) can help overcome catastrophic forgetting in Graph Continual Learning (GCL) tasks. 

- They have identified a serious flaw, task ID leakage, in common GCL evaluations, where models inadvertently access task identifiers during testing.

- They introduce LLM4GCL, the first systematic benchmark for LLMs in GCL.

- Proposed a method, SimGCL, that consistently achieves state-of-the-art results and mitigates forgetting effectively.
weaknesses: ### 1. Theoretical Limitations
- They assume graph structure can be effectively encoded as text prompts. While innovative, this verbalization step might oversimplify structural relationships, potentially losing fine-grained topology information.

- Use of LoRA only in the first session assumes that continual adaptation can be achieved purely through frozen embeddings and prototype updates — a strong assumption that may not generalize across highly dynamic graph evolutions.

- Missing formal theoretical analysis of why the prototype mechanism preserves generalization in LLM-embedded graph representations (e.g., no bounds on forgetting or feature drift).

### 1. Experimental Limitations
- The authors do not deeply dissect which component of SimGCL contributes most (LoRA, prototype, or prompt design). Without that, causal claims (“graph prompts mitigate forgetting”) remain partially speculative.

- In small-scale datasets like Cora and Citeseer, very large LLMs might overfit to textual node attributes, inflating reported improvements.


Decision: Reject