Self-Evolving Pseudo-Rehearsal for Catastrophic Forgetting with Task Similarity in LLMs
Authors: Jun Wang
Liang Ding
Shuai Wang
Hongyu Li
Yong Luo
Huangxuan Zhao
Han Hu
Bo Du
Venue: NeurIPS 2025 poster ()
Keywords: Continual Learning; Large Language Models; Pseudo Sample Rehearsal; Similarity Regularization
Primary area: deep_learning

Abstract: Continual learning for large language models (LLMs) demands a precise balance between $\textbf{plasticity}$ - the ability to absorb new tasks - and $\textbf{stability}$ - the preservation of previously learned knowledge. Conventional rehearsal methods, which replay stored examples, are limited by long-term data inaccessibility; earlier pseudo-rehearsal methods require additional generation modules, while self-synthesis approaches often generate samples that poorly align with real tasks, suffer from unstable outputs, and ignore task relationships. We present $\textbf{\textit{Self-Evolving Pseudo-Rehearsal for Catastrophic Forgetting with Task Similarity}}(\textbf{SERS})$, a lightweight framework that 1) decouples pseudo-input synthesis from label creation, using semantic masking and template guidance to produce diverse, task-relevant prompts without extra modules; 2) applies label self-evolution, blending base-model priors with fine-tuned outputs to prevent over-specialization; and 3) introduces a dynamic regularizer driven by the Wasserstein distance between task distributions, automatically relaxing or strengthening constraints in proportion to task similarity. Experiments across diverse tasks on different LLMs show that our SERS reduces forgetting by over 2\% points against strong pseudo-rehearsal baselines, by ensuring efficient data utilization and wisely transferring knowledge. 
The code will be released at https://github.com/JerryWangJun/LLM_CL_SERS/.

Self-Evolving Pseudo-Rehearsal for Catastrophic Forgetting with Task Similarity in LLMs
Jun Wang Liang Ding Shuai Wang' Hongyu Li
Yong Luo't
Huangxuan Zhao'
Han Hu Bo Du
School of Computer Science, National Engineering Research Center of Multimedia Software and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University. Wuhan, China
The University of Sydney, Australia IIndividual Researcher
*School Information and Electronics, Beijing Institute of Technology, Beijing, China { juunwag_ai, langshuai123, luoyong, zhaohuangxuan, dubo)@@hu. 1e@@h.eedu edu. (liangding.. liam, hongyul11027999)@mamll com hhu@bit... edu.
Abstract
Continual learning for large language models (LLMs) demands precise balance between plasticity- - the ability to absorb new tasks and stability the preservation of previously learned knowledge. Conventional rehearsal methods, which replay stored examples, are limited by long-term data inaccessibility; earlier pseudo- rehearsal methods require additional generation modules, while self-synthesis approaches often generate samples that poorly align with real tasks, suffer from unstable outputs, and ignore task relationships. We present Self-Evolving Pseudo- Rehearsal for Catastrophic Forgetting with Task Similarity (SERS), a lightweight framework that 1) decouples pscudo-input synthesis from label creation, using semantic masking and template guidance to produce diverse, task-relevant prompts without extra modules; 2) applics label self-evolution, blending base-model pri- with fine-tuned outputs prevent wer-specializztion; and 3) introduces dynamic regularizer driven by the Wasserstein distance between task distributions, automatically relaxing strengthening constraints in proportion task similarity. Experiments across diverse tasks on different LLMs show that our SERS reduces forgetting by over 2% points against strong pseudo-rehearsal baselines, by ensuring efficient data utilization and wisely transferring knowledge. The code will be released htttps: peaetts://githhie wtt.huu..oo//eeeraaannnuun Lttp:/unnLLMMMCS_SEEERE CL_SERS//
Introduction
Enabling large language models (LLMs) acquire new knowledge continuously (Wu al. 2024 Zheng et al. 2025b) holds significant importance for developing artificial intelligence systems with lifelong learning abilities. While practical applications demand LLMs continually adapt to evolving downstream tasks, conventional learning methods (Hu et al. 2022 Han et al. 2024) often struggle to preserve existing capabilities during such situations. Continual learning enables LLMss flexibly integrate new and existing knowledge tasks increase, addressing the limitations of static training preserving prior performance while incorporating new information. The core challenge lies in achieving optimal balance between plasticity and stability (Mermillod et al. (013).. Excessive plasticity will result in catastrophic forgetting, whereas overly strong stability may prevent efficient and effective knowledge transfer.
Egqul contribution Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

60
Real Samples Synthess
QA
Our Methed
SA SUM
40
TRANS
0-
-20
-40
-60
-60
440
-20 0
20 40
60
Figure 1: Clustering analysis of pseudo samples generated by our method (SERS) and Self- Synthesized Rehearsal (SSR) approaches across five tasks, alongside real samples. can be observed that the pseudo samples generated by our method are closer to the real samples than SSR, indicating that SERS produces more similar pseudo samples that better reflect knowledge from previous tasks.
series of works (Zhao et al. 2024 Zheng al. 2025a Wang et al. 2024a; (unn and Gao 2024
have been proposed to mitigate this balancing challenge. Rehearsal-basee methods (Yin et al. 2022
de Masson D A'Atum
2019 Rolnick al. 2019) preserve model capabilities on previous tasks by utilizing real samples from prior training processes, which are not always consistently available in practice. To tackle the challenge of limited access to get historical data, existing solutions (Sun et al.) 2020; Zhao et al. (024) apply pseudo-sample generation, yet the additional generation modules increase the number of trainable parameters, [Huang al. (2024) leverage in-context learning capacity of L.MSs for self-synthesis rehearsal, effectively alleviating parameter burdens. We unexpectedly found that self-synthesized samples often exhibit low similarity to real data, failing to adequately reflect the knowledge structure and thus undermining the effectiveness of rehearsals,
shown
by the clustering analysis in Figure [ Regularization-basel approaches (Guo et al., 2024 Wang et al. 2023) impose constraints on loss functions to penalize parameter updates that affect prior lask knowledge. However, traditional static constraint methods, with their fixed trade-off between facilitating knowledge transfer and preventing forgetting, lack consideration for task diversity.
To generate pseudo samples that better support knowledge consolidation during rehearsal, while flexibly balancing knowledge transfer and forgetting prevention across tasks, we propose Self- Evolving Seeudo-Rehearse with Task Similarity (SERS). Specifically, we generate pseudo inputs using template guidance and semantic masking, eliminating task-specific instructions, where dynamic guidance and mask ratios ensure the diversity. After generating the pseudo inputs, to supplement pseudo labels, over-specialized samples are selected via label self-questioning and ease the demand for task-specifi, knowledge through label self-evolution. In the rehearsal stage, to fully promote permissible knowledge transfer, we design regularization loss function based on task similarity. When tasks are similar, the regularization is relaxed to encourage the integration of new and old knowledge; otherwise, constraints strengthened to alleviate the forgetting of previous knowledge.
We conducted extensive experiments on the SuperNI dataset (Wang et al. 2022) using LLaMA2-7B (Touvron et al., 2023) and ChatGLM-6B (GLM et al.) 2024) to evaluate the per- formance of SERS across varying task chains. Results show that SERS consistently outperforms existing methods and is more stable across variety of task orders. On LLaMA2-7B, it achieved 2. 2.16% relative improvement over advanced pseudo-sample rehearsal approaches, closely matching Multi-Tase Task Learning (MTL) performance; on ChatGLM-6B, it even surpassed MTL.
The main contributions of our work are as follows:
We propose SERS, a continual learning framework for L.MSs that decouples input and label synthesis. SERS generates pseudo inputs via template guidance and semantic masking and uses

---

label self-evolution module to prevent over-specialization in pseudo labels, brings human learning strategies into machine learning.
We introduce task similarity-ban based dynamic regularization to effectively balance stability and plasticity, reducing the sensitivity of knowledge transfer to task order.
Experiments show that SERS significantly improves learning accuracy under muskiincrementa conditions, alleviates catastrophic forgetting, and even facilitates additional knowledge transfer.
2 Related Work
2.1 Self-Evolution Learning
Self-evolution (Zhong et al., 2022 Peng et al. 2023 Zhong et al. 2023 Zheng et al. 2023 Tao
jet al., [2024] Song et al. (025) is paradigm that enables models to learn and improve through self-generated knowledge, inspired by human learning from experience. In this process, LLMs create new tasks and solutions based on predefined goals, collect feedback from the environment, refine the acquired experience eliminate errors, and update their parameters context accordingly.
Zhong al. (2022) improve pretraining efficiency through a two-stage process of eeff-questioning and self-evolution. In the first stage, the model uses masking to detect tokens it struggles to understand; in the second, it generates soft labels with richer knowledge patterns to enhance training. Similarly, Singh et al. (2023) apply reinforcement learning actively generate using a binary reward function, and select high-quality data for model updates.
samples, evaluate them
Motivated by these strategies and aiming to address the instability of -seude generation in LLMs, we propose label-level self-evolution method. By imitating self oue-suusti and self-evolution structure, our approach detects over-specialized pseudo labels and smooths them with general knowledge, mitigating the local overfitting to a specific task caused by rehearsal.
2.2 Continual Instruction Tuning for LLMs
Continual instruction tuning for LLMs extends traditional LLMs tuning by enabling LLMs to incrementally absorb new tasks and feedback without forgetting prior knowledge. Compared to standard continual learning, it introduces unique challenges due to generative outputs, global semantic relationships, and model scale. Existing methods can be broadly categorized into:
(1) Architecture- based (Ren et al.) 2024 Zhao et al., 2024 Ke et al., 2023): These methods adjust model architecture or parameter distribution separate the knowledge of new and old tasks, thus mitigating forgetting caused by parameter interference. For example, Ren et al. (2024) use fast and slow learners to balance stability and plasticity. /Zhao et al. (2024) introduce Attentive Learning & Selection module by combining multiple PET blocks in different ways to fit different tasks. However, as task numbers increase, adding new modules raises computational costs, and separate architecture adjustments limit flexibility and universality.
(2) Rehearsal-baat methods (Wang et al. 2024b Huang al. 2024 Maekawa et al. 2023 These methods involve real or pseudo-sample rehearsal. Real sample rehearsal, as in Wang et al. (2024b), helps recall past knowledge but relies access to original data during each training stage, which is often impractical. Pseudo-sample rehearsal typically necessitates an extra generation module, increasing trainable parameters. To our knowledge, [Huang al. (2024) are the first use self- synthesis to generate pseudo samples from a few real samples, solving storage issues, but still facing challenges with pseudo samples instability and task comprehension.
(3) Regularization-be based methods (Wang et al. 2023 Jin et al., 2021 Li et al. 2024 Guo et al.
2024): : These methods constrain excessive parameter updates with regulan regularization. For example, Wang et al. 2023) learn tasks different low-rank vector subspaces and keep these subspaces orthogonal minimize interference. However, orthogonal subspaces limit the knowledge transfer between tasks.
Our method combines pseudo-sample rehearsal and regularization. For pseudo-sample generation, we leverage template guidance and semantic masking to ensure the stability and real-sample similarity of synthesized pseudo samples, while varying templates and masking ratios promote diversity. For regularization, dynamically adjust the regularization strength based on task similarity and account

---

Semantic-Guided Pseudo Input Generation
Rehearsal with Similarity Regularization
L.M
Xp
Task Similarity W
Dx
Label Self-Evolution
Lreg
D
LLM
X
(X vD
L.M
LLM
LLM
D
Figure 2: The overall framework of our SERS method. In the Semantic-Guided Pseudo-Input Generation stage, small set of real samples produces pseudo inputs X Then, the Label Self- Evolution module refines these inputs by integrating knowledge from LLM and LLM', yielding rehearsal pseudo samples (X,,, Y;)). Finally, in the Rehearsal with Similarity Regularization stage, the rehearsal samples and the training data Dé+1 are combined for fine-tuning, with regularization applied based on task similarity.
for the impact of task order parameter updates, effectively balancing knowledge transfer and resistance to catastrophic forgetting.
Methodology
3.1 Problem Definition
We consider the problem Task Incremnall Continual Instruction Tuning. Given a sequence N follloctoonfollooinng tasks Tı, T2. ....... TN, each associated with dataset Di (X', Y*), the goal is to continually fine-tune a pre-trained language model LLMo on these tasks sequence. At each step i, the model receives only the current task dataset D, and fine-tunes the model LLMi-1 to obtain LLM'. The objective is to learn each new task while maintaining performance on all previous tasks, without requiring large-scat scale retraining.
3.2 Framework Overview
In this paper, we propose a continual learning framework for L.LMs that combines pseule-sample rehearsal with regularization. As shown in Figure 2 our approach consists of three main compo- nents: semantic-guide pseudo-input generation, label self-evolution, and rehearsal with similarity regularization. In the following sections, we provide detailed explanation of each module.
3.3 Semantic-Guided Pseudo-Input Generation
The self-synthesis approach proposed by Huang al. (2024) effectively addresses the limitations discussed above, but still faces key challenges: the generated pseudo samples cannot well reflect the original knowledge structure and thus provide limited support for rehearsal. Additionally, appending task instructions and labels increases the model's comprehension burden, while the generated labels lose meaning after further refinement. To address these problems, we propose Semantic-Guided Seeuoo-Inpu Generation module. As shown in Figure 3 real examples are masked in two roles: as Example Template that providing structure guidance, and as Semantic Guidance that offering semantic context. Experiments in Wang et al. (2024c) indicate that models not fine-tuned on specific task have stronger contextual understanding. so that we use LLMM² to fill in the masks to get pseudo inputs X without extra generating block. Varying mask ratios and example templates enhance diversity, while removing task instructions and labels reduces cognitive load. Figure I shows clustering of pseudo samples from method, the self-synthesis approach, and real data. Our pscudo

---

(a) Semantic-Guided Pseudo-Input Generation
Example Template
Semantic Guidance
Generated Pscudo Input
What is the capital of <musk>?
What the largest <mask>
What largest solar system?
What capital of France?
our solar system??
What largest object system?
When and <mask> was <mask>> bomn? When and where was Finstein hom?
What is largest <mask> smask>??
LLM
is largest Earth?
What is the largest largest country in Africa?
(b) Label Self-Evvlution Stagel: Self- self-Questioning
Stage2: Self-Evolution
Top k% differences
2
+(1 - αc)
L.M
Xp expertion
L.M
I.M
x
Others
D' (x.. r
LLM
X potters
L.M
Oper-Speed Input:
The biggest occan is the one between America and Asia
Label Self-Evolution
What largest ocean Earth?
LLM
The Pacific Occan is
Earth's largest
The largest ocean is the Pacific Occan with 165,250,000 kmm
LLM
Figure 3: Detailed illustration of the core modules. (a) Semantic-Guided Pseudo-Input Generation: real sample and its masked version serve as an example template, while an additional masked sample provides semantic guidance. These combined and passed through LLM" to generate pseudo inputs. Varying combinations and mask ratios promote diversity. (b) Label :elf-Evolution: (Top) In self-questioning stage, the top-k% pseudo inputs with high domain dependence are identified as over-specialized samples. In self-evolution stage, these are relabeled by blending knowledge from LLM and LLM'; others directly use the output of LLM' labels. (Bottom) An example shows that an over-specialized input leads to poor output from LLM" and an overly detailed output from LLM After label self-evolution, the final label becomes more acceptable, with reduced reliance on domain-specific knowledge.
samples are more similar to real ones, preserving knowledge structure while maintaining diversity. Examples in real-task settings shown in Appendix A
3.4 Label Self-Evolution
Considering that randomness in pseudo-input generation can lead to over-specialized instances requiring excessive expertise, rehearsing with such samples may cause large parameter shifts and disrupt existing knowledge. We therefore introduce label self-evolution method inspired by human review. As shown at the top of Figure 3] the process consists of two stages: ceff-questioning and self-evolution. In self questiuust,,,, both the base model LLM and fine-tuned model L LLM' generate labels. Samples with the top-k output differences are treated as over-specialized. evolution stage, regular samples adopt LLM'ss outputs as labels, while over-specialized samples are relabeled by integrating the outputs from both models using a weighted combination. The coefficient adjusts the contributions of LLM" and LLM' to balance general and task-specific knowledge.
the self-
5

---

As shown in the lower part of Figure 3 over-specialized inputs produce vague outputs on LLM and highly specific ones on LLM'. The label self-evolution module merges these to produce acceptable labels, mitigating overfitting during rehearsal. Examples in Figure illustrate the effectiveness and reliability of this process on real tasks.
3.5 Rehearsal with Similarity Regularization
This section explores how task similarity, reflected through task order, influences training results. After generating pseudo samples avoiding over-specialization, pseudo samples are used for rehearsal training. At the training stage of T the model is updated using, pseudo samples of old tasks D2 DE...--, D2 and new task training data D*+1, Previous works (Huang et al. 2024 Zhao al. 2024
fine-tune using LoRA (Hu et al. 2022) without adapting to task characterisues, making the results highly sensitive task order. Since the model already contains knowledge from earlier tasks, similar new tasks should allow more knowledge transfer, whereas dissimilar ones require stronger constraints maintain prior knowledge. To achieve this, design a regularization loss based on task similarity, which is incorporated into the original cross-entropy loss to adjust LoRA fine-tuning:
(1) where Lce is the standard cross- cros-entropy loss, À controls regularization strength, and Lreg is the regularization term, balancing knowledge sharing and parameter stability. Specifically, we impose the regularization constraint on all training target parameters during optimization, as formulated in Equation
L L + X Lreg,
1 Σ EE [[0,]]]] 2
Treg =
We modulate regularization strength based on task similarity W and rehearsal ratio Treplay. When Treelay is low or W is large, indicating limited rehearsal or low task similarity, stronger constraints are applied to stabilize knowledge retention. In contrast, higher Treplay or smaller W suggests task alignment, allowing more relaxed regularization to facilitate knowledge transfer. This dynamic adjustment enables SERS maintain a balance between stability and plasticity, supporting more robust continual learning. Accordingly, À is formally defined in Equation 3
[Amin + (Amax >mii) (1 (1- wi ] (1 Treelay)) )
(3)
where Amin and Amax control the range of regularization strength. and Wth adjusts the curvature of the scaling function. The Wasserstein Distance (Chen el al. 2022 Liu et al., 2025), a representative of the optimal transport framework (Alvarez-Melis and Fusi] 2020), provides metric for assessing the similarity between the distributions of two datasets, which is defined as in Equation 4
W(P.Q) = inf B(x,))~ ||| - Yl]..
TEll(P.Q)
4 Experiment
4.1 Dataset and Metries
Our experiments are conducted on the SuperNI (Wang et al. (2022) dataset, a large and comprehensive benchmark for instruction tuning. For fair comparison, we adopt the same ten tasks as Huang et al.) (2024). divided into two groups: one with five tasks and the other with ten. Each group is evaluated under three different task orders. Experiments were carried out on LLaMA2-7B
et al. 2023)
and ChatGLM-6Bole (B(G et al. 2024). For more details about the dataset, please refer to Appendix
We adopt the Rouge-L score LLn 2004) assess generation quality, where R2 denotes the model's performance on task j at stage i. To evaluate overall performance, knowledge transfer ability, and retention of prior knowledge, the following commonly used continual learning metrics selected:
(1) Average Rouge-L (AR). After training on the final task, the average performance across all tasks is computed shown in Equation 5
N
AR
(5)
N

---

Table I: Results LLaMA2-78 and ChatGLM-6B under different task orders and settings.
Order
Order 2
Order 3
Avg.
Model
AR
BWT 1
AR
BWT
AR 1
BWT
AR 1 BWT
LLaMA2-7B STasks
MTL
53.07
53.07
53.07
53.07
KMcansSe((155
49.73
-5.22
50.14
-4.17
50.12
-3.61
50.00
-4.33
1.2
28.62
-28.99
29.22
-28.45
28.33
-30.71
28.72
-29.38
SAPT
50.47
-3.75
51.04
-2.98
50.22
-4.37
50.58
-3.70
SSR
51.33
-1.97
52.41
52.02
51.92
-1.39
SERS
52.90
-0.55
33.01
-0.27
32.84
-0.63
32.92
-0.48
ChatGLM-6B 5Tasks
MTL
48.92
48.92
48.92
KMeansSe([[%6]
43.72
-5.64
43.74
-5.07
45.13
-4.37
44.19
-5.02
L.
25.19
-35.32
26.46
-32.47
26.18
-34.92
25.94
-34.24
SAPT
49.01
-1.54
48.65
-2.11
49.23
-1.89
48.96
-1.84
SSR
48.95
2.12
49.02
-1.94
49.38
-0.51
49.11
-1.22
SERS
49.97
-0.89
49.86
-1.17
50.04
-0.48
49.98
-0.85
LLaMA2-7B 10Tasks
MTL
64.72
64.72
64.72
64.72
KMeansSe((1%)
59.13
-5.88
60.71
-5.39
60.44
-7.17
60.09
-6.15
1.2
33.13
-28.99
34.71
-25.12
37.02
-22.71
34.95
-25.42
SAPT
62.51
-2.06
61.90
-2.81
62.29
-2.30
62.23
-2.39
SSR
62.29
-1.84
62.64
-1.86
62.36
-3.95
62.43
-2.55
SERS
63.42
-1.72
63.45
-1.11
64.46
-2.27
63.78
-I.7
ChatGLM-6B 10Tasks
MTL
62.04
62.04
62.04
62.04
RMeansSS([[1%]
60.84
55.4T
61.24
-4.77
6T.04
-5.27
6f.04
55.13
L2
40.18
-29.71
41.37
-28.66
41.99
-26.12
41.18
28.166
SAPT
61.30
-3.44
61.56
-2.73
60.87
-2.92
61.24
-3.03
SSR
62.68
-1.79
62.27
-2.42
61.80
-1.56
62.25
-192
SERS
63.30
-1.17
63.22
-1.52
63.16
-1.49
63.23
-1.39
(2) Backward Transfer (BWT). BWT measures the degree which the learning of subsequent tasks affects the performance of the learned tasks, which is defined as:
N-1
BWT
Σ
(R -R)
(6)
N
4.2 Experiment Details
All experiments were conducted on a single A100 GPU. For seuudo-sample generation, 1% of real samples are used to create pseudo samples equivalent to 10% of the training data. In the self- questioning stage, we k 20, a 0.5 for LLaMA2-7B, and 10, 0.6 for ChatGLM-6B to control the selection and evolution of over-specialized samples. LoRA is used for fine-tuning, and Wasserstein distance based on model embeddings guides the regularization process.
4.3 Experiment Results
We compare our SERS method with several representative baselines, including the classic rehearsal- based KMeansSel, which selects real samples via KMeans clustering; the advanced pesudo-sample rehearsal approach SSR (Huang et al. 2024). which leverages self-synthesis to generate pseudo samples for rehearsal; advanced structure-based method SAPT (Zhao al., (0244), which employs Shared Attentive Learning & Selection module to align the PET learning and selection; and the regularization-basee 1.2 method. A multi-task learning (MTL. baseline, which jointly trains all tasks without considering forgetting. is also included for reference. As shown in Table SERS consistently

---

Table 2: The ablation studies each proposed module. SGG refers Semantic-Guidee Pseudo- Input Generation. LSE denotes Label Self-Evolution strategy. SR represents Similarity Regularization. A "V" indicates that our module is applied, while a"= denotes the use of corresponding strategy from existing advanced pseudo-rchearsal approaches.
Ablation Setting
LLaMA-7B AR (%)
ChatGLLM6B AR (%) 1
SGG
LSE
SR
Order
Order 2
Order
Order
Orde r2
Order
51.33
52.41
52.02
48.95
49.02
49.38
52.37
52.63
52.54
49.87
49.58
49.35
V
51.99
52.83
52.33
49.25
49.40
49.02
52.40
52.42
52.61
49.68
49.62
49.50
$2.47
52.78
$2.71
49.78
49.33
49,80
52.56
52.92
52.73
49.08
49.43
49.21
$2.59
52.95
52.56
49.70
49.80
49.62
52.90
53.01
52.84
49.97
49.86
50.04
TRANS
SA
QA
SUM
QG
66.5
1000
37.0
27.0
39.5
3600
26.5-
99.0
64.0
35.0
26.0
35.0
63.5
95.5
34.5
63.0
34.0
25.5
33.5
07.5
38.5
Truining Stage
Truining Stage
Training Stuge
Training Stage
Training Stage
- SGG+LES-SR
-
LSE
++--
SGG-LSE
LSE-SR
SGG
-
SR
42-
SGG SR
Advanced Pscudo Rehearsal Method
Figure 4: Ablation results detailing the performance variations of the LLaMA2-7B model across Stasks sequence under Order 1 (TRANS + SA - QA - SUM - QG). More details of ablation results are shown Figure 9
outperforms these baselines and maintains stable performance across task orders, demonstrating the effectiveness of similarity regularization. On LLaMA2-7B. it achieves results close to MTL. surpassing the next best method by 2.16% and exhibiting significantly lower BWT. Notably, in ChatGL-M-6,, where MTI. suffers from task confusion due to global attention and 2D positional encoding, SERS surpasses MTL by incrementally refining decision boundaries through rehearsal with staged updates.
Ablation and Comparison Experiments
5.1 Module Ablation
In this section, we carry ablation studies to verify the effectiveness of each module. All experi- ments are on Stasks, and we measure performance with the AR metric. The results appear in Table 2 and detailed ablation results are provided in Figure 4 SERS introduces three core improvements: Semantic- Guided Generation, Label Self-Evolution and Similarity Regularization. We evaluate the effectiveness of these components with different configuration settings. For settings that do not include the SERS modules, we adopt corresponding strategies from Huang et (0024), where pseudo samples are generated via in-context learning, pseudo labels are directly refined on task-specific model, and no regularization method is applied. The settings with semantic-guides generation achieve higher overall performance compared to those using existing advanced method to generate pscudo samples. The curves with task similarity more likely to exhibit task-level performance

---

improvement during training, and those incorporating label self-evolution tend to perform better on new tasks, demonstrating the effectiveness of our proposed improvements.
5.2 Data Utilization Efficiency
In experiments, we first employed 1% of real samples to synthesize 10% pseudo samples, achieving strong performance with efficient data utilization. We then extended the analysis by generating different amounts of pseudo samples for rehearsal using various proportions of real data (1%, 0.75%, 0.5%, and 5% to investigate the trade-off between data efficiency and pseudo-sample redundancy. As shown in Figure even small number of real samples can produce pseudo samples that are diverse and capable of capturing the underlying task knowledge. However, as the peeudo-sample ratio increases, the improvement in AR gradually saturates. When generating 20% pseudo samples from 1% real data, performance begins to decline due to excessive redundancy and interference with learning new tasks.
When further reducing the number of real samples, we observe that using 0.75% or 0.5% real data yields slightly better performance than 1% real data when synthesizing small proportion of pscudo samples. Nevertheless, the performance degrades notably as the pseudo-sample ratio grows. This suggests that with small pseudo-sample ratio, fewer real samples can better capture the essential task knowledge and improve synthesis quality. In contrast, when larger number of pseudo samples generated, the limited diversity of real samples leads to higher redundancy, which hinders learning effectiveness. Moreover, pseudo samples generated from a larger real dataset tend to form more cluster centers. Under low pseudo-sample ratios, this results in less coherent knowledge structures and slightly worse performance than using 1% real data. Yet, as the pseudo-sample ratio increases to 20%, the performance improves substantially and approaches that of multi-task learning (MTL).
We also compare pseudo-sample rehearsal with real-sample rehearsal, as presented in Table of the Appendix B The results are consistent with findings from SSR, showing that even when 10% of real samples are replayed, the performance remains inferior to that of pseudo-sample rehearsal. This is because labels synthesized by the old model facilitate learning, improving the model's s task adaptation. In contrast, reallsample rehearsal is directly constrained by the limited proportion of available real data, whereas pscudo-sample rehearsal can flexibly cxpand data diversity by synthesiz- ing new samples from a fixed real set. Consequently, the 1 1% real-sample rehearsal fails to match the performance achieved with 5% real samples, the smaller rehearsal ratio restricts the model's ability to preserve prior knowledge.
5.3 Analysis of Parameters
We analyze the impact of two key parameters in label self-evolution. The proportion threshold determines the selection of over-specialized samples during self self-questioning. As shown in Figure 6 setting too high omits valuable specialized knowledge, while a low allows too many over- specialized samples for rehearsal, causing bias in model parameters and affecting overall performance. controls the balance between general (T..M)) and task-specific (L..M))) knowledge when refining labels. Figure 6 illustrates that high a may tend to less accurate labels, whereas low a reduces the smoothing effect, weakening the integration of general and specialized knowledge.
Parameter adjustment should consider model capability. Stronger mask-filling models better preserve prior knowledge in pseudo samples, allowing for smaller k; weaker models require larger k to avoid excessive specialization. Similarly, models with stronger downstream abilities benefit from a higher a to increase the general knowledge in ove-sspedad samples, while less capable models require a lower a to avoid inaccurate labels.
Conclusion
In this work, we propose SERS for catastrophic forgetting mitigation in LLMs. SERS gen- erates pseudo samples that better reflect the structural knowledge of previous tasks, prevents over-specialization on rehearsal pseudo samples from harming overall performance and dynamically adjusts regularization strength based on the similarity between previous and new tasks. Extensive experiments demonstrate that, compared to various representative methods, SERS achieves more

---

Data Utilization Efficiency Stndy
Comparison Study & (a=0.5)
53
AR (59)
52
52.5
AR0%)
0.1
0.2
0.3
0.4
0.5
52.0
Comparison Study (k-0.2)
53
AIR (56
52
1%
9%
10%
15%
20%
Pseudo Sample Ratio
MTT
0.75% Real Sumple - 5% Rod Sample
Redl
1% Real Sample
51
0.3
0.4
0.5
0.6
0.7
Figure 5: Rehearsal Analysis. We generate various proportions of pseudo samples us- ing different amounts of real samples rang- ing from 0.5% to 5% on LLaMA2-7B to evaluate data utilization efficiency.
Ordert
Ordar2
Order3
Figure 6: Parameter Analysis. We evaluate SERS performance on LLaMA2-7B by varying and a while fixing the other parameter respectively.
effective forgetting mitigation and enhanced performance stability, underscoring SERS's potential as general solution for continual learning in LLMs.
Limitations
Although ablations confirm each module's contribution AR and show score performance during training. the complex relationships between tasks make it hard to pinpoint why some tasks improve or decline. A deeper analysis how new tasks affect previous tasks may unlock further gains in continual accuracy. Moreover, while pseudo-sample rehearsal boosts review of past knowledge, remains unclear whether these synthetic examples can introduce knowledge beyond the original data. Exploring the ability of pseudo samples enrich the model with unseen knowledge could be key to surpassing MTL in future continual learning work.
Acknowledgments and Disclosure of Funding
This work is supported by the National Key Research and Development Program of China (2033YFC7705000, the National Natural Science Foundation of China (Grant No. 62225113, U23A20318, U2336211 I and 62276195), the Foundation for Innovative Research Groups of Hubei Province (Grant No. 2024AFA017) and the Science and Technology Major Project of Hubei Province (Grant No. 2024BAB046). The numerical calculations in this paper have been done the supercom- puting system the Supercomputing Center of Wuhan University.
References
David Alvarez-Meihs and Nicolo Fusi. Geometric dataset distances via optimal transport. Advances in Neural Information Processing Systems, 33:21428 -24399,, 2020.
Yao Chen, Qingyi Gao, and Xiao Wang. Inferential wasserstein generative adversarial networks.
Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(1):83-113, 2022.
Cyprien de Masson D' Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. Episodic memory lifelong language learning. Advances Neural Information Processing Systems, 32, 2019.
Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, al. Chatgim: family of large language models from glm-1300 130b to glm-4 all tools. arXiv preprint 24X::2406.11793 2024.
10

---

Yanhui Guo, Shaoyuan Xu, Jinmiao Fu, Jia Liu, Chaosheng Dong, and Bryan Wang. Q-tuning: Queue-based prompt tuning for lifelong few-shot language learning. In Findings of the Association for Computational Linguisties: NAACI. 2024, pages 2595-2622, 2024.
Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey, arXiv preprint arXiv:2403. 14608, 2024.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL battps:://pperreee nZeVKeeFYf9
net. net/forum?id
Jianheng Huang, Leyang Cui, Ante Wang, Chengyi Yang. Xinting Liao, Linfeng Song, Junfeng Yao, and Jinsong Su. Mitigating catastrophic forgetting in large language models with self-syythesized rehearsal. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 1416-1428, 2024.
Xisen Jin, Bill Yuchen Lin, Mohammad Rostami, and Xiang Ren. Learn continually, generalize rapidly: Lifelong knowledge accumulation for few-shot learning. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 714-729, 2021.
Zixuan Ke, Bing Liu, Wenhan Xiong, Asli Celikyilmaz, and Haoran Li. Sub-network discovery and soft-masking for continual learning of mixed tasks. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15090-15100 2023.
Hongyu Li, Liang Ding, Meng Fang, and Dacheng Tao. Revisiting catastrophic forgetting in large language model tuning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4297-4308, 2024.
Chin Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81, 2004.
Xinran Liu, Yikun Bai, Yuzhe Lu, Andrea Soltoggio, and Soheil Kolouri. Wasserstein task embedding for measuring task similarities. Neural Networks, 181:106796, 2025.
Aru Maekawa, Hidetaka Kamigaito, Kotaro Funakoshi, and Manabu Okumura. Generative replay inspired by hippocampal memory indexing for continual language learning. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 930-990 942, 2023.
Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin. The stability-plasticity dilemma: Investi- gating the continuum from catastrophic forgetting to age- aimitee learning effects, 2013.
Keqin Peng, Liang Ding, Qihuang Zhong, Yuanxin Ouyang, Wenge Rong, Zhang Xiong, and Dacheng Tao. Token-level self-evolution training for sequence-to-sequemie learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 841- 2023.
Weijieying Ren, Xinlong Li, Lei Wang, Tianxiang Zhao, and Wei Qin. Analyzing and reducing catastrophic forgetting in parameter efficient tuning. arxiv preprint arXiv:2402.18886 2024.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances in neural information processing systems, 32, 2019.
Avi Singh, John D Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J Liu, James Harrison, Jachoon Lee, Kelvin Xu, et al. Beyond human data: Scaling self-training for problem-solving with language models. Transactions on Machine Learning Research, 2023.
Yuncheng Song, Liang Ding, Changtong Zan, and Shujian Huang. Self-evolution knowledge distillation for lmm-based machine translation. In Proceedings of the 31st International Conference on Computational Linguisties, pages
2025.
Fan-Keng Sun, Cheng Hao Ho, and Hung-Yr Lee. Lamol: Language modeling for lifelong language learning. In International Conference Learning Representations, 2020.
11

---

Huashan Sun and Yang Gao. Reviving dormant memories: Investigating catastrophic forgetting in language models through aationale-guidanee difficulty. arXiv preprint arXiv:2411.11933 2024.
Zhengwei Tao, Ting-En Lin, Xiancai Chen, Hangyu Li, Yuchuan Wu, Yongbin Li, Zhi Jin, Fei Huang, Dacheng Tao, and Jingren Zhou. A survey on self-evolution of large language models. CoRR, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXh::2307.09288 2023.
Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, and Hinrich Schütze. Rehearsal-free modular and compositional continual learning for language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 469-480, 2024a.
Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. Orthogonal subspace learning for language model continual learning. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
Yifan Wang, Yafei Liu, Chufan Shi, Haoling Li, Chen Chen, Haonan Lu, and Yujiu Yang. Insel: A data-efficient continual learning paradigm for fine-tuning large language models with instructions. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pupers), pages 663-677, 2024b.
Yihan Wang, Si Si, Daliang Li, Michal Lukasik, Felix Yu, Cho-Jui Hsieh, Inderjit S Dhillon, and Sanjiv Kumar. Two-stage 11m fine-tuning with less specialization and more generalization. In The Twelfth International Conference on Learning Representations, 2024c.
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, et al. Supprr-naturalinssuuttinsss Generalization via declarative instructions on 1600+ nlp tasks. In Proceedings of the 2022 Conference Empirical Methods in Natural Language Processing, pages 5085-5109, 2022.
Tongtong WU, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. Continual learning for large language models: A survey. CoRR, 2024.
Wenpeng Yin, Jia Li, and Caiming Xiong. Contintin: Continual learning from task instructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3062 -072,, 2022.
Weixiang Zhao, Shilong Wang, Yulin Hu, Yanyan Zhao, Bing Qin, Xuanyu Zhang, Qing Yang, Dongliang Xu, and Wanxiang Che. Sapt: A shared attention framework for parameter-efficient continual learning of large language models. In Proceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1 11641-11661, 2024.
Haoqi Zheng, Qihuang Zhong, Liang Ding, Zhiliang Tian, Xin Niu, Changjian Wang, Dongsheng Li, and Dacheng Tao. Self-evolution learning for mixup: Enhance data augmentation on few-shot text classification tasks. In The 2023 Conference on Empirical Methods in Natural Language Processing. 2023.
Junhao Zheng, Xidi Cai, Shengjie Qiu, and Qianli Ma. Spurious forgetting in continual learning of language models. In The Thirteenth International Conference on Learning Representations, 2025a.
Junhao Zheng, Shengjie Qiu, Chengming Shi, and Qianli Ma. Towards lifelong learning of large language models: A survey, ACM Computing Surveys, 57(8):1-35, 2025b.
Qihuang Zhong, Liang Ding, Yibing Zhan, Yu Qiao, Yonggang Wen, Li Shen, Juhua Liu, Baosheng Yu, Bo Du, Yixin Chen, et al. Toward efficient language model pretraining and downstream adaptation via self-evolution: A case study on superglue. arXiv preprint arXiv:2212.01883, 2022.
12

---

Qihuang Zhong, Liang Ding, Juhua Liu. Bo Du, and Dacheng Tao. Self-evolution learning for discriminative language model pretraining. In Findings of the Association for Computational Linguistics: ACI. 2023, pages 4130-4145, 2023.
13

---

A Comparative Case Study on Pseudo-Sample Generation
To illlustrate the effectiveness of our proposed method, we present comparison between the traditional ICL approach and the Semantic- Guiied Pseudo-Input Generation we introduced. Using two samples from the same task, generate pseudo data in ways for two retries: the ICL method generates both inputs and outputs with 2-shot prompting, while our method focuses on generating only pseudo inputs. As shown in Figure 7 is evident that the 2-shot ICL approach fails capture the essence of the "cosmosqa" task. The generated questions resemble generic QA samples, lacking the implicit cues that emphasize common-sense reasoning. In contrast, our method is able to produce samples that closely resemble real ones under a low masking ratio. When the masking ratio is high, the generated outputs are diverse but still retain the core features of the" chosmoaaaa""" task. This not only ensures the diversity of pscudo samples, but also supports the model's review of previous knowledge.
Real Sample:
Input:
Emmy isn't feeling well. woke up this morning with a little of vom on the near her mouth. Dog's 's puke all the time though, so no big deal. let her out and she took a big old solid 2-point conversion. But she didn't 't run or scuff her feet afterwards, she just ambled back inside. Question: What did I mean when said she took solid 2-point conversion? Output: She pooped.
Input:
blaine was only able stay for little while since he had to work sunday and while i wanted go home with him to NOT face the party the rest time though. there was making out involved among some of the party guests. oh, might add that the party took place on the rooftop of this lovely apartment building.
evening. i stayed with nick and brenna. had such a good
Question: Where might James have gone after leaving the party? Output:
He went home.
In-Context Learning Output:
Semantie-Guided Generation:
Input:
Input:
The best thing about being single is you whatever you whenever you feel like eating
Blaine reluctant leave early because he had work Sunday and wanted spend more time with Nick and Brenna. He decided not to go home with them but instead to return later tonight. was having such good time at the party as were many others who made into the exclusive located on top of this amazing apartment complex. Oh, might add that the party location was right outside our living room window!
it!
Question: When would Katic most likely be thinking these thoughts?
Output: After having an argument over dinner at someone else's house.
Input:
Whenever meet up at cafe something, he always seems very keen to see me again, wants us to do things together I'm ready to leave, his eyes get all glassy and sad looking and he says don't 't want you think hadly about me.It makes me feel like maybe my presence isn't actually welcome- can enjoy being around me, even though he tries hard to hide it.
Question: Where else could James have gone after leaving the party?
the time! But then when
Input:
Blaine seemed like such nice guy since it came down to him being the last person standing the bar Sunday night. He wanted me to stay for just little longer; however, didn't 't want to face the remainder of the evening alone. So went home with Nick and Brenna instcad. They were very kind and offered us plenty of space even we made our own drinks or snacks from what they provided. Oh! And by the way, their roof-top patio absolutely gorgeous!
doesn't really
Question: What does the speaker most likely think about herself?
Output: She thinks highly of herself.
Question: What would you think about James based upon his behavior
Figure 7: case study for Pseudo-Sample Generation
14

---

B Comparative Case Study on Output Refinement
This part demonstrates the value of self-evolution in label refinement through real examples. By adjusting the balance between general knowledge from LLM and task-specific knowledge from LLM', the model can reduce overly specialization and produce more acceptable responses. In the the output from LLM contains hallucinations and fails to provide a correct answer. In contrast, the output from L.LM* correctly identifies the key term "family". The final response refined through self-evolution also delivers a precise and reasonable answer.
QA example shown Figure X
Input: A small part of her face lit up when she heard the news. Her eyes glistened but there wasn't it any trace of joy on them happiness even though this was great news. She simply looked relieved because finally things were starting to go well for her family. And they had been through hell lately every little thing counted nowadays. So she couldn't 't help feeling grateful. Question: What made the person feel relief?
LLM Output: Things went wrong recently!
LLM Output: They got good news for their family.
Output after Self-Evolution:
The person was told that things was going to be okay.
Figure 8: A case study for output refinement
C Additional Experimental Details
All samples are selected from the SuperNI dataset. To ensure fair comparison, we follow the same task settings as advanced pseudo-sample rehearsal methods (Huang et al. 2024). The selected tasks are shown in Table During pseudo-sample generation, we begin with 1% of the real samples and apply the Comantic-Guuddde Pseudo-Input Generation method with three retries to create large number of pseudo samples. Then, we use the K-Means algorithm to select 10% of these for rehearsal. During the training phase, we selected two numbers of tasks and two task sequences to form four task chains, with the task chains as shown Table 4
Table 3: Details of task names and abbreviations
Abbreviation
Task Name
QA
task024_ ltt__oooooosssssssee satpsssanswwe__eeeeeeeeeeeeeeeee
QG SA SUM
1t1l3//2aammzoneviie___ppppael pottr/ecoolarrtttcclscccttooooon claassificcatoon
task511 1ask5111redddi_____ tifu Liallit_tiu_loogg_eee __to__tex__summariaatiiiii summarization
TRANS
task1219_teed tedd_tranllation_ee
DSG
task574 air dialogue sentenceeeeeeeeeeeee generation
EXPL
task192 hotpotta sentence generation
PARA
task177 para-nmt paraphrasing
POS
task346 hyyridga__lassiiiiiiiieeee
PE
cettsssecceeettttt__ctceee first_i
Table 4: Details of Task Chains under Different Task Numbers and Orders
Settings
Task Chain
5Tasks Order
TRANS - SA - QA SUM QG
5Tasks Order
QA + QG - SA - SUM - TRANS
5Tasks Order 3
SUM - QG -, TRANS - QA - SA
10Tasks Order 1
TRANS - SA - QA - - SUM - QG -, PE PARA + POS - DSG + EXPL
10Tasks Order 2
QA - QG - SA SUM TRANS + DSG EXPL- PARA PE POS
10Tasks Order 3
SUM + QG TRANS - QA + SA PARA + DSG - POS + EXPL - PE
15

---

D Ablation Study Details
In this section, we present the detailed performance of each ablation setting across different task chains and models. The results show that our generation strategy consistently leads better outcomes, label self-evolution generally benefits the learning of new tasks, and similarity regularization facilitates knowledge transfer, increasing the likelihood of performance gains throughout training. Details are shown in Figure
Ablation Study LLaMA2-7B Order2
QG
36.9
40.5
1000
27.5
56.0
33.5
40.0
35.9
99.5
34.5
65.5
33.5
38.5
26.5
65.0
38.0
32.5
37.5
2
Training Stage
Training Stage
Training Stage
Training Stage
Training Stage
LGG+LES
LSE
SGG-LSE
LSE+SR
SGG
SR
SGG SR
Advanced Pseudo Rehearsal Method
Ablation Study on LLaMA2-7B Order3
SUM
QG
TRANS
QA
SA
27.5
40.0
37.0
39.5
39.0
ARR%)
26.5
38.5
65.0
36.0
26.0
38.0
64.5
99.5
35.5
37.
24.5
36.0
63.0
36.5
99.0
Training Stage
Training Slage
Training Stage
Training Stage
Training Stage
LGG
LSE
SGG-LSE SGG-SR
LSE:
SGG
SR
Advances Pscudo Rehearsal Method
Ablation Study ChatGL Orderl
TRANS
QA
QG
99.5
345
38.0
34.0
25.0
99.0
33.5
37.5
98.5
32.5
37.0
32.0
21.5
$4.0
95.0
31.5
21.0
36.5
2
4
2
3
2
3
Truining Stage
Training Stage
Training Stage
Training Stage
Training Stage
SGG+LES-SSD
LSE
SGG-LSE SGG-SR
LSE+
SGG
Advanced Pseudo Rehearsal Method
16

---

Ablation Study ChatGLM-6B Order2
QA
QG
SA
SUM
TRANS
39,0
995
60.0
33.0
57.5
99.0
99.5
32.5
37.0
32.0
36.5
36.0
98.5
59.0
55.
22.0
30.0
2
34.5
98.0
2
21.5
58.5
3
Training Stage
Training Stage
Training Stage
Training Stage
Training Stage
SGG+LES
LSE
SGG-LSE
LSE+SR
SGG
SGG-SR
Advanced Pacudo Rehcarsal Method
Ablation Study on ChatGLM-6B Order3
SUM
QG
TRANS
SA
24.0
38.0
60.5
34.0
99.5
$7.5
37.0
59.5
35.5-
99.0
23.0
36.5
58.5
33.0
22.5
36.0
98.5
35.5
57.5
22.0
35.0
57.0
21.5
345
36.5
98.0
Truining Stage
Truining Stage
Training Stuge
Training Stage
Training Stage
SGG+LES-SR
-
LSE
SGG-LSE
LSE-SR
SGG
SR
-0
SGG-SR
Advanced Pscudo Rehearsal Method
Figure 9: Ablation results detailing the performance variations of different models across different task chains
E
Real-Sample Rehearsal Details
This section presents additional results on real-sample rehearsal for comparison with pscudo-sample rehearsal. As shown in Table E, even when 10% of real samples used for rehearsal under the same continual learning setup, the performance remains lower than that of pseudo-sample rehearsal. This observation be explained by the fact that labels synthesized by the old model facilitate learning, improving the new model's 's task adaptation. In contrast, samlle rehearsal is constrained by the limited number of available samples, resulting in reduced diversity and weaker knowledge coverage. Consequently, its performance degrades more noticeably under low rehearsal ratios.
Table 5: Real-Sample Rehearsal Results on LLaMA2-7B
Data Rehearsal
Order1
Order2
Order3
Avg
% real samples
48.11
49.02
48.74
48.62
5% real samples
50.18
50.65
50.02
50.28
10% real samples
50.24
51.09
50.84
50.73
real samples synthesis 10% pseudo samples
52.90
53.01
52.84
52.92
F Comparison Study Details
In this section, we provide additional experimental details on the ChatGLM-6B model to demonstrate the impact of the hyperparameters & and a on the SERS framework.
17

---

Comparison Study of k ChatGL.M-B
50
Orderl
Order2
Order3
AR
48
47
0.1
0.2
0.3
0.4
0.5
values
Figure 10: Comparison study of (a=0.6) ChatGLM-6B
Comparison Study of a on ChatGLM-6B
Orderl
50.0
Order2
Order3
(R(%)
49.5
49.0
48.5
0.3
0.4
0.5
0.6
0.7
a values
Figure 11: Comparison study of a (k=0.1) on ChatGLM-6B
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction, clearly demonstrate the contribution and scope of this paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
18

---

Answer: [Yes],
Justification: In Section 6 we have thoroughly discussed the limitations of our article, hoping to guide more future work.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: In Section 3]
we elaborated the motivation and theoretical derivation of our method, with complete proof process in place.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
19

---

We have provided detailed deseriptions of the experimental details in section 4.2 and methods in section to ensure that our experiment can be reproduced.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Our datasets are derived from publicly available datasets, and our code will also be fully open-sourced.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
mublic/guides
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
20

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes)
Justification: In Section 4.2 we presented the experimental setup and the selection of key parameters. Additional details could refer to our code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Due to the computational cost continual learning, we did not perform multiple runs for each experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative eтror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
21

---

we have provided sufficient information on the computer resources needed to reproduce the experiments.
Justification: In Section 4.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't oodn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: We guarantee that the research conducted in the paper complies with NeurIPS Code of Ethics in all aspects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We outlined the societal benefits of continual learning research in Section | and highlighted the limitations and challenges of existing techniques.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploymeett of technologics that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
22

---

11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The creators or original owners of the assets used in the paper, such code, data, and models, have been appropriately recognized, and the licenses and terms of use have been clearly mentioned and properly respected.
Justification:
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
23

---

The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMSs if it is important, original, or non-standard component of the core methods in this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: This study consistently adheres to relevant policies governing the use LLMSs and provides a detailed description of their application.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy for what should or should not be described.
24


Reviews:
[{"review_id": "mNrgcq08XC", "reviewer": "Reviewer_ZMGj", "summary": "This paper addresses a problem of pseudo sample replay in continual learning, where the generated pseudo samples cannot well reflect the original knowledge structure and thus provide limited support for rehearsal. It proposes a method that first generates pseudo samples with semantic guidance and masked inputs, then performs label self-evolution by mixing the outputs of the current and initialized LLM. It also adds a regularization term to encourage more knowledge transfer across similar tasks. Results show that the proposed method outperforms other baselines.", "questions": "What is the performance gap between replay with pseudo examples and real examples? In the rehearsal analysis in Fig 5, using 1% real data can generate better pseudo samples for rehearsal than using 5%. Does using 1% real data also outperforms 5% real data in rehearsal?", "limitations": "The model needs extra LLM forwarding to generate pseudo examples and label evolution. It also needs extra memory to store the LLM_0 and LLM_0, which is natural for PEFT models but have extra cost for fine-tuning models.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. The paper studies an important pseudo rehearsal problem for CL with LLM.\n2. The experiments are performed on two LLMs, with thorough comparison to baselines. Ablation studies are included.\n\n**Weaknesses**\n1. Some details are unclear in the method:\n* When masking the real sample for pseudo sample generation, how to decide the tokens to mask? Is it random masking? If so, the masked place may not contain much semantic information.   \n* How to integrate the output from different models? Is it combining a sequence of output tokens or representations? What are the real output tokens after integration?\n\n2. The intuition for task similarity regularization is unclear to me. During pseudo example generation and label evolution, the model already preserves the information of LLM_0 for all tasks. Then why should it take additional regularization for task similarity? If the tasks in the CL sequence are dissimilar, will it limit the model’s plasticity to learn each task?", "was_revised": true, "final_justification": "The authors' rebuttal have addressed most of my concerns: the efficacy of pseudo samples and the influence of masking. But as mentioned in the rebuttal, if the tasks are dissimilar, the method will sacrifice plasticity to reduce forgetting which I believe can be further improved. So I raised my score to 4.", "rebuttal": "**Response to Reviewer ZMGj:**\n\nWe sincerely appreciate for your constructive and valuable comments, which will play an important role in improving the paper. We hope that the additional experiments and responses will help you better understand our work.\n\n***Q1: Masking Strategy and Semantic Information***\n\nThis is a good comment. As you have correctly noted, we indeed use a random masking approach. **Random masking has been widely used in related generation tasks. For example, Huang et al.[1] employed random masking to generate pseudo anomalies in unsupervised video anomaly detection, and we have adopted this approach as well.** We have considered the semantic information in the masked parts, where a larger dynamic masking ratio and post-processing clustering help preserve the semantic content. **Our masking ratio, set between 20% and 60%, which allows for better coverage of semantic content**, while filtering and clustering after pseudo-sample generation help improve the quality and diversity. The following real-world example may help you intuitively experience the effect of our masking.\n\n*Before Masking*: The item was said to have been delivered Friday and was not delivered. The shipping address was correct and payment information correct but the tracking information was wrong or it was never actually shipped.\n\n*After Masking*: The ____ said to have been ____ and was not delivered. The ____ was ____ and ____ information correct but ____ was wrong or it was never ____ shipped.\n\n***Q2: Model Output Merging***\n\nTo integrate the outputs from the current model and the initial model, **we weight their logits using the weight parameter α and select the token with the highest probability as the final output.** By leveraging the general knowledge from the initial model, we smooth the specialized knowledge of the current model, avoiding biases caused by over-specialized pseudo labels during the rehearsal process.\n\n***Q3: Intuition and Necessity of Similarity Regularization***\n\nThank you for pointing out this issue. Pseudo-sample generation is closely tied to task similarity regularization. **The goal of pseudo-sample generation and label evolution is to represent old task knowledge as data. Task similarity regularization governs how this knowledge is revisited and balances old knowledge retention with new task learning.**  Intuitively, it imposes constraints on parameter changes based on task similarity. When tasks are similar, regularization is relaxed, facilitating knowledge transfer. If tasks differ significantly, regularization becomes stricter to prevent forgetting key old task knowledge.\n\nIf tasks in the CL sequence are dissimilar, similarity regularization prioritizes stability over plasticity, sacrificing some plasticity to mitigate catastrophic forgetting. This is the core motivation behind our similarity regularization design.\n\n***Q4: Comparison between Pseudo-Sample Rehearsal and Real-Sample Rehearsal***\n\nThis is an insightful and practically valuable suggestion. Regarding Figure 5, when synthesized pseudo samples are few (≤15%), 1% real samples outperforms 5% real samples. Since real samples are selected through clustering, when fewer pseudo samples are synthesized, 1% real samples lead to fewer clustering centers, resulting in pseudo samples that better reflect core knowledge. However, with a higher count of synthesized pseudo samples (20%), 5% real samples provide more comprehensive knowledge, approaching multi-task learning performance.\n\nFollowing your suggestion, **we included real-sample rehearsal results, as shown in the table below.** **Our findings align with those of the SSR[2] method, where even with 10% real sample rehearsal, the performance is lower than using pseudo samples.**  This is because labels synthesized by the old model facilitate learning, improving the new model’s task adaptation. However, unlike pseudo-sample rehearsal, where different numbers of real samples are used to synthesize the same quantity of pseudo samples, **the 1% real sample rehearsal did not achieve the same results as the 5% real sample rehearsal due to the smaller rehearsal ratio**.\n\n| Data Rehearsal | Order 1 | Order 2 | Order 3 | Avg |\n| --- | --- | --- | --- | --- |\n| 1% real samples | 48.11 | 49.02 | 48.74 | 48.62 |\n| 5% real samples | 50.18 | 50.65 | 50.02 | 50.28 |\n| 10% real samples | 50.24 | 51.09 | 50.84 | 50.73 |\n| 1% real samples synthesis 10% pseudo samples | 52.90 | 53.01 | 52.84 | 52.92 |\n\n***Q5: The Need for Storing LLM_0***\n\nWe fully understand your concern. Storing LLM_0 dose imposes some burden, and storage is one of the challenges faced by pseudo-sample rehearsal methods. However, it is worth noting that most pseudo-sample rehearsal methods also rely on additional generative modules or LLMs for pseudo-sample generation. For instance, the SAPT[3] method depends on training an additional generative module, which increases the training burden, while the SSR method similarly uses LLM_0 for generating pseudo samples. **Although our work requires storing LLM_0, it only involves forward inference, independent of the training process, and does not introduce additional training costs.** We will also further explore more lightweight storage solutions.\n\nWe would like to express our sincere gratitude again for your insightful comments, and we hope the above responses address your concerns. We will attend to all comments to the best extent in the revised version. \n\n*[1] Huang, Xiangyu, et al. \"Synthetic pseudo anomalies for unsupervised video anomaly detection: A simple yet efficient framework based on masked autoencoder.\" ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023.*\n\n*[2] Huang, Jianheng, et al. \"Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal.\" Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.*\n\n*[3] Zhao, Weixiang, et al. \"SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models.\" Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.*"}, {"review_id": "QZlTqFtYho", "reviewer": "Reviewer_izSP", "summary": "This paper presents Self-Evolving Pseudo-Rehearsal with Task Similarity (SERS), a novel continual learning framework designed to mitigate catastrophic forgetting in large language models (LLMs). SERS introduces three key innovations: (1) it decouples pseudo-input generation from label synthesis by leveraging semantic masking and template-guided prompt construction; (2) it incorporates a self-evolution mechanism that adaptively fuses base and task-specific label signals to improve generalization; and (3) it employs a task-similarity-aware regularizer based on the Wasserstein distance to dynamically control the stability–plasticity trade-off. Evaluations on the SuperNI benchmark using LLaMA2-7B and ChatGLM-6B across diverse task permutations demonstrate that SERS consistently outperforms state-of-the-art pseudo-rehearsal and regularization methods, and in some cases achieves performance comparable to or better than multi-task training.", "questions": "see above", "limitations": "see above", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The SERS framework integrates pseudo-rehearsal with dynamic, task-similarity-aware regularization, jointly addressing the stability–plasticity dilemma and the fidelity of pseudo-samples. Its use of semantic masking and template-guided input synthesis is well-motivated and yields pseudo-data that closely aligns with the true task distribution.\n2. The proposed two-stage mechanism, which consists of a self-questioning phase followed by the integration of outputs from both pre-trained and task-adapted models, aligns well with recent developments in self-evolution frameworks. This design effectively mitigates over-specialization by maintaining a balance between generalization and task-specific adaptation.\n\nWeakness:\n1. While the proposed method is effective in benchmark settings, its applicability to real-world domain-specific scenarios, such as medicine or law,  is not well explored. In such cases, large language models already perform well on general tasks, and continual learning becomes necessary only when adapting to specialized domains. However, determining which domain-specific knowledge should be retained and what type of pseudo-samples should be synthesized remains an open and non-trivial challenge.\n2. The framework relies on measuring task similarity to guide regularization, but the method for quantifying meaningful relationships between heterogeneous tasks is insufficiently justified. For example, assessing similarity between a mathematical reasoning task and a clinical diagnosis task poses significant challenges due to their fundamentally different semantics. Although the experiments are conducted on the SuperNI benchmark, the task diversity in this dataset may not fully capture the complexity of real-world task heterogeneity.", "was_revised": true, "final_justification": "This is a borderline paper. Paper is written well and easy to follow. However, the proposed method is sensitive to the order of tasks, which limits it applications. I choose borderline accept.", "rebuttal": "**Response to Reviewer izSP:**\n\nWe sincerely appreciate for your constructive and valuable comments, which will play an important role in improving the paper. We hope that our responses will help you better understand our work.\n\n***Q1: Applicability in Real-World Domain-Specific Scenarios, Knowledge Retention, and Pseudo-Sample Generation***\n\nTo enable comparison with state-of-the-art methods, we conducted our study using the widely adopted SuperNI benchmark in continual learning. Nevertheless, we believe the SERS method is also applicable to real-world, domain-specific tasks. For example, **in the legal domain, tasks like Question Answering, Summarization, and Text Classification in the multilingual dataset LawInstruct[1]  share similarities with our dataset.** Similarly, **in other domains where the model faces the challenge of learning new tasks with new data while retaining the ability to preserve old tasks, we believe SERS will also exhibit strong generalization capability.**\n\nFor knowledge retention: Knowledge retention strategies are more evident in architecture-based continual learning methods, which add extra parameter blocks to store task-specific knowledge. However, this approach requires continuously expanding the parameter space, making it difficult to accommodate long task sequences. In contrast, SERS employs a rehearsal strategy, allowing the model to balance plasticity and stability by reviewing old knowledge while learning new tasks.\n\nFor pseudo-sample generation: SERS leverages the in-context learning of large models. By using a few real samples, SERS generates pseudo-samples for each task that effectively reflect domain-specific knowledge, helping the model review old knowledge during the training of new tasks.\n\n***Q2: Task Similarity Quantification***\n\nWe appreciate your thoughtful feedback. In SERS, we use the Wasserstein (WS) distance to measure task similarity. **Liu et al.[2] have theoretically proven the effectiveness of this quantification method.** This similarity is applied through a regularization loss in continual learning tasks, which quantifies the **relative differences between new and old tasks.** As you mentioned, a mathematical reasoning task and a clinical diagnosis task have fundamental semantic differences, so it is indeed more likely for the model to experience forgetting when learning a clinical diagnosis task after a mathematical reasoning task. Therefore, a larger regularization loss constrains parameter changes to mitigate forgetting. In contrast, when the model learning a statistical analysis task after a mathematical reasoning task, a smaller regularization loss reduces the constraint and encourages knowledge transfer.\n\n***Q3: Dataset Diversity and Its Gap with Real-World Tasks***\n\nWe completely understand your concern; the diversity of the dataset and its alignment with real-world tasks are indeed crucial factors in determining the practical applicability of our method. **The SuperNI benchmark encompasses a variety of classic text-to-text tasks**, such as question answering (QA), question generation (QG), sentiment analysis (SA), summarization (SUM), and translation (TRANS), all of which exhibit a good degree of diversity. **Moreover, the selected sub-tasks in our study are closely related to real-world tasks.** For example, QA comes from real conversations on large social platforms, QG includes questions extracted from Wikipedia articles, SA is based on product reviews from Amazon, SUM is from Reddit’s \"Today I Fucked Up\" (TIFU) section, and Trans is derived from TED Talks. All of these tasks are based on common, real-life scenarios and represent familiar question types. We will also further experiment with domain-specific real-world datasets such as LawInstruct and supplement the results in the final version.\n\nWe would like to express our sincere gratitude again for your insightful comments, and we hope the above responses address your concerns. We will attend to all comments to the best extent in the revised version.\n\n*[1] Niklaus, Joel, et al. \"LawInstruct: A Resource for Studying Language Model Adaptation to the Legal Domain.\" Findings of the Association for Computational Linguistics: NAACL 2025. 2025.*\n\n*[2] Liu, Xinran, et al. \"Wasserstein task embedding for measuring task similarities.\" Neural Networks 181 (2025): 106796.*"}, {"review_id": "myevEckgVw", "reviewer": "Reviewer_fHFB", "summary": "The work introduces a novel method for fine-tuning LLMs within the continual learning setup. The method is a combination of a pseudo-rehearsal and regularization strategies. The evaluation of the method shows a clear advantage of the method over the baselines and closes the gap to the Multi-Task reference, which is a sign of a really strong performance.", "questions": "I've listed most of my questions in the Strengths & Weaknesses section.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n1. The paper is written clearly, and all the concepts, motivations behind the whole work, and respective experiments are well articulated. \n2. The introduced method is correctly evaluated using a metric typical for continual learning. In both metrics (AR, BWT), the method beats other baselines, showing strong performance. Additionally, the authors took care of the order of tasks, which is known to be an important factor in CL experiments.\n3. The idea to create Semantic guided Pseudo-Input and Self-Evolving Labels is interesting, and through ablations and examples, the authors highlight the role of these components in the overall performance.\n4. The method requires storing as little as 1\\% of the overall data samples to use them as a \"skeleton\" for artificial sample creation. This is a nice idea to reduce the requirements for the storage of samples. I'm wondering how far this process can go. I.e., I'd like to see the performance degradation of the overall method when further reducing the number of samples stored in the buffer. These experiments would nicely complement the findings from the experiment in Figure 5.\n### Weaknesses\n1. The length of the take sequence is relatively short. While I'm aware that it's harder to create equally long sequences of tasks as in the case of computer vision datasets, we know that longer sequences have a different set of challenges; thus, it'd be beneficial to see how this method works on such tasks.\n2. There is little elaboration on why Wasserstein Distance is the right way to measure task similarity. While I like the idea of adjusting the level of regularization based on the task similarity, I find it quite disappointing that this (quite important!) choice is not discussed (or compared) in a broader context.  \n3. The proposed method for generating pseudo-samples seems to be particularly suited for the specific QA type of datasets. Do you have any ideas how this method could be adjusted to make it applicable to other types of datasets? Have you performed any experiments with different datasets?", "was_revised": true, "final_justification": "Given my positive rating since the beginning and the thoughtful answers provided by the authors during the rebutal period, I keep my recommendation to accept the article.", "rebuttal": "**To Reviewer fHFB:**\n\nWe sincerely appreciate for your constructive and valuable comments, which will play an important role in improving the paper. We hope that the additional experiments and responses will help you better understand our work.\n\n***Q1: Effectiveness with Fewer Real Samples***\n\nFollowing your suggestion, **we conducted experiments using fewer real samples to generate pseudo-samples.** Given that the benchmark training set contains 2,000 samples, 1% corresponds to just 20 samples. Since our pseudo-sample generation method relies on combinations of real samples, an extremely small number of real samples will lead to repetition. Therefore, we tested with 15 (0.75%) and 10 (0.5%) real samples, with results shown below. \n\n**The results show that 0.75% and 0.5% real samples slightly outperform 1% real samples when synthesizing 5% pseudo samples. However, in the remaining cases, they are less effective than synthesizing with 1% real samples.** This is particularly evident when a large number of pseudo-samples are generated, where performance drops significantly. This observation aligns with the conclusion presented in our paper. Since real samples are obtained through k-means clustering, **fewer real samples better capture core knowledge when synthesizing a small number of pseudo samples. In contrast, when synthesizing a larger number of pseudo-samples, the repetition of generated samples becomes too strong.**\n\n| Real Samples | Synthesized Pseudo Samples | Trans | SA | QA | SUM | QG | AR Metric |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| 15 (0.75%) | 100 (5%) | 62.35 | 98.8 | 35.02 | 26.51 | 39.57 | 52.45 |\n| 15 (0.75%) | 200 (10%) | 63.95 | 98.8 | 35.77 | 26.45 | 39.49 | 52.89 |\n| 15 (0.75%) | 300 (15%) | 64.17 | 98.8 | 35.74 | 25.89 | 38.98 | 52.71 |\n| 15 (0.75%) | 400 (20%) | 64.49 | 98.8 | 35.21 | 25.95 | 37.97 | 52.48 |\n| 10 (0.5%) | 100 (5%) | 62.78 | 98.8 | 34.11 | 25.83 | 39.64 | 52.23 |\n| 10 (0.5%) | 200 (10%) | 64.00 | 98.8 | 35.46 | 26.46 | 39.23 | 52.79 |\n| 10 (0.5%) | 300 (15%) | 64.09 | 98.6 | 35.26 | 25.40 | 38.63 | 52.40 |\n| 10 (0.5%) | 400 (20%) | 64.01 | 98.4 | 34.18 | 26.18 | 38.78 | 52.31 |\n\n**Q2: Short Task Sequence Length**\n\nIn our experiments, **considering the size of the LLM parameters, we followed the task sequence length used in SSR[1] and conducted experiments with 5-task and 10-task sequences,** as shown in Table 1. The detailed task selection and sequence can be found in Table 3 and Table 4 of Appendix C. **Compared to other previous works in continual learning, the sequence length is representative in the context of task-incremental continual learning.** Extending the task sequence further would introduce more complex challenges. Some architecture-based methods such as SAPT[2] require expanding modules for each new task, while **our approach can effectively balance stability and plasticity without continuously increasing the training parameters, allowing it to still perform relatively well in long sequence tasks.**\n\n***Q3: Theoretical Explanation of Task Similarity Regularization***\n\nIn SERS, we use Wasserstein (WS) distance to quantify task similarity. **The WS distance measures the difference between two probability distributions, defined as the minimum \"transportation cost\"** to move one distribution to another. In other fields, **WS distance has been employed for similarity quantification, such as in Wasserstein GANs[3]**, where it compares generated data to real data.\n\nThe reason it is effective in our work for quantifying task similarity is that, **in the context of continual learning, task \"similarity\" typically refers to how close their feature distributions are in the feature space.** If the feature distributions of the new and old tasks are very similar, the model is less likely to face catastrophic forgetting when learning the new task. On the other hand, if the feature distributions differ significantly, the model needs to apply stronger parameter constraints to mitigate forgetting. This is the core idea behind our similarity regularization.\n\n***Q4: Is the Pseudo-Sample Generation Specific to QA Tasks?***\n\nFor ease of visualization, we used a simple QA task in the figures of the paper. However, **our pseudo-sample generation method is generalizable to text-to-text tasks. Our dataset also includes other tasks such as question generation (QG), sentiment analysis (SA), translation (Trans), and summarization (SUM)**. As shown in the light yellow line in Figure 4, which optimizes only the pseudo-sample generation process, our method can generate high-quality pseudo-samples for these tasks.\n\nDue to the generality of our masking strategy, we believe our method could also generalize to multimodal tasks. We plan to further explore this potential in future work.\n\nWe would like to express our sincere gratitude once again for your insightful comments, and we hope the above responses address your concerns. We will attend to all comments to the best extent in the revised version.\n\n*[1] Huang, Jianheng, et al. \"Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal.\" Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.*\n\n*[2]Zhao, Weixiang, et al. \"SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models.\" Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.*\n\n*[3] Arjovsky, Martin, Soumith Chintala, and Léon Bottou. \"Wasserstein generative adversarial networks.\" International conference on machine learning. PMLR, 2017.*"}, {"review_id": "qUO4Jvl7He", "reviewer": "Reviewer_36jW", "summary": "This paper proposes Self-Evolving Pseudo-Rehearsal with Task Similarity (SERS), a continual learning framework for large language models (LLMs) for addressing catastrophic forgetting. SERS decouples pseudo-input synthesis from label creation - generating pseudo inputs via template guidance and semantic masking and uses a label self-evolution module to prevent over-specialization in pseudo labels.\nMoreover, SERS introduces a dynamic regularizer driven by the Wasserstein distance between task distributions to balance stability and plasticity. Experiments on different LLMs demonstrate consistent improvements over several rehearsal, regularization, and structure-based baselines, achieving relative gains in Rouge-L and backward transfer metrics.", "questions": "* Is continual learning LLM more of conceptual value but limited current production adoption? Likely using prompt engineer or API models with no internal training is more popular.\n* Does SERS work well with efficient tuning methods, e.g. LORA and adapters?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "* Strengths\n  * This paper is well motivated and easy to follow.\n  * The weaknesses in existing replay, pseudo-rehearsal and self-synthesis methods for continual LLM tuning paper are clearly identified, and novelty of this paper over prior methods is well illustrated.\n  * The proposed methodology is clearly described and well-justified. Comprehensive experiments across multiple tasks and models (LLaMA2-7B, ChatGLM-6B) demonstrate the method’s superiority over SOTA methods (SSR, SAPT from 2024).\n  \n* Weaknesses\n  * It is helpful to show qualitative analysis or visualization of the similarity metric.\n  * It is helpful to show qualitative analysis or visualization of the pseudo-samples.", "was_revised": true, "final_justification": "I keep my current rating as the authors addressed my questions regarding continual learning LLM vs prompt tuning, as well as applying SERS to LORA and adapters.", "rebuttal": "**To Reviewer 36jW:**\n\nWe sincerely appreciate for your constructive and valuable comments, which will play an important role in improving the paper. We hope that our responses will help you better understand our work.\n\n***Q1: Further Analysis on Task Similarity Measurement***\n\nWe apologize for the limited discussion on similarity measurement due to space constraints. From a qualitative perspective, the Wasserstein (WS) distance measures the difference between two probability distributions, defined as the minimum \"transportation cost\" required to move one distribution to another. Liu et al.[1] have provided theoretical analysis demonstrating that this method can be effectively used for quantifying task similarity. Wasserstein GANs[2], for instance, utilize this distance to assess the similarity between generated data and real data in the adversarial learning domain.\n\nIn the context of continual learning, task \"similarity\" typically refers to how closely their feature distributions align in the feature space. **If the feature distributions of new and old tasks are very similar, the model is less likely to experience catastrophic forgetting while learning the new task. On the other hand, if the feature distributions differ significantly, the model must apply stronger parameter constraints to mitigate forgetting.** This concept forms the basis of our similarity regularization approach.\n\n***Q2: Further Analysis on Pseudo-Sample Generation***\n\nWe hope that Figures 1 and Figure 7 in Appendix A help address your concern regarding the visualization of pseudo-sample generation. As shown in Figure 1, **compared to the SSR[3] method (blue), our generated pseudo samples (green) exhibit better similarity with real samples (red) and also demonstrate improved diversity, unlike SSR's more concentrated pseudo samples.** In Figure 7, we present real data from the dataset to further illustrate the differences between SSR generated and our generated pseudo samples. It is evident that the SSR approach fails to capture the essence of the \"cosmosqa\" task, producing generic QA samples that lack key common-sense reasoning cues. In contrast, our method generates pseudo samples that closely resemble real ones at low masking ratios, and even with higher masking, the outputs remain diverse while retaining the core features of the \"cosmosqa\" task.\n\n***Q3: Applications of Continual Learning and Comparison with Prompt and API Methods***\n\n1.The Widespread Demand for Continual Learning\n\nAs the scale of LLMs continues to grow, the data volume is also rapidly increasing, making it impractical to retrain the model using all data each time. On the other hand, fine-tuning only with new data can lead to catastrophic forgetting, where the model forgets previously learned knowledge. Continual learning methods exhibit significant value in addressing these challenges.\n\n2.Application of Prompt and API Methods in Continual Learning\n\nYour observation is astute, and indeed, prompt methods are commonly used in continual learning. However, **since prompt methods require fewer trainable parameters and typically freeze the LLM's core components, they are usually applied in class-incremental learning and domain-incremental learning scenarios with relatively small knowledge spans.** In contrast, task-incremental learning, which is the focus of our work, involves a greater knowledge gap between tasks, making it challenging for the lightweight, trainable parameters of prompt methods to effectively learn multi-task knowledge. As a result, most advanced task-incremental continual learning methods rely on replay, architecture, and regularization strategies.\n\n**API methods address catastrophic forgetting from a different perspective.** While they have certain advantages, they also pose challenges, notably the need to access an external API during each inference. This introduces additional latency and computational overhead. In comparison, continual learning methods offer a more efficient, cost-effective alternative by enabling the model to learn new tasks while retaining old knowledge. **By continuously training and fine-tuning the model locally, continual learning avoids the reliance on external APIs, thus reducing computational and time costs associated with external access,which remains a popular approach to address catastrophic forgetting.**\n\n***Q4: Can SERS be effectively combined with efficient fine-tuning methods such as LoRA and adapters?***\n\n**SERS is fully compatible with the LoRA method.** For the LLaMA2-7B model, we apply LoRA to the q_proj and v_proj layers; for the ChatGLM-6B model, we apply LoRA to the query_key_value layers. This approach aligns with advanced pseudo-sample replay methods such as SSR.\n\nAs for the adapter method, although we employ LoRA for PEFT in SERS rather than the adapter method, **we do not make architectural changes to the model. Therefore, even with the use of adapter methods, we believe they can be effectively combined with SERS.** We will explore and incorporate methods that combine SERS with adapter techniques to provide additional PEFT options.\n\nWe would like to express our sincere gratitude once again for your insightful comments, and we hope the above responses address your concerns. We will attend to all comments to the best extent in the revised version.\n\n*[1] Liu, Xinran, et al. \"Wasserstein task embedding for measuring task similarities.\" Neural Networks 181 (2025): 106796.*\n\n*[2 ]Arjovsky, Martin, Soumith Chintala, and Léon Bottou. \"Wasserstein generative adversarial networks.\" International conference on machine learning. PMLR, 2017.*\n\n*[3] Huang, Jianheng, et al. \"Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal.\" Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.*"}]

Author rebuttal:

Meta-review: This paper addresses catastrophic forgetting in LLM continual fine-tuning by introducing a pseudo-rehearsal framework that combines semantic masking, label self-evolution, and task-similarity regularization. The reviewers were generally positive finding the method motivated and strong experiments on multiple LLMs and diverse tasks, showing consistent improvements. While reviewers raised concerns about the justification of some design choices the main concerns were addressed in the rebuttal. Overall, the paper makes a technically solid and timely contribution
Decision: Accept (poster)