Title: Self-Evolving Pseudo-Rehearsal for Catastrophic Forgetting with Task Similarity in LLMs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Continual Learning; Large Language Models; Pseudo Sample Rehearsal; Similarity Regularization
Abstract: Continual learning for large language models (LLMs) demands a precise balance between $\textbf{plasticity}$ - the ability to absorb new tasks - and $\textbf{stability}$ - the preservation of previously learned knowledge. Conventional rehearsal methods, which replay stored examples, are limited by long-term data inaccessibility; earlier pseudo-rehearsal methods require additional generation modules, while self-synthesis approaches often generate samples that poorly align with real tasks, suffer from unstable outputs, and ignore task relationships. We present $\textbf{\textit{Self-Evolving Pseudo-Rehearsal for Catastrophic Forgetting with Task Similarity}}(\textbf{SERS})$, a lightweight framework that 1) decouples pseudo-input synthesis from label creation, using semantic masking and template guidance to produce diverse, task-relevant prompts without extra modules; 2) applies label self-evolution, blending base-model priors with fine-tuned outputs to prevent over-specialization; and 3) introduces a dynamic regularizer driven by the Wasserstein distance between task distributions, automatically relaxing or strengthening constraints in proportion to task similarity. Experiments across diverse tasks on different LLMs show that our SERS reduces forgetting by over 2\% points against strong pseudo-rehearsal baselines, by ensuring efficient data utilization and wisely transferring knowledge. 
The code will be released at https://github.com/JerryWangJun/LLM_CL_SERS/.


Peer reviews:
comments: SUMMARY:
This paper proposes Self-Evolving Pseudo-Rehearsal with Task Similarity (SERS), a continual learning framework for large language models (LLMs) for addressing catastrophic forgetting. SERS decouples pseudo-input synthesis from label creation - generating pseudo inputs via template guidance and semantic masking and uses a label self-evolution module to prevent over-specialization in pseudo labels.
Moreover, SERS introduces a dynamic regularizer driven by the Wasserstein distance between task distributions to balance stability and plasticity. Experiments on different LLMs demonstrate consistent improvements over several rehearsal, regularization, and structure-based baselines, achieving relative gains in Rouge-L and backward transfer metrics.

STRENGTHS AND WEAKNESSES:
* Strengths
  * This paper is well motivated and easy to follow.
  * The weaknesses in existing replay, pseudo-rehearsal and self-synthesis methods for continual LLM tuning paper are clearly identified, and novelty of this paper over prior methods is well illustrated.
  * The proposed methodology is clearly described and well-justified. Comprehensive experiments across multiple tasks and models (LLaMA2-7B, ChatGLM-6B) demonstrate the method’s superiority over SOTA methods (SSR, SAPT from 2024).
  
* Weaknesses
  * It is helpful to show qualitative analysis or visualization of the similarity metric.
  * It is helpful to show qualitative analysis or visualization of the pseudo-samples.

QUESTIONS:
* Is continual learning LLM more of conceptual value but limited current production adoption? Likely using prompt engineer or API models with no internal training is more popular.
* Does SERS work well with efficient tuning methods, e.g. LORA and adapters?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-29

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work introduces a novel method for fine-tuning LLMs within the continual learning setup. The method is a combination of a pseudo-rehearsal and regularization strategies. The evaluation of the method shows a clear advantage of the method over the baselines and closes the gap to the Multi-Task reference, which is a sign of a really strong performance.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n1. The paper is written clearly, and all the concepts, motivations behind the whole work, and respective experiments are well articulated. \n2. The introduced method is correctly evaluated using a metric typical for continual learning. In both metrics (AR, BWT), the method beats other baselines, showing strong performance. Additionally, the authors took care of the order of tasks, which is known to be an important factor in CL experiments.\n3. The idea to create Semantic guided Pseudo-Input and Self-Evolving Labels is interesting, and through ablations and examples, the authors highlight the role of these components in the overall performance.\n4. The method requires storing as little as 1\\% of the overall data samples to use them as a \"skeleton\" for artificial sample creation. This is a nice idea to reduce the requirements for the storage of samples. I'm wondering how far this process can go. I.e., I'd like to see the performance degradation of the overall method when further reducing the number of samples stored in the buffer. These experiments would nicely complement the findings from the experiment in Figure 5.\n### Weaknesses\n1. The length of the take sequence is relatively short. While I'm aware that it's harder to create equally long sequences of tasks as in the case of computer vision datasets, we know that longer sequences have a different set of challenges; thus, it'd be beneficial to see how this method works on such tasks.\n2. There is little elaboration on why Wasserstein Distance is the right way to measure task similarity. While I like the idea of adjusting the level of regularization based on the task similarity, I find it quite disappointing that this (quite important!) choice is not discussed (or compared) in a broader context.  \n3. The proposed method for generating pseudo-samples seems to be particularly suited for the specific QA type of datasets. Do you have any ideas how this method could be adjusted to make it applicable to other types of datasets? Have you performed any experiments with different datasets?\n\nQUESTIONS:\nI've listed most of my questions in the Strengths & Weaknesses section.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents Self-Evolving Pseudo-Rehearsal with Task Similarity (SERS), a novel continual learning framework designed to mitigate catastrophic forgetting in large language models (LLMs). SERS introduces three key innovations: (1) it decouples pseudo-input generation from label synthesis by leveraging semantic masking and template-guided prompt construction; (2) it incorporates a self-evolution mechanism that adaptively fuses base and task-specific label signals to improve generalization; and (3) it employs a task-similarity-aware regularizer based on the Wasserstein distance to dynamically control the stability–plasticity trade-off. Evaluations on the SuperNI benchmark using LLaMA2-7B and ChatGLM-6B across diverse task permutations demonstrate that SERS consistently outperforms state-of-the-art pseudo-rehearsal and regularization methods, and in some cases achieves performance comparable to or better than multi-task training.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The SERS framework integrates pseudo-rehearsal with dynamic, task-similarity-aware regularization, jointly addressing the stability–plasticity dilemma and the fidelity of pseudo-samples. Its use of semantic masking and template-guided input synthesis is well-motivated and yields pseudo-data that closely aligns with the true task distribution.\n2. The proposed two-stage mechanism, which consists of a self-questioning phase followed by the integration of outputs from both pre-trained and task-adapted models, aligns well with recent developments in self-evolution frameworks. This design effectively mitigates over-specialization by maintaining a balance between generalization and task-specific adaptation.\n\nWeakness:\n1. While the proposed method is effective in benchmark settings, its applicability to real-world domain-specific scenarios, such as medicine or law,  is not well explored. In such cases, large language models already perform well on general tasks, and continual learning becomes necessary only when adapting to specialized domains. However, determining which domain-specific knowledge should be retained and what type of pseudo-samples should be synthesized remains an open and non-trivial challenge.\n2. The framework relies on measuring task similarity to guide regularization, but the method for quantifying meaningful relationships between heterogeneous tasks is insufficiently justified. For example, assessing similarity between a mathematical reasoning task and a clinical diagnosis task poses significant challenges due to their fundamentally different semantics. Although the experiments are conducted on the SuperNI benchmark, the task diversity in this dataset may not fully capture the complexity of real-world task heterogeneity.\n\nQUESTIONS:\nsee above\n\nLIMITATIONS:\nsee above", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses a problem of pseudo sample replay in continual learning, where the generated pseudo samples cannot well reflect the original knowledge structure and thus provide limited support for rehearsal. It proposes a method that first generates pseudo samples with semantic guidance and masked inputs, then performs label self-evolution by mixing the outputs of the current and initialized LLM. It also adds a regularization term to encourage more knowledge transfer across similar tasks. Results show that the proposed method outperforms other baselines.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper studies an important pseudo rehearsal problem for CL with LLM.\n2. The experiments are performed on two LLMs, with thorough comparison to baselines. Ablation studies are included.\n\n**Weaknesses**\n1. Some details are unclear in the method:\n* When masking the real sample for pseudo sample generation, how to decide the tokens to mask? Is it random masking? If so, the masked place may not contain much semantic information.   \n* How to integrate the output from different models? Is it combining a sequence of output tokens or representations? What are the real output tokens after integration?\n\n2. The intuition for task similarity regularization is unclear to me. During pseudo example generation and label evolution, the model already preserves the information of LLM_0 for all tasks. Then why should it take additional regularization for task similarity? If the tasks in the CL sequence are dissimilar, will it limit the model’s plasticity to learn each task?\n\nQUESTIONS:\nWhat is the performance gap between replay with pseudo examples and real examples? In the rehearsal analysis in Fig 5, using 1% real data can generate better pseudo samples for rehearsal than using 5%. Does using 1% real data also outperforms 5% real data in rehearsal?\n\nLIMITATIONS:\nThe model needs extra LLM forwarding to generate pseudo examples and label evolution. It also needs extra memory to store the LLM_0 and LLM_0, which is natural for PEFT models but have extra cost for fine-tuning models.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}