S elf A ug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignment (2025)

Abstract: Recent advancements in large language models (LLMs) have revolutionized natural language processing through their remarkable capabilities in understanding and executing diverse tasks. While supervised fine-tuning, particularly in Retrieval-Augmented Generation (RAG) scenarios, effectively enhances task-specific performance, it often leads to catastrophic forgetting, where models lose their previously acquired knowledge and general capabilities. Existing solutions either require access to general instruction data or face limitations in preserving the model’s original distribution. To overcome these limitations, we propose SelfAug, a self-distribution alignment method that aligns input sequence logits to preserve the model’s semantic distribution, thereby mitigating catastrophic forgetting and improving downstream performance. Extensive experiments demonstrate that SelfAug achieves a superior balance between downstream learning and general capability retention. Our comprehensive empirical analysis reveals a direct correlation between distribution shifts and the severity of catastrophic forgetting in RAG scenarios, highlighting how the absence of RAG capabilities in general instruction tuning leads to significant distribution shifts during fine-tuning. Our findings not only advance the understanding of catastrophic forgetting in RAG contexts but also provide a practical solution applicable across diverse fine-tuning scenarios.

## introduction large language models (llms) like gpt @xcite , palm @xcite , glm @xcite , and llama @xcite have revolutionized nlp by learning complex linguistic patterns from extensive pre-training * corresponding author data, demonstrating excellence in contextual understanding and few-shot learning capabilities. supervised fine-tuning @xcite with general instruction datasets @xcite improves models' instruction following abilities but often inadequately addresses specialized domain tasks. task-specific fine-tuning provides targeted solutions for specialized applications @xcite @xcite @xcite . particularly, retrieval-augmented generation (rag) @xcite @xcite @xcite @xcite enhances llms by incorporating external knowledge through retrieval, reducing hallucinations. recent work @xcite @xcite improves how models utilize relevant information and handle insufficient information. however, fine-tuning for downstream tasks introduces catastrophic forgetting @xcite @xcite @xcite , where models lose previously acquired knowledge and instructionfollowing abilities when adapting to new tasks. this causes performance deterioration across diverse applications. for example, a model finetuned on document extraction may generate structurally incorrect code, despite improved document parsing abilities. recent research attributes this problem to distribution shift when models adapt to specialized task distributions during fine-tuning @xcite . to address capability degradation, recent studies @xcite @xcite suggest incorporating general instruction data during downstream fine-tuning to maintain llm's general capabilities. however, these strategies are limited by the scarcity of publicly available instruction datasets. researchers have therefore explored alternative approaches that retain the model's original distribution without accessing general data. instruction synthesis methods like magpie @xcite use the model to generate instruction-response pairs for data replay, though they depend heavily on generation quality. parameter constraint methods such as orthogonal loss @xcite enforce orthogonality between parameters but compromise downstream task performance. knowledge reconstruction approaches like sdft @xcite approximate the original distribution by regenerating responses from fine-tuning data but struggle with format-specific tasks, particularly when structured outputs like json are required. while each approach offers certain benefits, they all have limitations. these limitations underscore the need for more efficient solutions that better balance capability preservation and task adaptation. to address the limitations of previous methods, we propose selfaug, a novel approach that improves downstream performance while preserving the model's original capabilities. selfaug is flexible and applicable to various fine-tuning scenarios. the core idea is to align the logits of input sequences during fine-tuning, leveraging the rich information in these logits generated by large language models during sequential processing. these logits reflect both learned knowledge and decision boundaries, helping to prevent catastrophic forgetting and ensuring the model's behavior remains consistent while enabling it to learn new tasks @xcite . our analysis reveals that catastrophic forgetting is most severe in rag scenarios, especially when longer reference documents are used. we find a direct link between larger distribution shift and greater forgetting, as well as a correlation between longer contexts and more significant shift. selfaug effectively mitigates these issues, achieving performance similar to lora while maintaining the model's original abilities, demonstrating the value of aligning logits distributions @xcite . the main contributions of this work are as follows: ## method in this section, we first outline the output logits of llms and the fine-tuning process. subsequently, we introduce our selfaug method and provide details on its implementation. ## experiment to evaluate the effectiveness of selfaug and its impact across different scenarios, we aim to answer the following research questions: ## conclusion our research explores the problem of catastrophic forgetting when fine-tuning language models for retrieval-augmented generation tasks. we find that distribution shift during fine-tuning weakens the model's general performance, especially its ability to follow instructions. to address this, we propose selfaug, a method that does not use data replay or change the model architecture, and can be applied to any fine-tuning setting. selfaug uses only the original training data and aligns the model's input distributions by constraining input sequence logits. this simple approach reduces distribution shift and helps prevent catastrophic forgetting. our experiments show that there is a clear link between distribution shift and catastrophic forgetting. self-aug reduces this shift and preserves model abilities, while matching or exceeding the downstream task performance of standard fine-tuning methods.