Title: Investigating and Mitigating Catastrophic Forgetting in Medical Knowledge Injection through Internal Knowledge Augmentation Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Language Model

Medical Knowledge Injection

Catastrophic Forgetting
Abstract: Large Language Models (LLMs) are expected to possess comprehensive medical knowledge to support real-world clinical applications. While domain-specific fine-tuning effectively injects medical knowledge into LLMs, it often causes catastrophic forgetting of previously acquired knowledge and instruction-following capabilities. In this paper, we investigate this issue and reveal a pattern of proximity-dependent forgetting: knowledge that is semantically or topically close to the injected content is more likely to be forgotten, while unrelated knowledge shows minimal degradation. Moreover, we observe that existing mitigation techniques fail to address this type of forgetting effectively. Motivated by this observation and inspired by human learning mechanisms, we proposeInternAL (\Internal Knowledge Augmentation Learning), a novel approach that leverages LLMs' own internal knowledge to mitigate forgetting. InternAL first probes internal knowledge closely related to the injection by prompting the model with questions derived from the injected knowledge. This knowledge is then used to augment the original injection dataset, guiding the model to retain related prior knowledge during training. Experimental results on multiple LLMs (LLaMA, Qwen) demonstrate that InternAL significantly mitigates proximity-related forgetting while maintaining strong knowledge injection performance. Our findings provide new insights into the nature of catastrophic forgetting in medical knowledge injection and highlight a promising direction for robust domain adaptation in LLMs. Code and datasets are available at https://github.com/THUMLP/InternAL.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates catastrophic forgetting in Large Language Models (LLMs) during domain-specific fine-tuning, particularly for medical knowledge injection. The authors identify a \"proximity-dependent\" forgetting pattern, where knowledge semantically close to the newly injected content is most likely to be forgotten. Finding existing mitigation techniques insufficient, they propose a novel method called InternAL (Internal Knowledge Augmentation Learning). This approach probes the LLM for its own internal knowledge related to the facts being injected and then uses this retrieved knowledge to augment the fine-tuning dataset. By co-training on both the new knowledge and related existing knowledge, InternAL significantly mitigates catastrophic forgetting, especially for closely related concepts, while maintaining strong performance on the injected knowledge. The method's effectiveness is demonstrated on multiple LLMs, such as LLaMA and Qwen.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper addresses the critical and highly practical problem of catastrophic forgetting during domain-specific fine-tuning of LLMs. This is particularly significant in high-stakes domains like medicine, where retaining broad knowledge while incorporating new, specialized facts is essential for safe and effective real-world application. \n2. It leverages the model's own internal knowledge, probed and extracted based on the new knowledge being injected, to create a rehearsal dataset. This is a creative and resource-efficient solution inspired by human learning\n\nWeaknesses:\n It is unclear how these findings on proximity-dependent forgetting would translate to other knowledge-intensive domains like law, finance, or even more general commonsense reasoning.\n\nQUESTIONS:\nQ1.Could the authors provide an analysis of the quality of the knowledge extracted by the f_probe module? I suggest conducting a small-scale manual evaluation on a random sample.\n\nQ2. Could the authors please provide more specific details on the negative sampling strategy used in the RefInject method? Specifically, are the m-1 negative tail entities sampled uniformly at random from the entire PrimeKG knowledge graph, or are they \"hard negatives\" (e.g., entities of the same type as the true answer, or entities that frequently co-occur with the head entity)?\n\nLIMITATIONS:\nNo potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 1, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the problem of catastrophic forgetting in large language models (LLMs) during domain-specific knowledge injection, focusing on the medical domain. The authors identify a proximity-dependent forgetting phenomenon, where knowledge that is semantically close to the injected content is more likely to be forgotten. To address this, they propose InternAL, a novel method that leverages the LLM’s own internal knowledge to augment the injection data.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1.  The proposed InternAL method is simple and easy to implement, yet achieves strong empirical performance in mitigating catastrophic forgetting.\n\n2.  The paper provides a valuable analysis revealing a proximity-dependent forgetting pattern.\n\n\n**Weakness**\n\n1.  One limitation of this work lies in its reliance on multiple-choice questions to construct the training data. This results in a constrained learning format that may fail to reflect the diverse forms in which knowledge is typically conveyed. In real-world scenarios, knowledge injection should ideally leverage free-form text, such as medical textbooks or clinical narratives. Similarly, evaluation should incorporate a broader range of question types to better assess the model’s generalization ability.\n\n2.  While InternAL demonstrates empirical effectiveness in mitigating proximity-dependent forgetting, its underlying mechanism remains relatively simple. Fundamentally, InternAL can be seen as a data augmentation strategy that enriches the training set by incorporating prior knowledge closely related to the injection targets. It shares conceptual similarities with replay-based methods.\n\n3.  InternAL relies on self-generated knowledge without explicit quality control, which may introduce noise or reinforce hallucinated content during training.\n\n4.  The paper does not provide a representation-level analysis to explain how InternAL preserves semantically proximal knowledge, leaving its internal mechanisms somewhat opaque.\n\nQUESTIONS:\nHow is the MEMIT method adapted to the multiple-choice question format used for knowledge injection in this paper?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focusing on solving the catastrophic forgetting problem in LLMs happend during knowledge injection in medical domain. Firstly, the authors argue that they observed that LLMs tend to forgetting knowledge related to the new knowledge that will be injected. Based on this discovery, they propose a method that will first probe the knowledge which closely related to the injecting knowledge. Then, they use the extracted knowledge to augment the knowledge materials to help the LLMs to retain the information.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper presents a clear and practically meaningful objective: addressing the problem of catastrophic forgetting in LLMs during the process of knowledge injection. The authors highlight an intriguing observation—LLMs tend to forget pre-existing knowledge that is closely related to the newly injected content. To mitigate this issue, the paper proposes a simple yet effective solution, offering valuable insights into the continual learning dynamics of LLMs.\n2. To evaluate the effects of knowledge forgetting and injection, the authors design a comprehensive and well-structured experimental setup. They partition the knowledge data into two sets: one with low factual accuracy, used for the knowledge injection experiments, and another with high factual accuracy, employed to assess the extent of knowledge forgetting.\n3. This work is well-illustrated. The author first use enough experiments to show their claim holds. Then introduce their proposed method in a expectable manner. Readers will accept their proposed method more smoothly.\n\nWeaknesses:\n1. One potential concern I have with this work is that the authors’ definition of “related knowledge” appears somewhat overly broad. For instance, in some parts of the paper, all knowledge within the medical domain is treated as related, while in specific experiments, a distinction is made between proximal and distal knowledge. This inconsistency may affect the clarity and interpretability of the findings.\n2. A second concern lies in the generalizability of the proposed approach. While the problem identified by the authors is indeed broadly relevant, the experiments are conducted on a relatively structured and constrained setting—primarily focusing on knowledge represented in the form of triples. However, in many real-world scenarios, knowledge does not naturally conform to such structured formats. It remains unclear how the proposed method would extend to more unstructured or complex forms of knowledge representation.\n\nQUESTIONS:\n1. Can you briefly why are you choosing the RefInject method to do the knowledge injection? Is it possible that the knowledge relativeness pattern you observed will only happen for this injection method?\n2. Did I miss something? Why the models performance dropped on EVERy datasets after injection? I expected some improvement on specific dataset after injection (Otherwise, what is the point of injection?)\n3. L197-198, why are the numbers in your lines are different from the numbers in the table?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper investigates catastrophic forgetting in LLMs when injecting medical knowledge via fine-tuning. The authors reveal a proximity-dependent pattern of forgetting: knowledge semantically close to the injected content is more susceptible to being overwritten, while unrelated knowledge is largely unaffected. The study empirically assesses several existing mitigation techniques and finds their performance lacking in the medical setting, particularly for knowledge close to injected facts. Inspired by mechanisms of human learning, the authors introduce InternAL, which augments the injection dataset by probing the LLM for its own closely related internal knowledge and incorporating these outputs into fine-tuning. Experiments with LLama3-8B and Qwen3-8B across a range of medical and general benchmarks demonstrate that InternAL significantly reduces proximity-based forgetting while maintaining injection effectiveness.

STRENGTHS AND WEAKNESSES:
Strengths:
Well-Motivated Problem Formulation: The paper addresses an under-explored and practically pertinent issue: catastrophic forgetting during domain-specific knowledge injection in LLMs, with a focus on the medical domain.
Practical solution : InternAL is simple yet effective, leveraging the model’s own knowledge to mitigate forgetting.

Weaknesses:
Limited scope : Only tested in medical domain; other domains might behave differently.
Model size limitation : Experiments only cover 8B-parameter models (LLama, Qwen). Larger (e.g., 30B) or smaller (e.g., 1B) models may exhibit different forgetting dynamics, but this is untested.

QUESTIONS:
Check the weakness part mentioned above.

LIMITATIONS:
Check the weakness part mentioned above.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-06