NeurIPS 2025

Investigating and Mitigating Catastrophic Forgetting
in Medical Knowledge Injection through Internal
Knowledge Augmentation Learning
Yuxuan Zhou1, Xien Liu∗1, Xiao Zhang1, Chen Ning1, Shijin Wang2, Guoping Hu2, Ji Wu1,3,4
1Department of Electronic Engineering, Tsinghua University, Beijing, China
2iFLYTEK Research, Hefei, China
3College of AI, Tsinghua University, Beijing, China
4Beijing National Research Center for Information Science and Technology, Beijing, China
Abstract
Large Language Models (LLMs) are expected to possess comprehensive medical
knowledge to support real-world clinical applications. While domain-specific
fine-tuning effectively injects medical knowledge into LLMs, it often causes catas-
trophic forgetting of previously acquired knowledge and instruction-following ca-
pabilities. In this paper, we investigate this issue and reveal a pattern of proximity-
dependent forgetting: knowledge that is semantically or topically close to the
injected content is more likely to be forgotten, while unrelated knowledge shows
minimal degradation. Moreover, we observe that existing mitigation techniques
fail to address this type of forgetting effectively. Motivated by this observation
and inspired by human learning mechanisms, we propose InternAL (Internal
Knowledge Augmentation Learning), a novel approach that leverages LLMs’ own
internal knowledge to mitigate forgetting. InternAL first probes internal knowledge
closely related to the injection by prompting the model with questions derived
from the injected knowledge. This knowledge is then used to augment the original
injection dataset, guiding the model to retain related prior knowledge during train-
ing. Experimental results on multiple LLMs (LLaMA, Qwen) demonstrate that
InternAL significantly mitigates proximity-related forgetting while maintaining
strong knowledge injection performance. Our findings provide new insights into
the nature of catastrophic forgetting in medical knowledge injection and highlight
...