Overcoming Catastrophic Forgetting: A Novel Fine-Tuning Method

Authors: Fei Ding
Primary area: transfer learning, meta learning, and lifelong learning
Keywords: lifelong learning

Abstract:
Despite remarkable advances in Large Language Models (LLMs), a persistent challenge remains: the potential for these models to acquire erroneous or outdated information from their training data. Direct fine-tuning with data containing new knowledge can be ineffective due to conflicts between old and new knowledge. This paper proposes a novel fine-tuning paradigm called Delicate Fine-Tuning (DFT ) that leverages parametric arithmetic to pinpoint the location of knowledge and update only the minimal set of relevant parameters. Experimental results on two publicly available datasets demonstrate that our proposed DFT significantly improves the knowledge updating performance of full fine-tuning, consistently outperforming existing baselines in most cases.

Decision: desk reject

Reviews and discussion:

Metareview: