L o RAM o E : Alleviating World Knowledge Forgetting in Large Language Models via M o E -Style Plugin (2024)

Abstract: Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks. Substantially increasing instruction data is a direct solution to align the model with a broader range of downstream tasks or notably improve its performance on a specific task. However, we find that large-scale increases in instruction data can damage the world knowledge previously stored in LLMs. To address this challenge, we propose LoRAMoE, a novelty framework that introduces several low-rank adapters (LoRA) and integrates them by using a router network, like a plugin version of Mixture of Experts (MoE). It freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting. Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM. Our code is available at https://github.com/Ablustrund/LoRAMoE.

## introduction supervised fine-tuning (sft) provides a pivotal technique to make large language models (llms) follow human instructions and improve their performance of downstream tasks @xcite . although some studies @xcite indicate that llms trained on a little data can follow instructions well, increasing the amount of data is a straightforward with the amount of instruction data increasing, finetuning the language models results in a decline in performance on the benchmarks that measure their world knowledge, such as triviaqa @xcite , natural questions @xcite . the details of training implementation can be seen in section 2.1. however, the large-scale increase in instruction data can destroy the world knowledge stored in llms, as illustrated in the right of figure 1 . specifically, as the amount of instruction data increases, we observe a notable decline in performance on closed-book question answering (cbqa) datasets, which are used to measure world knowledge in llms @xcite . in the paradigm of supervised fine-tuning, the conflict between maintaining world knowledge inside llms and improving their performance on downstream tasks by scaling up instruction data has not been thoroughly examined. in this paper, we propose loramoe, a novel framework for sft, to enhance the models' capability of solving downstream tasks, while alleviat- w s c w in o g ra n d e f lo re s x s u m r a c e -m id d le r a c e -h ig h r t e r e c o r d a x -g m u lt ir c -p p l f il te re d n q t ri v ia q a h o tp o tq a n q f il te re d t ri v ia q a task 0 20 40 60 80 performance 1000 3 million 5 million data scale downstream tasks world knowledge benchmarks figure 2: performance on the various tasks after expanding the amount of fine-tuning data. for most of the downstream tasks (e.g., nli and summarization), with the expansion of training data, performance on these tasks remains stable after the improvement. whereas, for the world knowledge benchmark, a significant decline can be witnessed after a large amount of instruction data. to demonstrate the efficacy of our proposed method, we conduct extensive experiments across a range of downstream tasks. experiment results show that loramoe can significantly improve llm's ability to address the various downstream tasks by fine-tuning the model on a large amount of instruction data, while maintaining the world knowledge stored in the model. in addition, we further evaluate our method by visualizing the expert weight for tasks. the result indicates that loramoe adequately alleviates world knowledge forgetting and achieves an improvement of models by fostering collaboration among experts. the main contributions of our paper are as follows: ## motivation in this section, we verify that a large-scale sft can cause irreversible damage to world knowledge within the llms while improving the llms' performance in various downstream tasks. ## loramoe in this section, we elaborate on the methodological details of loramoe, which is an moe-style plugin and introduced localized balancing constraint during the training phase to alleviate the world knowledge, as shown in figure 4 . ## conclusion in this paper, we first delve into the conflict between improving llm's performance on downstream tasks by scaling up data during the sft phase and discouraging world knowledge forgetting. to address this conflict, we then introduce lo-ramoe, a novel framework for sft, which introduces loras as experts and integrates them by the router. extensive experimental results demonstrate that loramoe can foster collaboration among experts to enhance the model's performance of downstream tasks, while preserving the world knowledge inside it.