LM -Cocktail: Resilient Tuning of Language Models via Model Merging (2024)

Abstract: The pre-trained language models are continually fine-tuned to better support downstream applications. However, this operation may result in significant performance degeneration on general tasks beyond the targeted domain. To overcome this problem, we propose LM-Cocktail which enables the fine-tuned model to stay resilient in general perspectives. Our method is conducted in the form of model merging, where the fine-tuned language model is merged with the pre-trained base model or the peer models from other domains through weighted average. Despite simplicity, LM-Cocktail is surprisingly effective: the resulted model is able to achieve a strong empirical performance in the whole scope of general tasks while preserving a superior capacity in its targeted domain.

## introduction language models (lm) are fundamental pillars of artificial intelligence and natural language processing. thanks to the considerable expansion of training scale and model size @xcite @xcite @xcite , language models have made remarkable breakthroughs on a wide variety of nlp tasks, including representation, understanding, reasoning, and generation. in many of the applications, language models are frequently used via the "pretraining and fine-tuning" paradigm. particularly, a generalist lm is pre-trained in the first place through an unsupervised or general-purpose supervised learning process @xcite @xcite @xcite ; then, the pre-trained generalist model is finetuned to be a specialist model for a down-stream task on top of certain in-domain data. ## experimental setup we conducted experiments with two types of models: decoder-based lm and encoder-based lm. we fine-tuned 9 encoder-based models and 9 decoderbased models separately, and then evaluated the performance of fine-tuned models and resilient-tuned models. ## experimental results in this section, we show the experimental results and represent the key findings. firstly, we compare the performance of fine-tuned models and resilienttuned models. next, we evaluate the performance of lm-cocktail when fine-tuning on target task is unavailable. finally, we investigate the impact of weight α and the number of examples. we also investigate the effectiveness of lm-cocktail under different fine-tuning strategies and base models. additionally, we compare the effects of various merging methods. these results are presented in the appendix b. ## conclusion in this work, we introduce the lm-cocktail, a simple method to improve performance on target tasks without decreasing accuracy on other unrelated tasks. lm-cocktail produces a resilient-tuned model by weighted averaging the parameters from different models: the model fine-tuned on the target task, the pre-trained base model, and the peer models from other domains. the empirical results on both decoder and encoder models demonstrate that lm-cocktail can achieve strong performance in the whole scope of general tasks while preserving a superior capacity in its targeted domain. we further demonstrated the effectiveness of lm-cocktail when unable to fine-tune on domain-specific data.