Arcee’s M erge K it: A Toolkit for Merging Large Language Models (2024)

Abstract: The rapid growth of open-source language models provides the opportunity to merge model checkpoints, combining their parameters to improve performance and versatility. Advances in transfer learning have led to numerous task-specific models, which model merging can integrate into powerful multitask models without additional training. MergeKit is an open-source library designed to support this process with an efficient and extensible framework suitable for any hardware. It has facilitated the merging of thousands of models, contributing to some of the world’s most powerful open-source model checkpoints. The library is accessible at: https://github.com/arcee-ai/mergekit.

## introduction over the past year, open-source large language models (llms) have rapidly developed and are accessible via the hugging face model hub @xcite . these models, trained on up to trillions of tokens, typically range from 1-70+ billion parameters @xcite . open-source checkpoints include pretrained and instruction-tuned models across domains like coding @xcite and medical applications @xcite . fine-tuning separate models for each task presents challenges: storing and deploying each model separately and the inability of independently trained models to leverage insights from related tasks @xcite @xcite . training these models from scratch requires substantial investment. further fine-tuning can lead to catastrophic forgetting @xcite , degrading their general capabilities and performances across tasks @xcite . aligning models to respond favorably requires extensive human preference data, often unattainable for most teams @xcite . this raises the question of leveraging existing pretrained checkpoints. model merging has emerged as a transformative strategy, combining parameters from multiple models into a single one, enabling multitask and continual learning while reducing catastrophic forgetting @xcite . in this paper, we introduce mergekit foot_0 , a centralized library for executing community-formulated merging strategies, compatible with memoryconstrained cpus and accelerated gpus. our main contributions are: (1) an overview of current model merging research to date and (2) a presentation of mergekit's key objectives, architectural decisions, and development principles to establish an extensible foundation for the future efforts of the model merging community. ## library design: key design principles mergekit has been thoughtfully engineered to facilitate the straightforward application of both current and forthcoming model merging techniques. our repository includes detailed tutorials and ipython foot_5 notebooks to guide users through the process of utilizing mergekit effectively. this section is dedicated to outlining the fundamental design principles underpinning the library, with the aim of assisting the open-source community in adopting our toolkit and incorporating new techniques. ## extensibility of mergekit given the rapid success of model merging techniques and the anticipated development of innovative methods, we invite the community to develop novel merging strategies and enhancements, thereby contributing to the growth and refinement of mergekit. this section aims to provide a stream-lined guide on integrating new merging methods into mergekit, utilizing existing functionalities where applicable to facilitate the process. to incorporate a new merging method into mergekit, contributors should familiarize themselves with several key python modules within the repository: ## conclusion and future work in this paper, we introduce mergekit, an innovative open-source tool for seamlessly integrating llms. we detail its functionalities and provide an overview of recent model merging literature from an engineering perspective. additionally, we offer insights on incorporating new merging techniques, encouraging community contributions. mergekit is a dynamic project, committed to continuously integrating new methodologies through collaborative efforts with the open-source community.