LMF low: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models (2024)

Abstract: Foundation models have demonstrated a great ability to achieve general human-level intelligence far beyond traditional approaches. As the technique keeps attracting attention from the AI community, more and more foundation models have become publicly available.However, most of those models exhibit a major deficiency in specialized-domain and specialized-task applications, where the step of domain- and task-aware finetuning is still required to obtain scientific language models. As the number of available foundation models and specialized tasks keeps growing, the job of training scientific language models becomes highly nontrivial. In this paper, we take the first step to address this issue. We introduce an extensible and lightweight toolkit, LMFlow, which aims to simplify the domain- and task-aware finetuning of general foundation models.LMFlow offers a complete finetuning workflow for a foundation model to support specialized training with limited computing resources.Furthermore, it supports continuous pretraining, instruction tuning, parameter-efficient finetuning, alignment tuning, inference acceleration, long context generalization, model customization, and even multimodal finetuning, along with carefully designed and extensible APIs. This toolkit has been thoroughly tested and is available at https://github.com/OptimalScale/LMFlow .

## introduction foundation models (fms), and in particular large language models (llms), have demonstrated general abilities to perform different tasks beyond what was possible previously. while a number of pretrained large models, including gpt-j @xcite , @xcite , llama @xcite , etc., are publicly available and have already been incorporated into the hugging face model repository @xcite , there is no publicly available toolkit that can be easily used to perform finetuning and inference for these different models. for specialized domains or tasks, it is necessary to further finetune such llms to achieve improved performance on such domains or tasks. the purpose of this package is to offer a simple-to-use and lightweight toolkit so that developers and researchers can perform efficient finetuning and inference of scientific language models with limited resources. the typical processes to train a scientific language model are shown in figure 1 , which include: ## related work in recent years, the finetuning of large language models (llms) has gained significant attention, especially for scientific domain applications. the necessity of adapting these general-purpose models to specific domains or tasks has led to the development of various scientific language models. @xcite conducted an extensive empirical analysis on the performance of various language models in clinical tasks and found that specialized clinical models, even smaller in size, significantly outperform larger general-domain models when finetuned on domain-specific data. this emphasizes the importance of domain specialization in achieving higher accuracy in safety-critical fields like healthcare. therefore, a series of scientific large models have emerged, including but not limited to: language models for science @xcite @xcite , mathematics @xcite @xcite , physics @xcite @xcite , chemistry and materials science @xcite @xcite , biology and medicine @xcite @xcite @xcite @xcite , and information retrieval @xcite we recommend readers to refer to a paper list of scientific language models foot_0 , which includes a more comprehensive range of works related to scientific language models. among these works, lmflow has successfully helped in training astrollama-chat @xcite and marinegpt @xcite . the medical llama trained in the medical domain within this paper also demonstrates the effectiveness of lmflow. in summary, our proposed lmflow offers a comprehensive toolkit for efficient and effective finetuning of foundation models across various specialized domains. ## api documentation please refer to https://optimalscale.github. io/lmflow/autoapi/index.html for the details of api documentation. ## results in this section, we will provide experimental results and case studies of lmflow in task tuning, instruction tuning, and alignment tuning. ## conclusion in conclusion, the lmflow toolkit offers an extensible, lightweight, and easy-to-use solution for developers and researchers to perform efficient training of scientific language models with limited resources. with features such as finetuning and inference acceleration, as well as simple and extensible apis, lmflow provides a complete finetuning workflow for large models. moreover, with the ability to customize training and achieve comparable or even better performance than chatgpt, lmflow represents a significant step forward in the development of large scientific models and their application to specialized tasks.