LLM -Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models (2023)

Abstract: The success of large language models (LLMs), like GPT-4 and ChatGPT, has led to the development of numerous cost-effective and accessible alternatives that are created by finetuning open-access LLMs with task-specific data (e.g., ChatDoctor) or instruction data (e.g., Alpaca). Among the various fine-tuning methods, adapter-based parameter-efficient fine-tuning (PEFT) is undoubtedly one of the most attractive topics, as it only requires fine-tuning a few external parameters instead of the entire LLMs while achieving comparable or even better performance. To enable further research on PEFT methods of LLMs, this paper presents LLM-Adapters, an easy-to-use framework that integrates various adapters into LLMs and can execute these adapter-based PEFT methods of LLMs for different tasks. The framework includes state-of-the-art open-access LLMs such as LLaMA, BLOOM, and GPT-J, as well as widely used adapters such as Series adapters, Parallel adapter, Prompt-based learning and Reparametrization-based methods. Moreover, we conduct extensive empirical studies on the impact of adapter types, placement locations, and hyper-parameters to the best design for each adapter-based methods. We evaluate the effectiveness of the adapters on fourteen datasets from two different reasoning tasks, Arithmetic Reasoning and Commonsense Reasoning. The results demonstrate that using adapter-based PEFT in smaller-scale LLMs (7B) with few extra trainable parameters yields comparable, and in some cases superior, performance to powerful LLMs (175B) in zero-shot inference on simple math reasoning datasets.

## introduction large language models (llms), such as chat-gpt (openai, 2022) and gpt-4 (openai, 2023), * corresponding author. prior to the emergence of fft of llms (e.g., llama), a compelling solution called parameterefficient fine-tuning (peft) @xcite has been proposed in the nlp field, specifically for pre-trained models (e.g., bert @xcite ), offering a promising approach for efficiently fine-tuning llms. the advantage of peft lies in its ability to fine-tune only a small set of external parameters rather than the entire backbone model while still achieving comparable or even superior performance @xcite . moreover, peft can effectively mitigate catastrophic forgetting in comparison to fft @xcite . as shown in by incorporating these peft modules into backbone models (i.e., llms), we can capitalize on the remarkable capabilities of backbone models without requiring extensive computational resources. this opens up opportunities for a broader range of applications, enabling even those with limited access to high-performance computing to harness the power of llms in their specific tasks. despite the success of peft for pre-trained models, it remains unclear which peft module, in combination with which layer and hyperparameter configuration, is most suitable for a given task or dataset when meeting llms (e.g., llama @xcite ). therefore, further investigation is needed to determine the optimal peft setup that maximizes performance across different tasks and datasets. motivated by this, in this paper, we conduct a comprehensive empirical study of peft of three representative open-source llms, including bloom @xcite , gpt-j @xcite , and llama @xcite . specifically, we undertake an empirical study to address the following three research questions: (i) what is the optimal placement and configuration of different peft methods? (ii) how's the performance of different adapters across downstream tasks? and (iii) what are the differences in performance between in-distribution (id) and out-of-distribution (ood) scenarios for peft methods? the findings of our study are as follows: our contributions can be summarized as follows: ## peft overview in this section, we provide a brief overview of four parameter-efficient fine-tuning ( prompt. we take learnable vectors incorporated into the attention layer as an example of promptbased learning, which can be formulated as follows: reparametrization-based method. this type of methods aim to transform network weights using a low-rank technique. this approach effectively reduces the number of trainable parameters while preserving the ability to handle high-dimensional matrices. intrinsic said @xcite investigates the intrinsic dimensionality of finetuning within a low-rank subspace. lora @xcite introduces a simple approach to update the parameters of a weight matrix by decomposing it into a product of two low-rank matrices. krona @xcite improves upon the matrix factorization aspect of lora by utilizing the kronecker product in its technique. we take lora as an example of reparametrization-based learning, which can be formulated below: ## qualitative study the previous sections have presented the quantitative analysis. in this section, we will provide qualitative examples to demonstrate the quality of outputs from different models. ## conclusion in this paper, we develop a user-friendly framework, llm-adapter, seamlessly integrates diverse adapters into llms, empowering researchers to implement adapter-based peft methods for a wide range of tasks. to evaluate different peft methods on downstream tasks, we construct two highquality fine-tuning datasets to enhance peft performance on math reasoning and commonsense reasoning tasks. by utilizing the llm-adapter toolkit and the constructed fine-tuning datasets, we conduct a comprehensive empirical study and find the answer of research questions on the optimal placement and configuration of different peft methods, the impact of adapter architectures, and the influence of id and ood scenarios. we hope this work will encourage further research on peft methods for llms.