Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models (2024)

Abstract: Parameter-Efficient Fine-Tuning (PEFT) methods have gained significant popularity for adapting pre-trained Large Language Models (LLMs) to downstream tasks, primarily due to their potential to significantly reduce memory and computational overheads. However, a common limitation in most PEFT approaches is their application of a uniform architectural design across all layers. This uniformity involves identical trainable modules and ignores the varying importance of each layer, leading to sub-optimal fine-tuning results. To overcome the above limitation and obtain better performance, we develop a novel approach, Importance-aware Sparse Tuning (IST), to fully utilize the inherent sparsity and select the most important subset of full layers with effective layer-wise importance scoring. The proposed IST is a versatile and plug-and-play technique compatible with various PEFT methods that operate on a per-layer basis. By leveraging the estimated importance scores, IST dynamically updates these selected layers in PEFT modules, leading to reduced memory demands. We further provide theoretical proof of convergence and empirical evidence of superior performance to demonstrate the advantages of IST over uniform updating strategies. Extensive experiments on a range of LLMs, PEFTs, and downstream tasks substantiate the effectiveness of our proposed method, showcasing IST’s capacity to enhance existing layer-based PEFT methods. Our code is available at https://github.com/Kaiseem/IST

## introduction significant achievements in natural language processing (nlp) have been achieved this year from the use of large language models (llms) that are pre-trained on extensive general datasets (zhuang et @xcite . these llms typically require full fine-tuning (fft) @xcite to adapt them for specialized downstream tasks, an approach that necessitates retraining all model parameters. nevertheless, as the size of these models and the volume of data increase, fft becomes increasingly costly and impractical. aiming to reduce the cost, parameter-efficient fine-tuning (peft) methods, involving adapterbased @xcite @xcite , reparameterizationbased @xcite @xcite , and prompt-based methods @xcite @xcite , have been proposed to reduce the number of trainable parameters in fine-tuning for the downstream tasks. however, most existing peft methods employ a uniform approach that indiscriminately assigns trainable parameters to identical positions across all layers, which could be unnecessary. this strategy relies heavily on human heuristics and overlooks task-specific domain gaps and characteristics, limiting their performance across various downstream tasks. although some peft methods have improved the efficiency of fine-tuning llms, such as dynamic rank @xcite , they are tailored specifically for lora-based mod-els and do not extend their benefits to the additional learnable module-based methods, i.e., series and parallel configurations. this limitation creates a clear necessity for a more generalized algorithm to enhance model performance across various domains. inspired by lisa @xcite , we empirically found that training a small fraction of the full layers in peft can yield comparably promising results to those achieved with fft. the existing peft methods exhibit markedly redundancy in layer updating during the training process, leading us to investigate the differences among the layers of varying importance from the perspective of layerwise sparsity. motivated by these inherent insights, we propose a novel peft-compatible plug-andplay approach, importance-aware sparse tuning (ist), that estimates the task-specific importance score of each layer and fine-tunes the most important ones. as shown in figure 1 , our method substantially lowers memory demands during training by reducing the number of layers that require updates. furthermore, by integrating layer-wise sparsity into our methodology, we enhance the convergence of layer-based peft methods, thereby achieving improved performance. the experimental results show that ist consistently improves existing layer-wise peft methods without sacrificing performance and inference efficiency across a wide range of models. in summary, our contributions are as follows: ## experimental results in this section, we conduct a series of experiments to validate the effectiveness of our proposed ist. we integrate ist into the series adapter, parallel adapter, and lora, and then compare them with their original counterparts across various tasks. baselines we included the following widely used layer-based fine-tuning methods. for the optimal configuration and placement of peft methods, we adhere to the settings established by @xcite . specifically, series and parallel adapters are seamlessly integrated into the mlp layers with a bottleneck size of 256. similarly, lora is seamlessly incorporated into both the multi-head self-attention layers and the mlp layers, with a rank of 32. across all peft methods, we maintain the same tunable parameter budgets, adjusting only the learning rate. for ist, we consistently set n u to 25% of the layers for the fine-tuning loop, n v to 50% of the layers for the importance updating loop, and β to 0.25. further details on the experimental settings are available in the appendix. ## conclusion in this study, we proposed a novel importanceaware sparse tuning (ist) approach for peft of llms. by dynamically selecting the most important layers in the fine-tuning loop, ist achieves a significant reduction in memory usage and computational overhead. the importance updating loop refines the selection of layers using a reinforcement learning approach, ensuring that the most impactful layers are prioritized during training. this innovative method leverages the inherent sparsity of layerwise importance, leading to more efficient and effective fine-tuning through extensive experiments across various llms, peft methods, and downstream tasks. the proposed method holds promise for future applications where resource constraints and performance are critical considerations. (peft) methods can be roughly categorised into four types as shown in parallel adapters. parallel adapters focus on incorporating additional learnable modules in parallel with distinct sublayers within the backbone model. the parallel adapter can be formulated as follows: reparametrization-based method. this type of method aims to transform network weights using a low-rank technique. we take lora @xcite as an example of reparametrization-based learning, which can be formulated below: series adapters. series adapters involve incorporating additional learnable modules in a sequential manner within a specific sublayer. series adapter can be formulated as follows: