BIPEFT : Budget-Guided Iterative Search for Parameter Efficient Fine-Tuning of Large Pretrained Language Models (2024)

Abstract: Parameter Efficient Fine-Tuning (PEFT) offers an efficient solution for fine-tuning large pretrained language models for downstream tasks. However, most PEFT strategies are manually designed, often resulting in suboptimal performance. Recent automatic PEFT approaches aim to address this but face challenges such as search space entanglement, inefficiency, and lack of integration between parameter budgets and search processes. To overcome these issues, we introduce a novel Budget-guided Iterative search strategy for automatic PEFT (BIPEFT), significantly enhancing search efficiency. BIPEFT employs a new iterative search strategy to disentangle the binary module and rank dimension search spaces. Additionally, we design early selection strategies based on parameter budgets, accelerating the learning process by gradually removing unimportant modules and fixing rank dimensions. Extensive experiments on public benchmarks demonstrate the superior performance of BIPEFT in achieving efficient and effective PEFT for downstream tasks with a low parameter budget.

## introduction large pre-trained models (ptms) @xcite based on transformer architectures @xcite have achieved significant success across a variety of downstream tasks through fine-tuning, including applications of healthcare @xcite . however, the computational and storage demands of ptms limit the feasibility of full fine-tuning. to address this, parameter-efficient fine-tuning (peft) methods have garnered considerable attention. existing peft approaches have demonstrated superior performance on downstream tasks @xcite figure 1 : search space comparison among different automatic peft approaches when searching on low-rank adaption (lora) @xcite . s 3 delta @xcite uses a {0, 1} binary space to determine whether the module is kept. @xcite ) uses a multi-dimensional space to search module existence and dimension ranks simultaneously, where 0 means the module will be removed. the proposed bipeft uses a novel iterative search strategy to disentangle the binary module search and the rank dimension search. @xcite . however, most methods, such as adapters @xcite @xcite , bitfit @xcite , and lora @xcite @xcite @xcite require manual design of fine-tuning strategies. configuring peft for different layers of transformers can result in varying performance outcomes. to mitigate this issue, automatic peft approaches @xcite have been proposed to automatically search for the optimal peft configuration. s 3 delta @xcite is the first differential neural architecture search (nas)-based peft approach, which automatically searches for the optimal modules to include in the configuration. @xcite employs bayesian optimization to conduct the peft search across a large space. although these two automatic approaches are effective, they still suffer from the following issues: using a non-differential optimization strategy makes @xcite less efficient compared to s 3 delta @xcite . however, even with differential nas, the efficiency of s 3 delta is still unsatisfactory due to overlooking the unique characteristics of the peft task. unlike traditional automated machine learning tasks that typically learn model parameters and architectures from scratch, automatic peft only learns a small set of parameters, with most being fixed. the weight distributions for some modules or dimension ranks may quickly stabilize after a few training steps. thus, keeping them involved in the peft learning until the final training step is unnecessary and lowers search efficiency. to solve all the aforementioned issues simultaneously, in this paper, we propose a parameter budget-guided iterative search strategy bipeft for boosting the search efficiency of automatic peft, as shown in figure 2 . bipeft works as follows: at each training step t, bipeft uses the designed iterative search strategy to search optimal architecture weights for the binary peft module search space and rank dimension search space alternatively, which can handle the first issue. after a few training steps, bipeft will trigger the selection modules, where the trigger is generated based on the parameter budget b and the current module state, as detailed in §sec. 3.2. bipeft contains a novel parameter budget-guided module selection strategy in §sec. 3.3 to gradually remove the unimportant modules and a new parameter history-based dimension selection strategy in §sec. 3.4 to adaptively fix rank dimensions during the search. these two new selection strategies can perfectly address the second and third issues. after selecting modules and fixing rank dimensions, bipeft will repeat again until the number of triggers achieves the maximum number z. to sum up, this work has the following contributions: ## related work parameter efficient fine-tuning (peft). generally, peft is designed based on a transformer architecture and only optimizes a small portion of parameters and leaves the vast majority of parameters frozen for efficient adaptation to downstream tasks. the peft methods can be broadly classified into four categories @xcite : (1) additive peft such as adapter (houlsby et al., 2019) and prefix-tuning (li and liang, 2021) inserts new trainable modules or parameters in the model. (2) selective peft aims to optimize model performance by selectively fine-tuning a subset of q k v multi-head attention layer norm lnfit feed forward layer bitfit adapter lora x {0,1} {0,1} {0,1} {0,1} {1,4,8} {1,4,8} binary module search 𝜣 dimension rank search 𝜱 (a) fine-tuning a ptm layer using disentanglement search spaces binary module search 𝜣 dimension rank search 𝜱 training steps 𝑡 𝑡 -1 𝑡 + 1 trigger? trigger? trigger? budget-aware trigger generation module sensitivity module importance model stability binary module selection multiple rank dimension selection expected model parameters module sensitivity module selection dimension stability reduction size dimension selection (b) the proposed bipeft model with iterative search and selection strategies the model's parameters by masking @xcite @xcite or manual design @xcite . automated configuration search for peft. au-topeft @xcite using bayesian optimization and s 3 delta @xcite ) utilizing differential neural architecture search (nas) techniques to search for optimal architectures for natural language processing (nlp) tasks. in contrast, prunepeft @xcite adopts a straightforward pruning approach to identify essential peft parameters. in the vision domain, noah @xcite applies an evolutionary nas strategy to tailor peft configurations specifically for vision transformer @xcite . despite these advancements, significant challenges remain in optimizing search space design and improving search efficiency.