A uto PEFT : Automatic Configuration Search for Parameter-Efficient Fine-Tuning (2024)

Abstract: Large pretrained language models are widely used in downstream NLP tasks via task- specific fine-tuning, but such procedures can be costly. Recently, Parameter-Efficient Fine-Tuning (PEFT) methods have achieved strong task performance while updating much fewer parameters than full model fine-tuning (FFT). However, it is non-trivial to make informed design choices on the PEFT configurations, such as their architecture, the number of tunable parameters, and even the layers in which the PEFT modules are inserted. Consequently, it is highly likely that the current, manually designed configurations are suboptimal in terms of their performance-efficiency trade-off. Inspired by advances in neural architecture search, we propose AutoPEFT for automatic PEFT configuration selection: We first design an expressive configuration search space with multiple representative PEFT modules as building blocks. Using multi-objective Bayesian optimization in a low-cost setup, we then discover a Pareto-optimal set of configurations with strong performance-cost trade-offs across different numbers of parameters that are also highly transferable across different tasks. Empirically, on GLUE and SuperGLUE tasks, we show that AutoPEFT-discovered configurations significantly outperform existing PEFT methods and are on par or better than FFT without incurring substantial training efficiency costs.

## introduction and motivation pretrained language models (plms) are used in downstream tasks via the standard transfer learning paradigm, where they get fine-tuned for particular tasks @xcite . this achieves state-of-the-art results in a * equal contribution. parameter-efficient fine-tuning (peft) delivers a solution for alleviating the issues with full-model ft @xcite . by freezing the majority of pretrained weights of plms, peft approaches only update a small portion of parameters for efficiently adapting the plm to a new downstream task. recent studies have shown that peft can achieve competitive task performance while being modular, adaptable, and preventing catastrophic forgetting in comparison to traditional fft @xcite . recent developments have created diverse peft modules with distinctive characteristics @xcite , with one of the two main aims in focus: 1) improve task performance over other peft approaches while maintaining the same parameter budget as the competitor peft methods; or 2) maintain task performance while reducing the parameter budget needed. existing peft modules, optimizing for one of the two aims, have been successfully applied to transfer learning tasks @xcite . however, different tasks, with different complexity, show distinct sensitivity to the allocated parameter budget and even to the chosen peft approach @xcite . at the same time, most peft applications are limited to a single peft architecture (e.g., serial adapters, prefix-tuning) with fixed decisions on its components (e.g., hidden size dimensionality, insertion layers) resulting in potentially suboptimal peft configurations across many tasks. therefore, in this work, we propose a new, versatile, and unified framework that automatically searches for improved and task-adapted peft configurations, aiming to effectively balance between the two (often colliding goals) of (i) improving performance and (ii) keeping the desired low parameter budget for peft. while recent research has started exploring more dynamic peft configurations, prior studies remain limited across several dimensions, including how they define the configuration search space. namely, they typically focus only on a single peft architecture (e.g., adapters) or their simple combinations, or a single property (e.g., insertion layers-where to insert the module); see a short overview later in §3. here, we propose a unified and more comprehensive framework for improved configuration search. it covers multiple standard peft modules (serial adapters, parallel adapters, and prefix-tuning) as building blocks, combined with the critical parameter budget-related decisions: the size of each constituent module and the insertion layers for the modules. our defined comprehensive search space is huge; consequently, traversing it effectively and efficiently is extremely challenging. to enable search over the large configuration space, we thus propose the novel autopeft framework. it automatically configures multiple peft modules along with their efficiency-oriented design decisions, relying on a high-dimensional bayesian optimization (bo) approach. crucially, within the search space, we propose a multi-objective optimization which learns to balance simultaneously between maximizing the searched configurations' task performance and parameter efficiency. we conduct extensive experiments on the standard glue and superglue benchmarks @xcite , with encoder-only and encoder-decoder models. we first study the transferability of the autopeft-searched architecture by running autopeft on a single task with a low-fidelity proxy (aiming to reduce computational cost), followed by transferring the found architecture to other tasks. experimental results show that this architecture can outperform existing 1 ) compared to other baseline peft methods (markers) and full model ft that updates 100% of parameters (dashed horizontal bar), averaged across 8 glue tasks. our approach achieves the best trade-off between task performance and parameter efficiency. peft baselines while achieving on-par performance with the standard fft. further slight gains can be achieved with a larger computation budget for training, where we run autopeft per task to find a task-adapted peft configuration. as revealed in figure 1 , autopeft can find configurations that offer a solid trade-off between task performance and parameter efficiency, even outperforming fft. we also provide ablation studies over the search space, validating that the autopeft framework is versatile and portable to different search spaces. contributions. 1) we propose the autopeft search space containing diverse and expressive combinations of peft configurations from three representative peft modules as foundational building blocks and the binary decisions concerning transformer layers for inserting these modules as searchable dimensions. 2) to navigate the vast autopeft search space and to discover a set of transferable peft configurations that optimally trade performance against cost across various parameter ranges in a single run, we further propose an effective search method based on multi-dimensional bayesian optimization. 3) we demonstrate that the one-time search cost of autopeft is low, and autopeft yields task-shareable configurations, outperforming existing peft modules while being transferable across tasks. the autopeft framework can also be easily extended to other and new peft modules. the code is available at https://github .com/cambridgeltl/autopeft .