Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Models (2023)

Abstract: Parameter-efficient tuning (PET) methods fit pre-trained language models (PLMs) to downstream tasks by either computing a small compressed update for a subset of model parameters, or appending and fine-tuning a small number of new model parameters to the pre-trained network. Hand-designed PET architectures from the literature perform well in practice, but have the potential to be improved via automated neural architecture search (NAS). We propose an efficient NAS method for learning PET architectures via structured and unstructured pruning. We present experiments on GLUE demonstrating the effectiveness of our algorithm and discuss how PET architectural design choices affect performance in practice.

## introduction fine-tuning a large pre-trained language model is a popular method for solving many downstream natural language processing (nlp) tasks. full finetuning involves fine-tuning all parameters of the base plm, resulting in a fine-tuned copy of the model. however, full fine-tuning becomes cumbersome when fine-tuning on multiple downstream tasks due to the massive size of state-of-the-art language models, which range from the millions @xcite to billions @xcite ) and now trillions @xcite of parameters. full fine-tuning also carries a risk of catastrophic forgetting @xcite , wherein the plm's learned useful representation of natural language data is forgotten during fine-tuning. to address those problems, recent research has focused on parameter-efficient tuning (pet). rather than fine-tuning all parameters of the base plm, pet methods choose a small subset of parameters to fine-tune @xcite , or compute compressed parameter updates * work done while at amazon alexa ai @xcite , or append and fine-tune a small subset of new parameters @xcite @xcite . each of these methods has their own advantages and disadvantages, but one question relevant to all these methods is which parts of the network are most efficient to fine-tune, and what is the most parameterefficient way to fine-tune them? here we answer this question by designing and applying a fine-grain nas method for learning pet architectures. our method uses a first order approximation of the loss function and is computationally efficient. we compare our approach with several hand-designed pet methods and find that the architectures learned by our method generally achieve comparable or higher development set performance on glue tasks @xcite for the same number of parameters. we conclude by examining the pet architectures learned by our method and discuss the affect of architecture design choices on parameter efficiency. ## related work many different pet methods exist in the literature. adapter networks insert and fine-tune small adapter modules to a base plm. @xcite introduced adapter networks to the visual domain, and @xcite introduced adapters to transformers. adapters have been applied to text generation @xcite , translation @xcite , and multi-task learning @xcite . @xcite compare adaptation with full fine-tuning. adapterhub @xcite enables easy sharing of adapter models. additionally, @xcite propose best practices for producing strong full fine-tuning baselines. prompt-tuning methods fine-tune a plm by inserting prompt tokens into the input sequence. continuous prompts @xcite @xcite or discrete prompts @xcite ) can be learned or engineered @xcite . @xcite demonstrate the effectiveness of pretraining prompts for low resource tasks. some methods fine-tune a subset of parameters @xcite , or compute compressed parameter updates @xcite . these methods fine-tune the plm without increasing test-time inference latency. @xcite and @xcite combine multiple pet methods. beyond parameter-efficient tuning, nas has previously been used to discover more parameterefficient base language models. @xcite use magnitude pruning to reduce the number of parameters in bert. many efforts at pruning bert have focused on pruning attention heads from the multi-head attention (mha) modules @xcite @xcite . @xcite evaluate different adhoc strategies for shrinking the depth of a bert encoder. @xcite use an evolutionary nas method to learn an improved transformer cell. in contrast to nas, distillation can be used to compress language models @xcite @xcite . in our experiments section, we examine the architectures learned by our algorithm and consider what they say about which parts of the network are most parameter-efficient to fine-tune. merchant et al. (2020) explore a similar question, probing the network activations to understand how the network's representation of natural language data changes during full fine-tuning. ## method the architecture search space we choose for our nas method is based on bitfit @xcite and @xcite , two of the most popular methods for parameter-efficient fine-tuning in the literature. we consider both structured and unstructured variants of each of these, where the non-zero pattern of the learned pet parameters is restricted or unrestricted, respectively. specifically, our search space consists of the following: in structured lora, for each parameter matrix w , the nas algorithm must decide what the rank of the update u v ⊤ should be. in unstructured lora, the nas algorithm must decide which components of u and v should be non-zero. the collection of updates ∆b and ∆w are the pet parameters. in this search space, any number of the above pet modules can be applied to a base plm without increasing the latency of inference, just like bitfit @xcite and @xcite . ## experiments details of our experimental setup, including hyperparameter choices, are available in the appendix. in all experiments we report median validation score at the end of training over 5 random initializations using the glue development set for validation. ## conclusion in this paper, we considered the question which parts of the network are most efficient to fine-tune, and what is the most parameter-efficient way to fine-tune them? to answer that question, we developed a nas algorithm based on structured and unstructured pruning. we presented experimental results on roberta large demonstrating the effectiveness of our algorithm, achieving glue validation performance similar to warp at 25k parameters (9% of all biases), similar to bitfit at 50k parameters (18% of all biases), and similar to full fine-tuning at 3.4m parameters (10% of all parameters). from our learned architectures we observed that the bias parameters in the middle layers of the network are most efficient to fine-tune. we conclude that it is important to consider where to fine-tune as well as how.