Parameter-Efficient Fine-Tuning: Is There An Optimal Subset of Parameters to Tune? (2024)

Abstract: The ever-growing size of pretrained language models (PLM) presents a significant challenge for efficiently fine-tuning and deploying these models for diverse sets of tasks within memory-constrained environments.In light of this, recent research has illuminated the possibility of selectively updating only a small subset of a model’s parameters during the fine-tuning process.Since no new parameters or modules are added, these methods retain the inference speed of the original model and come at no additional computational cost. However, an open question pertains to which subset of parameters should best be tuned to maximize task performance and generalizability. To investigate, this paper presents comprehensive experiments covering a large spectrum of subset selection strategies. We comparatively evaluate their impact on model performance as well as the resulting model’s capability to generalize to different tasks.Surprisingly, we find that the gains achieved in performance by elaborate selection strategies are, at best, marginal when compared to the outcomes obtained by tuning a random selection of parameter subsets. Our experiments also indicate that selection-based tuning impairs generalizability to new tasks.

## introduction in recent years, the number of parameters used in language models has risen much faster than the memory available in gpus @xcite . this creates high memory requirements for finetuning such models on available hardware. further, this creates high memory requirements when deploying a collection of such models to address various downstream tasks. a single pretrained model is often adapted to a wide range of tasks. the storage requirements for such a collection of model versions can be significantly reduced if the difference between these models can be represented in a compact way. weight bias parameter-efficient fine-tuning techniques (peft) aim to reduce the number of parameters that need to be stored and fine-tuned while maintaining a performance that is comparable to the training of the complete model. one popular class of these methods is referred to as selective parameter-efficient fine-tuning @xcite . here, a subset of the parameters is selected for peft, keeping the remaining parameters frozen during training. we illustrate this intuition in figure 1 for a single weight matrix and bias vector in which most parameters are frozen and only a small subset is updated during the optimization procedure. since only a few parameters are fine-tuned, the sparse difference between the adapted and the pretrained model can be stored in a compact way @xcite . the same applies to gradient statistics that are stored by the optimizer during fine-tuning. reducing the required memory frees up space for the use of larger batches and therefore speeds up training. however, an open question pertains to which subset of parameters should best be tuned to maximize task performance and generalizability. contributions. in this paper, we investigate several theoretical questions that have been raised in the context of selective peft methods and the lottery ticket hypothesis for pretrained (language) models @xcite . we aim to explore if an optimal subset for tuning exists and how subset tuning affects generalizability of the model. in more detail, we examine the following two aspects: to address these questions, we systematically conduct experiments using a large number of subset sizes and various subset selection strategies. we conduct a comprehensive grid search over hyperparameters to identify optimal training parameters for each selection strategy. we compare these hyperparameter-optimized subset selection strategies to full fine-tuning (including the use of regularization), as well as an additional (non-selective) parameter-efficient fine-tuning technique, which recently gained a lot of popularity: low-rank adaptation @xcite . we make several observations in our experiments: first, the differences between different subset selection methods are marginal when hyperparameters (the learning rate specifically) are properly optimized and do not significantly outperform a baseline using a randomly selected subset. second, subset-tuning methods tend to modify embedding networks significantly more since they are limited to a small number of parameters and hence need to make more drastic changes. the prior function of the network which can exhibit a certain degree of general language capabilities may be more affected by these local but more drastic changes. ## background our work is informed by two lines of research: selective parameter-efficient fine-tuning and the lottery ticket hypothesis for pretrained language models. in this section, we discuss aspects of these two areas that are relevant to the work we present in this paper. ## subset selection and downstream task performance in this first series of experiments, we aim to investigate the impact of the subset selection strategy and the subset size on the performance of the embedding network on a downstream task. each configuration is evaluated with respect to the performance on each of the four downstream tasks. we first describe the used selection strategies and the experimental setup, before discussing the observed impact of these two variables. ## generality & adaptability of the embedding network we extend our evaluation to investigate how the generality of the embedding network is impacted by the applied fine-tuning method. to this end, we leverage the transformer networks fine-tuned with different selection strategies on a primary task from the previous experiment and evaluate their usefulness for a distinct secondary task. in total, we report the following differences in performance: we therefore assess how the embedding network's function changes in terms of its capability to adapt to new tasks. ## conclusion in our evaluation of fine-tuning strategies, full fine-tuning consistently outperforms all parameterefficient fine-tuning (peft) methods across various tasks. lora consistently ranks among the top two peft methods in our experiments. examining the utilization of gradient statistics, we observe that the method using combined gra-dient statistics consistently outperforms its counterparts, although the performance improvement is marginal. on average, this approach surpasses all proper subset tuning methods that do not necessitate initial full fine-tuning. nevertheless, it is worth noting that the differences in performance across these experiments may not be substantial enough to justify the added complexity. surprisingly, even the random baseline, with a sufficiently high learning rate, demonstrates competitive performance, occasionally outperforming other peft methods. @xcite demonstrate the impact of the subset size on the question of whether a "winning" lottery ticket can be found (with or without optimizing parameters to retrieve it). our experiments extend this analysis into much smaller subset sizes. the results indicate that random subsets may not necessarily produce worse results than "winning" tickets (c.f. @xcite . instead, using a higher learning rate when tuning random subsets may shrink or diminish these performance differences. given the strong results of the random baseline and the generally similar performance on primary tasks, our results call into question whether there is a clear optimal subset of parameters to tune. furthermore, our generalization experiments indicate that selective peft strategies impair rather than increase generalizability to secondary tasks, likely due to peft affecting more localized and severe changes to the transformer network.