Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts (2024)

Abstract: Pretrained Language Models (PLMs) have advanced Natural Language Processing (NLP) tasks significantly, but finetuning PLMs on low-resource datasets poses significant challenges such as instability and overfitting. Previous methods tackle these issues by finetuning a strategically chosen subnetwork on a downstream task, while keeping the remaining weights fixed to the pretrained weights. However, they rely on a suboptimal criteria for sub-network selection, leading to suboptimal solutions. To address these limitations, we propose a regularization method based on attention-guided weight mixup for finetuning PLMs. Our approach represents each network weight as a mixup of task-specific weight and pretrained weight, controlled by a learnable attention parameter, providing finer control over sub-network selection. Furthermore, we employ a bi-level optimization (BLO) based framework on two separate splits of the training dataset, improving generalization and combating overfitting. We validate the efficacy of our proposed method through extensive experiments, demonstrating its superiority over previous methods, particularly in the context of finetuning PLMs on low-resource datasets. Our code is available at https://github.com/Sai-Ashish/Attention_guided_weight_mixup_BLO.

## introduction pretraining large language models on a large corpus of unlabeled texts and further finetuning them on downstream tasks have been a common practice in natural language processing (nlp), resulting in significant advances in tasks such as sentiment classification, natural language inference, and text generation @xcite @xcite . however, conventional finetuning of pretrained language models (plms) presents several challenges. first, plms are prone to instability in finetuning, characterized by high variance in performance for different weight initializations even when using the same hyperparameters, especially on small datasets @xcite @xcite @xcite . moreover, plms, due to their extremely large capacity, are prone to overfitting when finetuned on small downstream datasets, leading to poor generalization on test set @xcite @xcite . consequently, adapting plms to a variety of low-resource tasks, while preserving stability and maximizing generalization, remains a significant challenge in the field. finetuning a strategically chosen sub-network on a downstream task, while keeping the remaining weights fixed to the pretrained weights, has effectively mitigated these challenges. within this umbrella, child-tuning d @xcite and dps dense @xcite are promising. in child-tuning d @xcite , a static subnetwork, termed "child network", is first selected based on the fisher information matrix (fim) and this child network is subsequently updated during finetuning. dynamic parameter selection (dps) @xcite further refines this approach by dynamically selecting the child network during plm finetuning using fim as a guiding principle. nevertheless, these prior works exhibit certain limitations. fim, which is empirically calculated using the training dataset to identify important network parameters, may not be optimal for child network selection, especially in low-resource settings where data scarcity can skew the gradient that is used to compute fim @xcite . this scenario can lead to unimportant parameters being subsequently chosen in the sub-network, deteriorating performance on downstream tasks. moreover, @xcite theoretically shows that empirically determined fim deviates significantly from the true fim when the number of samples is low. thus, the discrete selection of child networks based on heuristics (i.e., fim), may result in selection of suboptimal child networks for downstream tasks. these limitations necessitate a departure from fim-based discrete child network selection strategies in favor of one that selects a child network based on the model's downstream task performance. consequently, we advocate for a continuous optimization approach for child network selection that does not rely on fim, guided by the goal of optimizing performance in downstream tasks. in this work, we propose an end-to-end framework that converts prior heuristic-based approaches to discrete child network selection into a continuous relaxation that can be optimized using gradient descent. the crux of our method is an attentionguided weight mixup mechanism that facilitates this transformation. each weight is represented as a weighted sum of task weights (from downstream task finetuning) and pretrained weights (which is frozen), controlled by a mixing coefficient, which we refer to as "attention parameter" foot_0 . this setup leads to a continuous relaxation: a larger attention value of a weight indicates that it is more likely to belong to the child network and vice versa. we formulate the learning of task weights and the attention parameters in a bi-level optimization (blo) framework @xcite on two different splits of the training dataset. in the lower level of our formulation, we update task weights by minimizing loss on the first split, while in the upper level of the framework, we update the attention parameter by minimizing loss on the second split. this bi-level learning of task weights and the attention parameters on two different splits of the training dataset sidesteps the fim-based heuristic, ensuring it is more adapted to the downstream task, leading to better performance (sec. 4). our major contributions are summarized below: in this approach, each weight is represented as a mixup of task weights and pretrained weights, controlled by an attention parameter. this method is a continuous relaxation of the prior discrete sub-network selection approach. ## method we introduce "attention-guided weights mixup", depicted in fig 1 , an approach to improving stability and performance on small downstream datasets in plms. central to our approach is a unique representation of weights as a blend of the pretrained weights (those prior to finetuning) and the task weights (post-downstream task finetuning), modulated by an "attention parameter". this coefficient represents the degree of emphasis or "attention" to be placed on a pretrained weight during the computation of the resultant weight. to mitigate overfitting, we propose to learn the attention parameter and the task weights using blo on different splits of the training dataset; pretrained weights are frozen. this initial phase, termed search phase, aims to find optimal attention parameters, i.e., to find the optimal child network. subsequently, in the finetune phase, the task weights are further adjusted using the entire training dataset and the learned attention parameters. ## computational costs and trade-offs our method introduces attention-guided weights mixup through a blo framework, leading to additional computational demands summarized in we compare the training costs of peft methods with our approach. prompt tuning necessitates approximately 0.2 times the training cost of vanilla fine-tuning, utilizing only 0.0067% of the total parameters as trainable. in contrast, prefix-tuning and lora require about 0.4× and 0.3× the training cost of vanilla fine-tuning, respectively, with prefix-tuning utilizing 0.294% and lora 0.236% method vanilla r3f r-dropout child-tuningd dps ours time usage ×1 ×1.64 ×1.64 ×3.13 ×1.12 ×(1 + 0.8k) ## conclusions and future works in this work, we propose an attention-guided weight mixup mechanism to address issues in finetuning plms on low-resource datasets. specifically, we represent each weight as a linear interpolation of the task weights and the pretrained weights, controlled by an attention parameter. capitalizing on the inherent structure of this representation, we utilize a blo framework to learn task weights and attention parameters on two different splits of training dataset. our method demonstrated its effectiveness across several challenging datasets from the glue benchmark, outperforming baselines in low-resource scenarios. it also shows improved stability across different plm architectures. moreover, through ablation studies, we highlighted the importance of learning attention parameters continuously and the benefits of our blo framework over the straightforward joint-training of model weights and attention parameters. we believe our method has the potential to be applied to other important areas, such as lifelong learning @xcite . a common challenge in lifelong learning is retaining knowledge from previous tasks while adapting to new ones. exploring our attention-guided weights mixup, optimized using blo, for this application presents a promising avenue for future work.