Parameter-Efficient Language Model Tuning with Active Learning in Low-Resource Settings (2023)

Abstract: Pre-trained language models (PLMs) have ignited a surge in demand for effective fine-tuning techniques, particularly in low-resource domains and languages. Active learning (AL), a set of algorithms designed to decrease labeling costs by minimizing label complexity, has shown promise in confronting the labeling bottleneck. In parallel, adapter modules designed for parameter-efficient fine-tuning (PEFT) have demonstrated notable potential in low-resource settings. However, the interplay between AL and adapter-based PEFT remains unexplored. We present an empirical study of PEFT behavior with AL in low-resource settings for text classification tasks. Our findings affirm the superiority of PEFT over full-fine tuning (FFT) in low-resource settings and demonstrate that this advantage persists in AL setups. We further examine the properties of PEFT and FFT through the lens of forgetting dynamics and instance-level representations, where we find that PEFT yields more stable representations of early and middle layers compared to FFT. Our research underscores the synergistic potential of AL and PEFT in low-resource settings, paving the way for advancements in efficient and effective fine-tuning.

## introduction pre-trained language models (plms) have quickly become a staple in the field of natural language processing. with the growing demand for data for training these models, developing efficient finetuning methods has become critical. this is particularly relevant for many domains and languages where obtaining large amounts of labeled training data is difficult or downright impossible. in such low-resource settings, it becomes essential to effectively leverage and adapt plms while minimizing the need for extensive labeled data. data labeling is notoriously time-consuming and expensive, often hindering the development of sizable labeled datasets required for training highperformance models. active learning (al) @xcite has emerged as a potential solution to this challenge. in contrast to passive learning, in which the training set is sampled at random, al encompasses a unique family of machine learning algorithms specifically designed to reduce labeling costs by reducing label complexity, i.e., the number of labels required by an acquisition model to achieve a certain level of performance @xcite . with the advent of plms, al research has pivoted towards investigating training regimes for plms, such as task-adaptive pre-training (tapt; @xcite , that could be combined with al to further reduce the label complexity. while al aims at directly minimizing the label complexity of learning, training efficiency can also be improved by reducing the parameter complexity of the model. this becomes more important as plms grow larger, and fine-tuning becomes increasingly challenging due to the sheer number of parameters involved. to address this issue, adapters @xcite have been introduced as compact modules that can be incorporated between the layers of plms. adapters enable considerable parameter-sharing, facilitating parameterefficient fine-tuning (peft) through modular learning @xcite . in this process, only the parameters of the adapters are updated during the tuning for a specific downstream task. recent research @xcite @xcite has revealed that some peft methods outperform full fine-tuning (fft) in low-resource settings, potentially due to better stability and a decreased risk of overfitting. in contrast, fft has been shown to exhibit instability in scenarios with limited data. despite the promising results demonstrated by peft methods in low-resource settings, there is a striking gap in research on parameter-efficient training with respect to how peft interacts with al. given that the majority of real-world al scenarios involve a restricted amount of data, peft methods emerge as strong candidates for al acquisition models. however, there has been no exploration of al in conjunction with adapters. investigating this uncharted territory can further advance our understanding of al and reveal novel strategies for optimizing performance in low-resource settings. in this paper, we present an empirical study on the behavior of peft in low-resource settings for text classification tasks. we analyze peft with and without al and compare it against fft. while our results confirm that peft exhibits superior performance in low-resource setups compared to fft, we show that the improved performance with peft extends to al scenarios in terms of performance gains over passive learning. furthermore, we analyze the efficacy of tapt in conjunction with al and peft. we find that tapt is beneficial in al scenarios for both peft and fully fine-tuned models, thus representing a viable technique for improving performance in low-resource settings. finally, aiming to illuminate why peft and tapt improve al performance in low-resource settings, we analyze the properties of peft and fft via forgetting dynamics @xcite and plms' instance-level representations. we find that al methods choose fewer unforgettable and more moderately forgettable examples when combined with peft and tapt, where forgetfulness indicates the model's tendency to learn and forget the gold label of a particular instance. compared to fft, we observe that peft yields representations in the early and middle layers of a model that are more similar to the representations of the base plm. we hypothesize that this property mitigates the issue of forgetting the knowledge obtained during pretraining when fine-tuning for downstream tasks. in summary, we show that in al low-resource settings for text classification, (1) peft yields greater performance improvements compared to fft and (2) tapt enhances the overall classification performance of adapters and is well-suited for al scenarios. we also show that (3) al methods choose fewer unforgettable and more moderately forgettable examples with peft and that (4) peft produces instance-level representations of early and middle layers that are more similar to the base plm than fft. our results uncover the intrica-cies of positive interactions between al, peft, and tapt, providing empirical justification for their combined use in low-resource settings. ## related work our research involves combining al with plms and investigating the use of peft techniques within the confines of low-resource settings. al with plms. until recently, the conventional approach for integrating plms with al involved performing full fine-tuning with a fixed number of training epochs and training the model from scratch in each al step @xcite @xcite @xcite . however, studies by @xcite and @xcite revealed that fine-tuning in low-resource setups is prone to instability, particularly when training for only a few epochs. this instability, often sensitive to weight initialization and data ordering @xcite , presents a significant challenge for al, which frequently operates in lowresource settings. recent research has looked into the impact of plm training regimes on al performance @xcite @xcite , suggesting that the choice of training regime is more critical than the choice of the al method. notably, tapt has proven particularly effective in enhancing al performance @xcite . ## preliminaries we now describe the experimental setup, providing details on the datasets as well as the peft and al methods used in our study. ## experiments in this section, we first examine the performance of peft methods in comparison to fft with passive learning and then proceed to analyze the application of peft in al settings. ## analysis in section 4, we have demonstrated that peft exhibits larger gains than fft when combined with al in low-resource settings, which is also accompanied by superior performance with passive leaning. to better understand why peft displays superior behavior with limited data, we now examine two specific properties of adapters and fft models. first, we analyze the influence of tapt on the forgetting dynamics during training. we continue with example-level representation analysis, where we investigate the representation similarity of peft and fft to their respective base models. ## conclusion our study has shed light on the advantages of parameter-efficient fine-tuning (peft) in lowresource settings, confirming its superiority over full fine-tuning (fft) methods. importantly, we have demonstrated that the integration of peft with active learning (al) can offer substantial performance gains compared to passive learning, even in settings where labeled data is scarce. furthermore, we highlighted the potential of task-adaptive pre-training (tapt) to improve model performance further when used in conjunction with both peft and al. we found that al methods, in combination with peft, tend to select fewer unforgettable instances and more moderately forgettable examples. we further found that peft maintains the integrity of early and middle layer representations similar to the base model. we conjecture that this property mitigates forgetting during downstream task fine-tuning. these insights inform us of a possible underpinning mechanism that contributes to peft's superior performance and stability in low-resource settings. overall, our work highlights the potential of peft and al and establishes a foundation for developing increasingly efficient and cost-effective approaches for training models in low-resource settings.