Efficient Active Learning with Adapters (2024)

Abstract: One of the main obstacles for deploying Active Learning (AL) in practical NLP tasks is high computational cost of modern deep learning models. This issue can be partially mitigated by applying lightweight models as an acquisition model, but it can lead to the acquisition-successor mismatch (ASM) problem. Previous works show that the ASM problem can be partially alleviated by using distilled versions of a successor models as acquisition ones. However, distilled versions of pretrained models are not always available. Also, the exact pipeline of model distillation that does not lead to the ASM problem is not clear. To address these issues, we propose to use adapters as an alternative to full fine-tuning for acquisition model training. Since adapters are lightweight, this approach reduces the training cost of the model. We provide empirical evidence that it does not cause the ASM problem and can help to deploy active learning in practical NLP tasks.

## introduction recent progress in the natural language processing (nlp) tasks has become possible due to an abundant range of pre-trained language models. data annotation is a rather important process, since the performance of model depends greatly on the quality of data it was trained on. active learning (al), which is a technique used to annotate data and train models efficiently, has been first introduced in @xcite . this technique has been widely used to train language models to solve such nlp tasks as text classification @xcite , named entity recognition @xcite and sequence labeling tasks @xcite . active learning helps to reduce annotation costs by employing a specifically designed query strategy which works on sampling the data points that would bring the most substantial information gains for model training. one problem that has been described by @xcite is acquisitionsuccessor mismatch (asm). this refers to employing models of different architectures for acquisition (evaluating which samples would be the most beneficial) and successor (retraining with newly acquired samples) negatively impacts the performance. for some popular models, such as bert, distilled versions can be used as acquisitions to save time and computational resources. we suggest using parameter-efficient fine-tuning methods for those models that do not have a distilled version. the findings of this study indicate that utilizing an adapter model with a successor of identical architecture consistently yields superior outcomes compared to a distilled model with a different architecture. our main contributions are the following: ## related work in @xcite it was proposed to accelerate training and data selection steps for al by leveraging distilled versions of the successor model during al iterations. a similar approach was introduced in @xcite , where it was proposed to use on-the-fly knowledge distillation of the successor model to the acquisition model. however, model distillation is expensive in terms of both time and computational resources. furthermore, this approach cannot always be directly used in practice due to the lack of distilled models for several architectures. in @xcite , it was proposed to use pseudo labeling-based approach to mitigate the asm problem. however, this approach can also suffer from the lack of distilled/teacher model pairs, especially for some specific domains. furthermore, jukić and šnajder (2023) explores the application of adapters in active learning in low-resource settings. the research concludes that some adapter configurations provide performance gains over full fine-tuning. the authors also investigate learning stability and compare layerwise representations obtained from adapters and fully fine-tuned models. they find that adapter models are more similar to the base model in earlier layers which are considered to contain foundational knowledge. in our work, we provide another kind of analysis: we compare uncertainty scores of different kinds of models and conclude that adapters can be applied in many areas of active learning, since the they do not affect the uncertainty scores. the research jukić and šnajder (2023) also provides comparison of performance scores of full and adapter models. however, there is no mention of time taken to train the models. we close this gap by measuring the speed of full and adapter models. finally, in @xcite , adapters were used to improve time efficiency of the successor model, but their impact on the acquisition model was not analysed. we explore how adapters affect the time of the whole al loop. in our research, we bring empirical contribution by testing and analyzing the adapter application in active learning. we provide analysis of the uncertainty scores of adapter models to demonstrate the potential applicability of adapters in any domain of al. time efficiency of adapters in the al loop is explored as well. it is also shown that adapters can help solve the asm problem in active learning. ## analysis figure 1 represents an accuracy curve of the full and adapter electra models, as well as dis-tilbert trained on the amazon massive dataset. additionally, figures 2 , 3 and 4 can be found in the appendix a.7. these figures represent accuracy curves of three combinations of models and strategies, trained on classification and ner data. across all figures, each curve represents metrics averaged out over five seeds. for both classification (figures 1 , 2 , 3 ) and ner (figure 4 ) tasks, it is obvious that electra with an adapter performs better than distilbert, which means that the acquisition-successor mismatch problem can be solved with adapters. in addition, adapters save training time while preserving the performance scores at the same level as full models. in order to measure the speedup that adapter modules provide in the active learning loop, we train full electra and adapter electra on the four datasets. we measure the time it takes to train on 2, 6 and 12% of the data and report it in the ## results as it is observed from our experiments on four datasets show that models with the bottleneck adapter demonstrate a comparable performance on each active learning iteration with full models. we have also included experiments with distilbert as an acquisition model and this setup performs worse in comparison with all other setups due to the asm problem discussed earlier. in addition, we have concluded that the adapter helps speed up the active learning process when added to the acquisition model. all this makes the adapter models more efficient for classification in active learning. ## conclusion the findings of this study include the following: