On Dataset Transferability in Active Learning for Transformers (2023)

Abstract: Active learning (AL) aims to reduce labeling costs by querying the examples most beneficial for model learning. While the effectiveness of AL for fine-tuning transformer-based pre-trained language models (PLMs) has been demonstrated, it is less clear to what extent the AL gains obtained with one model transfer to others. We consider the problem of transferability of actively acquired datasets in text classification and investigate whether AL gains persist when a dataset built using AL coupled with a specific PLM is used to train a different PLM. We link the AL dataset transferability to the similarity of instances queried by the different PLMs and show that AL methods with similar acquisition sequences produce highly transferable datasets regardless of the models used. Additionally, we show that the similarity of acquisition sequences is influenced more by the choice of the AL method than the choice of the model.

## introduction pre-trained language models (plms) -large overparameterized models based on the transformer architecture @xcite and trained on large corpora -are the leading paradigm in modern nlp, yielding state-of-the-art results on a wide range of nlp tasks. however, large models require large amounts of data. active learning (al; @xcite addresses the data bottleneck problem by improving data labeling efficiency. it employs human-in-the-loop labeling with the model iteratively selecting data points most informative for labeling. recent work has demonstrated the effectiveness of al for fine-tuning plms @xcite @xcite @xcite . while al may considerably reduce model development costs, it also potentially limits the scope of use of the actively acquired datasets. since data sampling in al is guided by the inductive bias of the acquisition model, the dataset will typically not represent the original population's distribution @xcite . this is troublesome if one wishes to use the actively acquired dataset to train a different model (consumer model) from the one used for al (acquisition model). if the two models' inductive biases differ, the al gains can cancel or even revert: the consumer model may perform worse when trained on the actively acquired dataset than on a randomly sampled one. however, the robustness of the actively acquired dataset to the choice of the consumer model is obviously highly desirable, as the acquisition model may become unavailable or dated. the latter is common in nlp, where new and better models are being developed faster than new datasets. however, most al studies use the same acquisition and consumer models, and dataset transferability is seldom mentioned in al literature. a notable exception is the work of @xcite , who showed the unreliability of dataset transfer on standard nlp tasks. in this work, we examine the problem of al dataset transferability for transformer-based plms and conduct a preliminary empirical study on text classification datasets. we first probe whether al gains persist between different transformerbased plms, considering several al methods and datasets. observing that on most datasets, the transfer works in some cases but fails in others, we investigate the mechanisms underlying transferability. we hypothesize a link between al dataset transferability and how the acquisition and consumer models sample instances. to probe this, we introduce acquisition sequence mismatch (asm) to characterize to what extent the two models differ in how they sample instances throughout al iterations. we investigate how asm affects dataset transferability and how asm is affected by other al variables. we show that, while it is generally reasonable to transfer actively acquired datasets between transformer-based plms, al methods that retain low asm produce more transferable datasets. we also show that the choice of the al method affects asm more than the choice of models. to summarize our contributions: we (1) conduct an empirical study on the transferability of actively acquired datasets between transformer-based plms, (2) propose a measure to quantify the mismatch in the acquisition sequences of al models and link this to dataset transferability, and (3) analyze what design choices affect this mismatch. we provide code for the experiments foot_0 with the hope that our results will encourage nlp practitioners to use al when fine-tuning plms and motivate further research into the al dataset's transferability. ## related work although al has been extensively studied for shallow and standard neural models (without pretraining), research on combining al and plms lags behind. the initial studies showed promise, with al methods outperforming random sampling for text classification @xcite . the field is gradually gaining traction with studies demonstrating al effectiveness even with simple uncertainty-based methods @xcite . moreover, plms open up new possibilities, such as complementing al with model adaptation using unlabeled data @xcite . while there is much research on al for standard scenarios where the acquisition and consumer models are the same, there is little research on al dataset transfer. @xcite demonstrated that combining uncertainty al strategies with deep models produces sampled datasets with good sampling properties that have a large overlap with support vectors of svm trained on the entire dataset. likewise, @xcite showed that deep neural models benefit from the sample bias induced by the acquisition model (the opposite is true for shallow models). however, the jury is still out on the effects of sample bias on the consumer model. the most prominent empirical study on al transfer with neural models @xcite predates plms. @xcite focused on alleviating the effects of acquisitionconsumer mismatch in plms by using lightweight distilled models for acquisition and larger versions of the models as consumer models. even though the study focuses on improving the transferability of actively acquired datasets, the reasons behind the successful transfer are yet to be explored. an older study of al dataset transferability for text classification and shallow models by @xcite showed that transfer works in most cases but that neither sample nor model similarity explains transferability. our study explores these characteristics for acquisition-consumer pairings of different plms. ## experimental setup our study used four datasets, three models, and three al methods (cf. appendix b for details). the datasets we used are subjectivity (subj; @xcite , cola (cola; @xcite , ag-news (agn; @xcite , and trec (trec; @xcite ). the three transformer models we used are bert @xcite , roberta @xcite , and electra @xcite . the al methods we considered are entropy (ent; @xcite , core-set (cs; @xcite , and badge (ba; ash et al., 2019)). this gives 108 al configurations (72 transfer and 36 no-transfer configurations). furthermore, we ran each configuration with 20 different warm-start sets to account for stochasticity. the al acquisition was simulated until the budget of 1500 labeled data points was exhausted (model performance for all datasets reached a plateau), labeling 50 data points per step. we assessed dataset transferability using the difference in the area under the f 1 curve of the model trained on the actively acquired dataset and the same model trained on a randomly sampled dataset (∆auc). we deem the al dataset transfer successful if ∆auc is not significantly less than zero and unsuccessful otherwise. we chose ∆auc to make the notion of transferability independent of when the al acquisition terminates. on the other hand, as terminating the al after acquiring too few labeled data is unrealistic, we also report ∆auc 10 , which is ∆auc calculated with an offset of 10 iterations (500 labeled instances) of the al loop. comparing ∆auc 10 to ∆auc provides insights into how transferability changes through time. ## conclusion we presented an empirical study on the transferability of actively acquired text classification datasets for transformer-based plms. our results indicate no significant risk in transferring datasets, especially for larger amounts of data. we also showed that transfer is largely successful when preserving the sequence and similarity of acquired instances between the models, which is what methods combining uncertainty and diversity acquisition functions seem to do. transferability appears to differ considerably across datasets, so future work should examine what dataset characteristics are predictive of transfer success.