Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models (2024)

Abstract: Active learning is an iterative labeling process that is used to obtain a small labeled subset, despite the absence of labeled data, thereby enabling to train a model for supervised tasks such as text classification.While active learning has made considerable progress in recent years due to improvements provided by pre-trained language models, there is untapped potential in the often neglected unlabeled portion of the data, although it is available in considerably larger quantities than the usually small set of labeled data. In this work, we investigate how self-training, a semi-supervised approach that uses a model to obtain pseudo-labels for unlabeled data, can be used to improve the efficiency of active learning for text classification. Building on a comprehensive reproduction of four previous self-training approaches, some of which are evaluated for the first time in the context of active learning or natural language processing, we introduce HAST, a new and effective self-training strategy, which is evaluated on four text classification benchmarks. Our results show that it outperforms the reproduced self-training approaches and reaches classification results comparable to previous experiments for three out of four datasets, using as little as 25% of the data. The code is publicly available at https://github.com/chschroeder/self-training-for-sample-efficient-active-learning.

## introduction in supervised machine learning, a lack of labeled data is the main obstacle to real-world applications, since labeled data is usually non-existent, expensive to obtain, and sometimes even requires domain experts for annotations. one solution to create models despite the absence of labels, is active learning, where in an iterative process an oracle (usually realized through a human annotator) provides labels for unlabeled instances that have been deemed to be informative by a so-called query strategy. these labels are then used to train a model, which in turn is used by the query strategy during the next iteration. in this work, we investigate the combination of self-training and active learning to reduce the required amount of labeled data even further. during recent years, transformer-based pretrained language models @xcite have successfully been applied for active-learning-based text classification, thereby considerably raising the state-of-the-art results (e.g., @xcite . the dominant paradigm here is pool-based active learning @xcite where the query strategy repeatedly selects batches of instances to be labeled next from the pool, the entirety of unlabeled data. while language models have successfully been adopted for active learning (e.g., by @xcite , @xcite , and @xcite ), the total labeling effort, i.e. the number of queries and the number of instances per query, has remained similar to setups predating transformers. with regard to the size of queries, there are two prevailing setups: (1) absolute query sizes @xcite @xcite @xcite @xcite , where a fixed number of instances are queried during each iteration, and (2) relative query sizes @xcite @xcite , in which the number of queried instances is a percentage of the unlabeled pool. we argue that those query sizes of both aforementioned experiment setups are needlessly large. previous works query up to 1000 instances @xcite or up to 25% percent of the unlabeled pool @xcite , where the former is of considerable size and the latter is clearly infeasible in practice as soon as datasets reach average contemporary sizes or annotation costs are high. when using language models that have been trained on billions @xcite or even trillions @xcite of tokens, there is no need to label hundreds or even thousands of instances. in this work, we introduce a sample-efficient active learning approach which incorporates selftraining to reduce the amount of training data required for our key task of text classification. our contributions are as follows: (1) we propose a simple yet highly effective self-training approach that complements high-quality active learning labels with high-quantity pseudo-labels. (2) we reproduce four existing self-training approaches, enabling a fair comparison among them despite strongly diverging settings and hyperparameter choices in the original works. (3) in extensive experiments, we compare the new approach to the four reproduced methods on four text classification benchmarks using two query strategies and a baseline. (4) finally, we discuss possible implications for active learning that result from the observed effectiveness, as well as the trade-offs between favoring small versus large language models. the new approach complements active learning with pseudo-labels obtained from the current model. using as few as 130 instances, we achieve scores competitive with regard to the state of the art on three out of four datasets. ## related work in this work, we investigate the intersection of selftraining and active learning for text classification. ## active learning and self-training the goal for active learning is to minimize the annotation effort while maximizing performance regarding some task, such as text classification. problem formulation in pool-based active learning, the training data @xmath0: unlabeled pool u and labeled pool l (i.e., u ∩ @xmath1. during the active learning loop, the query strategy selects the best ranked instances x q ⊆ u, which are then removed from u, labeled by the oracle, and subsequently added to l. we refer to a model as m , and more specifically as m t when it has been trained after query t. we denote the predicted class distribution of instance x during query t (using model m t-1 ) as p t (y|x). ## hard-label regularized self-training based on a methodological analysis (sections 3.3 and 3.4) and a reproduction study (section 5), we present hard-label neighborhood-regularized selftraining (hast, pronounced "haste"), a novel selftraining method that aims to complement active learning with large quantities of pseudo-labels. the idea of our proposed approach is to rely on the generalization capabilities provided by contrastive (representation) learning and on the regularization provided by nearest neighbor relationships in the resulting embedding space. in contrastive learning, training is performed with ntuples of instances, in the following assumed to be pairs. a pair can be formed between any two instances, and consequently each additional pseudolabeled instance increases the number of possible pairings. obviously, when combining this with selftraining, this can considerably enhance the effective number of instances-at the risk of introducing noise due to incorrect pseudo-labels. ## experiments in the experiments, we evaluate the proposed selftraining method hast. moreover, in an extensive reproduction study, we re-implement the four most relevant previous self-training approaches, evaluating them on active learning for text classification. ## discussion the experiments have shown that hast is a highly effective self-training strategy that is able to leverage a large number of pseudo-labels. when combined with setfit models, hast outperforms most other approaches and is on par with nest in classification performance and area under curve. further investigation revealed that these gains can likely be attributed to the large number of pseudo-labels (as shown in appendix through an extensive reproduction, we have also investigated the relative strength of ust, actune, verips, and nest in the context of active learning for text classification. the strongest contender is nest, which is on par with hast but does not outperform, despite using a computationally more expensive pseudo-label acquisition. the primary impediment of previous approaches appears to be an overreliance on confidence thresholds, which are difficult to optimize in active learning scenarios. why did the experiments not incorporate the most recent large language models of 1b or more parameters? with a total runtime of 2600 hours, the experiments are already computationally expensive-despite using models that are considered small by today's standards-and would be infeasible with larger model sizes. moreover, research has demonstrated that smaller models can outperform larger ones @xcite when properly fine-tuned or distilled. therefore, we prioritize model efficiency in our active learning research, which ultimately aims to support real-world annotation where smaller models provide a more accessible and practical solution. why did the experiments use only a single selftraining iteration? while increasing the number of self-training iterations may further increase the classification performance, this also runs the risk of degradation @xcite . for this reason, by using only a single self-training iteration we minimize the risk of degradation, thereby using self-training not to replace but to complement active learning, and in favor of real-world settings at only little additional computational costs. for which real-world use cases is the proposed method a good fit? as previously stated, when self-training is combined with active learning it introduces another source of error. the most favorable is the transductive setting @xcite , where the models do not necessarily need to generalize on future unseen data e.g., when active learning is used for labeling corpora in social sciences @xcite or biomedicine @xcite . for datasets, whose class balance is heavily skewed, it might not be optimal yet, but this remains to be investigated in future research. will the reduced need in labeled data ultimately render active learning obsolete? although our work has shown that strong models can be trained using very few samples, this was conducted on established benchmark datasets that are relatively small in size and less challenging compared to realworld datasets, which may have hundreds of multilabel classes, hierarchies, or highly skewed class distributions. while simpler tasks, such as twoclass sentiment analysis, might be solvable with zero-shot learning, more complex problems will still benefit from active learning. should text classification become able to tackle the majority of problems through zero-shot or few-shot, active learning will remain valuable for refining class definitions by providing high-quality labels for instances where the current model exhibits high uncertainty. ## conclusions in this work, we devise and evaluate hast, a new self-training approach that is tailored to contrastive learning and aims to generate a large number of pseudo-labels to enhance the efficiency of contrastive training. we reproduce four existing selftraining approaches and evaluate all approaches on the task of active learning for text classification. using only small language models of 110m parameters and 25% of instances used in previous work, the proposed approach achieves results close to the state of the art on three out of four datasets.