Low-resource Interactive Active Labeling for Fine-tuning Language Models (2022)

Abstract: Recently, active learning (AL) methods have been used to effectively fine-tune pre-trained language models for various NLP tasks such as sentiment analysis and document classification. However, given the task of fine-tuning language models, understanding the impact of different aspects on AL methods such as labeling cost, sample acquisition latency, and the diversity of the datasets necessitates a deeper investigation. This paper examines the performance of existing AL methods within a low-resource, interactive labeling setting. We observe that existing methods often underperform in such a setting while exhibiting higher latency and a lack of generalizability. To overcome these challenges, we propose a novel active learning method TYROUGE that employs a hybrid sampling strategy to minimize labeling cost and acquisition latency while providing a framework for adapting to dataset diversity via user guidance. Through our experiments, we observe that compared to SOTA methods, TYROUGE reduces the labeling cost by up to 43% and the acquisition latency by as much as 11X, while achieving comparable accuracy. Finally, we discuss the strengths and weaknesses of TYROUGE by exploring the impact of dataset characteristics.

## introduction while fine-tuning pre-trained language models has become a standard practice for nlp tasks, data labeling remains a major bottleneck for nlp. to alleviate this problem, active learning (al) has been recently employed to fine-tune language models for downstream tasks @xcite @xcite . active learning aims to reduce the human labeling effort by focusing on the most informative samples that can enhance model performance efficiently. unfortunately, even with state-of-the-art al approaches, the number of labels needed to fine-tune language models is still significant. however, the availability of suitable annotators is often scarce, and obtaining human annotation can be expensive. for example, labeling tens of thousands of data samples may be impractical for domains such as medical or legal, considering the cost and time for labeling as well as the overhead of finding and training domain experts. this paper focuses on the low-resource setting where less than 1,000 samples are labeled in total, following @xcite ; @xcite ; @xcite . given such a low-resource setup, another challenge is the interactivity of al methods. al interactivity is often not considered in existing literature, but annotators' waiting time between labeling iterations can be a significant bottleneck. in addition, low latency is essential for the early stages of model building, where nlp researchers and practitioners aim to explore the model performance over faster al iterations. however, when selecting samples, existing methods operate over the entire unlabeled dataset leading to higher latency in acquiring labeling candidates. as a result, the acquisition time for sota methods often violates the time-constraint of interactive systems @xcite . therefore, we argue that an interactive low-resource acquisition strategy must balance the trade-off between labeling budget and the model performance, as they acquire samples while ensuring faster turnaround time. another relevant aspect we observe through experiments is that state-of-the-art al approaches acquire redundant samples. existing al acquisition strategies focus on uncertainty (houlsby figure 1 : illustration of the sample redundancy challenge on agnews dataset @xcite . et al. @xcite or diversity @xcite @xcite , or both @xcite . uncertainty-based methods select samples near decision boundaries, whereas diversity-based methods try to obtain diverse samples. as shown in figure 1b , uncertaintybased methods acquire similar samples within an al iteration -intra-iteration redundancy. diversity-based al approaches acquire similar samples across iterations -inter-iteration redundancy (figure 1c ). existing hybrid methods aiming to balance uncertainty and diversity objectives surprisingly prioritize one objective over another and do not provide any mechanism to control the balance between these objectives. therefore, even existing hybrid approaches also suffer from the aforementioned redundancies. such redundancy leads to a wasted labeling budget, indicating room for reducing labeling costs across al iterations. moreover, as existing methods optimize for specific objectives, i.e., uncertainty, and diversity, it is unclear whether these approaches generalize to datasets of varying degrees of difficulty with respect to scale and domain diversity. taking these observations into consideration, we propose tyrogue 1 , a low-resource interactive active learning method that minimizes labeling budget by reducing sample redundancy while achieving comparable accuracy. tyrogue adopts a hybrid strategy where it incorporates independent 1 tyrogue is a pokèmon that evolves in its appearance depending on the situation https://bulbapedia. bulbagarden.net/wiki/tyrogue_(pok%c3% a9mon) steps implementing diversity and uncertainty sampling while instrumenting a control parameter to adapt the degree of influence of each objective during acquisition. moreover, tyrogue performs an initial filter on the unlabeled data to reduce the candidate sampling pool, thereby reducing latency without negatively impacting model performance. our contributions are the following: ## related work we now discuss related work relevant to our setup. learning with pre-trained lms. two popular methods for learning with pre-trained language models (lms) are few-shot in-context learning (icl) and fine-tuning. icl enables pre-trained lms to perform a new task without any gradientbased training by providing a small number of training examples as part of the input. however, icl lacks interactivity as it processes all of the training examples for each prediction made, incurring significant storage and computational costs @xcite . fine-tuning, on the other hand, trains parameters to enable a model to perform the new task. we specifically focus on active learning strategies to acquire samples for fine-tuning. several settings for active learning have been proposed over the years @xcite . these methods can be either instance-based, i.e., acquiring a single data point, or batch-based, i.e., acquiring a collection of data points. batch active learning is suitable for meaningfully fine-tuning language models. in this paper, we adopt the batch active learning as in recent existing studies @xcite @xcite @xcite . acquisition strategies for batch al. as mentioned in section 1, uncertainty-based methods employ various techniques such as leveraging the model's predictive entropy, predictive confidence, and mutual information, to select data points for annotation @xcite @xcite . diversity-based methods employ strategies such as core-set construction-based sampling @xcite and discriminative learning @xcite to select data points that are representative of the unlabeled data pool. all of these approaches optimize for the accuracy of the model and do not capture other aspects such as labeling budget, acquisition latency, and generalizability to diverse datasets. tyrogue aims to address all three challenges for fine-tuning language models for nlp tasks. hybrid acquisition strategies combine uncertainty and diversity sampling. to combine both objectives, both badge (ash et al., 2020) and alps @xcite leverage model uncertainty to compute embeddings of data points and then perform clustering to acquire diverse samples. cluster-margin @xcite operates in large batches, i.e., selecting 100k data points, and combines hierarchical clustering with predictive uncertainty to acquire data points. these methods ignore the interactive setting and employ clustering over the entire unlabeled pool which can be timeconsuming. cal @xcite balances the trade-off between uncertainty and diversity by acquiring contrastive data points that are similar in the model feature space but differ in model's predictive likelihoods. to operate efficiently, cal requires a larger batch size than expected in a lowresource setting. hence, these hybrid methods are not suitable for the low-resource and interactive setting. moreover, they use a single acquisition objective combining diversity and uncertainty without any control afforded to the user. as a result, they tend to prioritize one objective over another (e.g., cal is highly uncertainty-focused), depending on the algorithm designs even though datasets may have different characteristics. unlike these approaches, tyrogue introduces a hybrid acquisition strategy with tunable control parameters, which is suitable for the low-resource and interactive setting. low-resource active learning for lms. @xcite explore the performance of state-of-the-art uncertainty-based active learning methods in a low-resource setting while fine-tuning language models such as bert @xcite and roberta @xcite . the authors highlight that there is significant drop in performance of the models in a low-resource setting. in our work, besides uncertainty-based methods, we investigate the performance of diversity-based and hybrid methods in a low-resource setting while exploring additional dimensions such as interactivity and dataset diversity. while alps @xcite also operates in low-resource setting, it's not interactive due to the higher latency in clustering the entire unlabeled data pool. hence, alps is not suitable for the interactive setting. finally, grießhaber et al. (2020) also explore low-resource active learning with regards to aspects such as model update strategies which is beyond the scope of this paper. ## low-resource interactive al we now present tyrogue, a novel acquisition function that performs effectively and efficiently in the low-resource and interactive setting. we first explain the limitations of existing al methods and then describe our problem setting. we then describe key design criteria and the corresponding pipeline for tyrogue to overcome the limitations. ## experiments we now present the experiment set-up and results. setup. to demonstrate the reduction in labeling cost and acquisition latency, we compare ty-rogue with sota uncertainty-based (entropy), diversity-based (ftbertkm), and hybrid (badge, alps, and cal) methods. note that entropy is a baseline used in @xcite . we also include random, which draws samples uniformly at random from the unlabeled pool. random acquisition is non-active (or passive) as its sample selection does not depend on any model output. we evaluate the methods by fine-tuning the pretrained bert in active iterations on seven datasets used by @xcite and the paws-qqp dataset @xcite . used datasets are described in ## conclusion and future work we present tyrogue, a novel active learning method that overcomes the two major limitations of existing methods: 1) high acquisition latency and 2) sample redundancy. through our experiments, we observe that tyrogue runs faster than existing methods for larger datasets and reduces sample redundancy through effective combination of diversity and uncertainty, thereby reducing the labeling cost. we also observe how adaptability is crucial for obtaining consistent performance across datasets and identify the importance of instrumenting mechanisms to balance the uncertainty-diversity trade-off. towards adaptive acquisition. the trade-off between uncertainty and diversity is essential for active acquisition algorithms. we believe tyrogue and the observations in this work lay the foundation for future work on adaptive acquisition functions that balance both objectives. future extensions to our work can investigate strategies for attaining the optimal balance of uncertainty and diversity by taking into account aspects such as model performances and dataset characteristics. adoption in practical systems. the multi-step adaptive method proposed, tyrogue can be incorporated into any annotation platforms. as outlined in section 1, such frameworks can enable rapid iterations in the early stages of modeling building. therefore, understanding how tyrogue can be integrated into the existing annotation platforms is an interesting research problem. transparency and control for practitioners. future studies may explore how users operate within the interactive al framework. our proposed design affords control to the users in balancing the acquisition dichotomy as mentioned above. however, it is imperative to understand how aspects such as transparency of the framework and interpretability of the model may impact users' experience as they reason over the control parameters.