Active Learning by Acquiring Contrastive Examples (2021)

Abstract: Common acquisition functions for active learning use either uncertainty or diversity sampling, aiming to select difficult and diverse data points from the pool of unlabeled data, respectively. In this work, leveraging the best of both worlds, we propose an acquisition function that opts for selecting contrastive examples, i.e. data points that are similar in the model feature space and yet the model outputs maximally different predictive likelihoods. We compare our approach, CAL (Contrastive Active Learning), with a diverse set of acquisition functions in four natural language understanding tasks and seven datasets. Our experiments show that CAL performs consistently better or equal than the best performing baseline across all tasks, on both in-domain and out-of-domain data. We also conduct an extensive ablation study of our method and we further analyze all actively acquired datasets showing that CAL achieves a better trade-off between uncertainty and diversity compared to other strategies.

## introduction active learning (al) is a machine learning paradigm for efficiently acquiring data for annotation from a (typically large) pool of unlabeled data @xcite @xcite . its goal is to concentrate the human labeling effort on the most informative data points that will benefit model performance the most and thus reducing data annotation cost. the most widely used approaches to acquiring data for al are based on uncertainty and diversity, often described as the "two faces of al" @xcite . while uncertainty-based methods leverage the model predictive confidence to select difficult examples for annotation @xcite , diversity sampling exploits heterogeneity in the feature space by typically performing clustering @xcite et @xcite . still, both approaches have core limitations that may lead to acquiring redundant data points. algorithms based on uncertainty may end up choosing uncertain yet uninformative repetitive data, while diversity-based methods may tend to select diverse yet easy examples for the model @xcite . the two approaches are orthogonal to each other, since uncertainty sampling is usually based on the model's output, while diversity exploits information from the input (i.e. feature) space. hybrid data acquisition functions that combine uncertainty and diversity sampling have also been proposed @xcite @xcite @xcite . in this work, we aim to leverage characteristics from hybrid data acquisition. we hypothesize that data points that are close in the model feature space (i.e. share similar or related vocabulary, or similar model encodings) but the model produces different predictive likelihoods, should be good candidates for data acquisition. we define such examples as contrastive (see example in figure 1 ). for that purpose, we propose a new acquisition function that searches for contrastive examples in the pool of unlabeled data. specifically, our method, contrastive active learning (cal) selects unlabeled data points from the pool, whose predictive likelihoods diverge the most from their neighbors in the training set. this way, cal shares similarities with diversity sampling, but instead of performing clustering it uses the feature space to create neighborhoods. cal also leverages uncertainty, by using predictive likelihoods to rank the unlabeled data. we evaluate our approach in seven datasets from four tasks including sentiment analysis, topic classification, natural language inference and paraphrase detection. we compare cal against a full suite of baseline acquisition functions that are based on uncertainty, diversity or both. we also examine robustness by evaluating on out-of-domain data, apart from in-domain held-out sets. our contributions are the following: we release our code online foot_0 . ## contrastive active learning in this section we present in detail our proposed method, cal: contrastive active learning. first, we provide a definition for contrastive examples and how they are related to finding data points that are close to the decision boundary of the model ( §2.1). we next describe an active learning loop using our proposed acquisition function ( §2.2). ## ablation study we conduct an extensive ablation study in order to provide insights for the behavior of every component of cal. we present all al experiments on the agnews dataset in figure 3 . ## analysis finally, we further investigate cal and all acquisition functions considered (baselines), in terms of diversity, representativeness and uncertainty. our aim is to provide insights on what data each method tends to select and what is the uncertainty-diversity trade-off of each approach. ## conclusion & future work we present cal, a novel acquisition function for al that acquires contrastive examples; data points which are similar in the model feature space and yet the model outputs maximally different class probabilities. our approach uses information from the feature space to create neighborhoods for each unlabeled example, and predictive likelihood for ranking the candidate examples. empirical experiments on various in-domain and out-of-domain scenarios demonstrate that cal performs better than other acquisition functions in the majority of cases. after analyzing the actively acquired datasets obtained with all methods considered, we conclude that entropy is the hardest baseline to beat, but our approach improves it by guiding uncertainty sampling in regions near the decision boundary with more informative data. still, our empirical results and analysis show that there is no single acquisition function to outperform all others consistently by a large margin. this demonstrates that there is still room for improvement in the al field. furthermore, recent findings show that in specific tasks, as in visual question answering (vqa), complex acquisition functions might not outperform random sampling because they tend to select collective outliers that hurt model performance @xcite . we believe that taking a step back and analyzing the behavior of standard acquisition functions, e.g. with dataset maps @xcite , might be beneficial. especially, if similar behavior appears in other nlp tasks too. another interesting future direction for cal, related to interpretability, would be to evaluate whether acquiring contrastive examples for the task @xcite is more beneficial than contrastive examples for the model, as we do in cal. most computationally heavy approaches, require o(knd) and o(cknd) respectively, given that gradient embeddings are computed for badge foot_7 . on the other hand, entropy only requires n forward passes though the model, in order to obtain the logits for all the data in d pool . instead, our approach, cal, first requires m + n forward passes, in order to acquire the logits and the cls representations of the the data (in d pool and d lab ) and then one iteration for all data in d pool to obtain the scores. we present the runtimes in detail for all datasets and acquisition functions in tables 4 and 5 . first, we define the total acquisition time as a sum of two types of times; inference and selection time. inference time is the time that is required in order to pass all data from the model to acquire predictions or probability distributions or model encodings (representations). this is explicitly required for the uncertainty-based methods, like entropy, and our method cal. the remaining time is considered selection and essentially is the time for all necessary computations in order to rank and select the b most important examples from d pool . we observe in in