Deep Active Learning for Sequence Labeling Based on Diversity and Uncertainty in Gradient (2020)

Abstract: Recently, several studies have investigated active learning (AL) for natural language processing tasks to alleviate data dependency. However, for query selection, most of these studies mainly rely on uncertainty-based sampling, which generally does not exploit the structural information of the unlabeled data. This leads to a sampling bias in the batch active learning setting, which selects several samples at once. In this work, we demonstrate that the amount of labeled training data can be reduced using active learning when it incorporates both uncertainty and diversity in the sequence labeling task. We examined the effects of our sequence-based approach by selecting weighted diverse in the gradient embedding approach across multiple tasks, datasets, models, and consistently outperform classic uncertainty-based sampling and diversity-based sampling.

## introduction sequence labeling is one of the commonly used techniques for solving natural language understanding (nlp) tasks such as named-entity recognition (ner) and slot filling. furthermore, for these tasks, the state-of-the-art results are typically based on deep neural networks @xcite @xcite . however, the performance of these models is highly dependent on the availability of large amounts of annotated data. moreover, compared with classification tasks, which require only one label for a sample, the sequence learning tasks require a series of token-level labels for an entire sequence, which makes them time-consuming and a costly annotation process. this problem can be mitigated using active learning (al), which achieves improved performance with fewer annotations by strategically selecting the examples to annotate @xcite @xcite . there are two major strategies for active learning, namely, diversitybased sampling and uncertainty-based sampling @xcite . traditionally, uncertainty-based sampling is the most common pool-based al approach. however, previous work pointed out that focusing only on the uncertainty leads to a sampling bias @xcite . it creates a pathological scenario where selected samples are highly similar to each other, which clearly indicates inefficiency. this may cause problems, especially in the case of noisy and redundant real-world datasets. another approach is diversity-based sampling, wherein the model selects a diverse set such that it represent the input space without adding considerable redundancy @xcite . this approach can select samples while ensuring a maximum batch diversity. however, this approach might select points that provide little new information, thereby reducing the uncertainty of the model. certain recent studies for classification tasks implemented an algorithm named batch active learning by diverse gradient embeddings (badge). this algorithm first computes embedding for each unlabeled sample based on induced gradients, and then geometrically picks the instances from the space to ensure their diversity @xcite . although it proves to be a robust improvement when performing an image classification task, its performance in sequence labeling tasks is yet unproven. in this study, we investigated some practical active learning algorithms that consider uncertainty and diversity in sequence labeling tasks over different datasets and models. moreover, we suggested a method to expand badge with weighted sampling based on the sequence length to ensure @xcite and submodularityinspired data ranking function @xcite to select low-data regime. some studies have explored hybrid methods that incorporate both diversity and uncertainty in classifica-tion. @xcite takes into account both the informativeness of the examples for the model as well as their diversity in a mini-batch during classification; moreover, it considers uncertainty as a scalar value similar to works discussed earlier herein. ash et al. (2020) also designed a badge method to incorporate both predictive uncertainty and sample diversity. this method exhibits a robust performance in different environmental settings while performing an image classification task. in this work, we have provided an empirical evaluation of diverse gradient embeddings for nlp tasks using different models, and explored a different method to consider variablelength input. ## diverse gradient embedding for sequences in this section, we introduce a method that simultaneously captures both uncertainty and diversity for active learning with sequence tagging. badge only addresses the image classification task. moreover, it does not consider tasks that have sequences. in this section, first, we discuss how the gradient embedding of the last layer is related to the representation information. second, we propose a simple modified sampling algorithm with length-normalized weight for improved sampling efficiency. algorithm 1 active learning with diverse gradient embeddings for sequence labeling tasks. input: unlabeled dataset u, sequence length of example l, number of budgets in query k. similar to @xcite , our method, described in algorithm 1, performs two main computations at each al round: (i) extracting gradient embeddings with respect to the parameters of final layers for all the unlabeled samples, and (ii) sampling a batch of query points based on these gradient embeddings using the weighted k-means++ initialization. it selects sentences for which the diversity and uncertainty are high when considering its length. the details of each computation to adapt it to the sequence labeling task is descried as follows: ## results we evaluated the performances of various al methods with different models for the slot-filling and ner tasks and plotted them in figures 2 and 3 , respectively. the x-axis represents the percent of words annotated and used for training and the y-axis indicates the best f1 scores obtained. moreover, standard errors are indicated by shaded regions. in all cases, we observed that the w-badge method indicated a significant improvement or over the baseline al methods. it consistently outperforms those employing either pure uncertainty or diversity-based sampling in both the earlier and latter rounds. the performance gap is clearer, especially in the earlier rounds of training. although badge is comparable with other methods, w-badge performs as well as or better than badge. more specifically, as can be observed from figure 2, w-badge performs satisfactorily in both the atis and fmtod datasets and shows significant improvement in all baselines when performing the slot-filling task. while badge also shows the similar performance in fmtod dataset with bil-stm tagger, w-badge slightly outperformed in bilstm encoder-decoder model. moreover, classic uncertainty-based sampling mnlp outperforms bald and coreset in both datasets. we suppose that the slot-filling task is relatively complex and sparse because it has more classes than the ner task; therefore, uncertainty-based sampling methods have an advantage when performing such a task. figure 3 graphs the learning curves of the al algorithms on conll 2003 dataset for ner task. figure 3: performance comparison of conll2003 dataset and various al sampling methods in the case of ner task we observed that w-badge presented significant improvement over bald, which shows the best performance in ner tasks @xcite and badge. as w-badge outperformed the coreset, it appears that w-badge can be advantageous not only over purely diversity-based approaches but also over classic uncertainty-based approaches in ner task. overall, the advantages of weighted diverse gradient embeddings can be substantial. for example, we find that active learning algorithms achieve 98-99% deep model trained on full data using only 15% of samples in atis dataset. the relative improvement remains significant over baselines. ## conclusion in this study, we explored the empirical study on al utilizing the advantages of both uncertainty and diversity by selecting weighted diverse gradient embeddings to perform a sequence labeling task. we proposed an efficient method and empirically demonstrated that it could consistently achieve a superior performance while consuming much less data. it adds robustness to the dataset and the architecture, thus proving to be a useful option for solving real-world active learning problems