Active ^2 Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation (2021)

Abstract: While deep learning is a powerful tool for natural language processing (NLP) problems, successful solutions to these problems rely heavily on large amounts of annotated samples. However, manually annotating data is expensive and time-consuming. Active Learning (AL) strategies reduce the need for huge volumes of labeled data by iteratively selecting a small number of examples for manual annotation based on their estimated utility in training the given model. In this paper, we argue that since AL strategies choose examples independently, they may potentially select similar examples, all of which may not contribute significantly to the learning process. Our proposed approach, Active \mathbf{^2} Learning (A \mathbf{^2} L), actively adapts to the deep learning model being trained to eliminate such redundant examples chosen by an AL strategy. We show that A \mathbf{^2} L is widely applicable by using it in conjunction with several different AL strategies and NLP tasks. We empirically demonstrate that the proposed approach is further able to reduce the data requirements of state-of-the-art AL strategies by \approx \mathbf{3-25\%} on an absolute scale on multiple NLP tasks while achieving the same performance with virtually no additional computation overhead.

## introduction active learning (al) @xcite reduces the need for large quantities of labeled data by intelligently selecting unlabeled examples for expert annotation in an iterative process. many natural language processing (nlp) tasks like sequence tagging (ner, pos) and neural machine translation (nmt) are very data-intensive and require a meticulous, timeconsuming, and costly annotation process. on the other hand, unlabeled data is practically unlimited. due to this, many researchers have explored applications of active learning for nlp @xcite . a general al method proceeds as follows: (i) the partially trained model for a given task is used to ( possibly incorrectly we claim that all al strategies select redundant examples in step (ii). if one example satisfies the selection criterion, then many other similar examples will also satisfy it (see the next paragraph for details). as the examples are selected independently, al strategies redundantly choose all of these examples even though, in practice, it is enough to label only a few of them (ideally just one) for training the model. this leads to higher annotation costs, wastage of resources, and reduces the effectiveness of al strategies. this paper addresses this problem by proposing a new approach called a 2 l (read as active-squared learning) that further reduces the redundancies of existing al strategies. any approach for eliminating redundant examples must have the following qualities: (i) the redundancy should be evaluated in the context of the trained model. (ii) the approach should apply to a wide variety of commonly used models in nlp. (iii) it should be compatible with several existing al strategies. the first point merits more explanation. as a model is trained, depending on the downstream task, it learns to focus on certain properties of the input. examples that share these properties (for instance, the sentence structure) are similar from the model's perspective. if the model is confused about one such example, it will likely be confused about all of them. we refer to a similarity measure that is computed in the context of a model as a model-aware similarity (section 3.1). ## related work active learning has a long and successful history in the field of machine learning @xcite . however, as the learning models have become more complex, especially with the advent of deep learning, the known theoretical results for active learning are no longer applicable @xcite . this has prompted a diverse range of heuristics to adapt the active learning framework to deep learning models @xcite . many al strategies have been proposed @xcite @xcite @xcite , however, since they choose the examples independently, the problem of redundancy (section 1) applies to all. we experiment with various nlp tasks like named entity recognition (ner) @xcite , part-of-speech tagging (pos) @xcite , neural machine translation (nmt) @xcite @xcite @xcite and so on @xcite . the tasks chosen by us form the backbone of many practical problems and are known to be computationally expensive during both training and inference. many deep learning models have recently advanced the state-of-art for these tasks @xcite @xcite . our proposed approach is compatible with any nlp model, provided it supports the usage of an al strategy. existing approaches have used modelindependent similarity scores to promote diversity in the chosen examples. for instance, in @xcite , the authors use cosine similarity to pre-calculate pairwise similarity between examples. we instead argue in favor of model-aware similarity scores and learn an expressive notion of similarity using neural networks. we compare our approach with a modified version of this baseline using cosine similarity on infersent embeddings @xcite . ## proposed approaches we use m to denote the model being trained for a given task. m has a module called encoder for encoding the input sentences. for instance, the encoder in m may be modeled by an lstm (hochreiter and schmidhuber, 1997). ## experiments we establish the effectiveness of our approaches by demonstrating that they: (i) work well across a variety of nlp tasks and models, (ii) are compatible with several popular al strategies, and (iii) further reduce the data requirements of existing al strategies, while achieving the same performance. in particular, we experiment foot_0 with two broad categories of nlp tasks: (a) sequence tagging (b) neural machine translation. ## results figure 2 compares the performance of our methods with baselines. it shows the test-set metric on the y-axis against the percentage of training data used on the x-axis for all tasks. see figures 7 and 8 in the appendix for additional results. our ablation studies in figure 3 show the utility of using model-aware similarity. 2. in sequence tagging, we match the performance obtained by training on the full dataset using only a smaller fraction of the data (3 -25% less data as compared to state-of-art al strategies) ( we wish to state that our approach should be evaluated not in terms of the gain in the f1 score but no. of examples selected by al strategy spectral (examples processed per second) integrated (examples processed per second) 5000 491.64 2702.70 10000 248.08 2557.54 15000 163.10 2508.36 20000 122.89 2493.76 in terms of the reduction in data required to achieve the same (3-25 % on multiple datasets). more importantly, this improvement comes at a negligible computation overhead cost. the reported improvements are not relative with respect to any baseline but represent an absolute value and are very significant in the context of similar performance improvements reported in the literature. in figure 5 , we provide a qualitative case study that demonstrates the problem of redundancy. ## conclusion this paper shows that one can further reduce the data requirements of active learning strategies by proposing a new method, a 2 l, which uses a model-aware-similarity computation. we empirically demonstrated that our proposed approaches consistently perform well across many tasks and al strategies. we compared the performance of our approach with strong baselines to ensure that the role of each component is properly understood.