Active Sentence Learning by Adversarial Uncertainty Sampling in Discrete Space (2020)

Abstract: Active learning for sentence understanding aims at discovering informative unlabeled data for annotation and therefore reducing the demand for labeled data. We argue that the typical uncertainty sampling method for active learning is time-consuming and can hardly work in real-time, which may lead to ineffective sample selection. We propose adversarial uncertainty sampling in discrete space (AUSDS) to retrieve informative unlabeled samples more efficiently. AUSDS maps sentences into latent space generated by the popular pre-trained language models, and discover informative unlabeled text samples for annotation via adversarial attack. The proposed approach is extremely efficient compared with traditional uncertainty sampling with more than 10x speedup. Experimental results on five datasets show that AUSDS outperforms strong baselines on effectiveness.

## introduction deep neural models become popular in natural language processing @xcite @xcite . neural models usually consume massive labeled data, which requires a huge quantity of human labors. but data are not born equal, where informative data with high uncertainty are decisive to decision boundary and are worth labeling. thus selecting such worth-labeling data from unlabeled text corpus for annotation is an effective way to reduce the human labors and to obtain informative data. active learning approaches are a straightforward choice to reduce such human labors. previous works, such as uncertainty sampling @xcite , needs to traverse all unlabeled data to find informative unlabeled samples, which are always near the decision boundary with large entropy. however, the traverse process is very time-consuming, thus cannot be executed frequently @xcite . a common choice is to perform the sampling process after every specific period, and it samples and labels informative unlabeled data then trains the model until convergence @xcite . we argue that infrequently performing uncertainty sampling may lead to the "ineffective sampling" problem. because in the early phase of training, the decision boundary changes quickly, which makes previously collected samples less effective after several updates of the model. ideally, uncertainty sampling should be performed frequently in the early phase of model training. in this paper, we propose the adversarial uncertainty sampling in discrete space (ausds) to address the ineffective sampling problem for active sentence learning by introducing more frequent sampling with significantly lower costs. specifically, we propose to leverage the adversarial attack @xcite to the selecting of informative samples with high uncertainty, which significantly narrows down the search space. fig. 1 shows the difference between uncertainty sampling and ausds. the typical uncertainty sampling (fig. 1 .a) traverses all the unlabeled samples to obtain samples of high uncertainty for each sampling run, which is costly with time complexity (o(unlabeled data size). ausds (fig. 1 .b) first projects a labeled text to the decision boundary, denoted as an adversarial data point, and searches nearest neighbors of this point. the computational cost of ausds is significantly smaller than typical uncertainty sampling with the time complexity o(batch size). but it is non-trivial for ausds to perform adversarial attacks, which requires adversarial gradients on sentences, since texts live in a discrete space. we propose to include a pre-trained neural encoder, such as bert @xcite , to map unlabeled sentences into a continuous space, over which the adversarial attack is performed. since not every adversarial data point in the encoding space can be mapped back to one of the unlabeled sentences, we propose to use the k-nearest neighbor (knn) algorithm @xcite ) to find the most similar unlabeled sentences (the adversarial samples) to the adversarial data points.besides, empirically, we mix some random samples into the uncertainty samples to alleviate the sampling bias issue mentioned by @xcite . finally, the mixed samples are sent to an oracle annotator to obtain their label and are appended to the labeled data set. we deploy ausds for active sentence learning and conduct experiments on five datasets across two nlp tasks, namely sequence classification and sequence labeling. experimental results show that ausds outperforms random sampling and uncertainty sampling strategies. our contributions are summarized as follows: ## related work this work focuses on reducing the labeled data size with the help of pre-trained lm in solving sentence learning tasks. the proposed ausds approach is related to two different research topics, active learning and adversarial attack. ## active sentence learning with ausds we propose ausds learning framework, an efficient and effective computational framework for active sentence learning. the overview of the learning framework is shown in fig. 2 . the learning framework consists of two blocks, a training block and a sampling block ausds. the training block learns knowledge from the labeled data, whereas the sampling block retrieves valuable unlabeled samples, whose latent states are close to the decision boundary over the latent space, from the unlabeled text corpus. note that the definition of latent spaces can be different across encoders and tasks. the samples retrieved by the sampling block will be further sent to an oracle annotator to obtain their label, and the new samples with labels are also appended to the labeled data. in this section, we first introduce ausds method by showing how ausds select samples that are critical to the decision boundary over the latent space. then we present the computational procedure of the full-fledged framework in detail. i ← i + 1 20: end while 3.1 ausds ausds first defines a latent space, over which sentences are distinguishable according to the model's decision boundary. the latent space is usually determined by the encoder architecture and the downstream task. we detail the latent space definition of specific encoders and tasks in sec. 4.1. at first, we sample a batch of labeled texts and compute their representation as well as their gradients in the latent space. using the latent states and their gradients, we perform adversarial attacks to generate adversarial data points a near the decision boundary in the latent space. adversarial attacks are performed using the following existing approaches: fgv is efficient in the calculation, whereas the other two methods typically find more precise adversarial data points but with larger computational costs. we use all of them in our experimental part to show the effectiveness of the ausds. in our sentence learning scenario, the adversarial data points a cannot be grounded on real natural language text samples. thus we perform knearest neighbor (knn) search (altman, 1992) to find unlabeled text samples whose latent states are k-nearest to the adversarial data points a. we implement the knn search using faiss foot_0 @xcite , an efficient similarity search algorithm with gpus. the computational cost of knn search results from two processes, including constructing a sample mapper m between text and latent space, and searching similar latent states of adversarial data points. the sampler mapper m here is constructed as a hash map, which is of high computational efficiency, to memorize the mapping between an unlabeled text s and its latent representation x. the sample mapper is only reconstructed when the encoder is updated, and infrequent encoder updates contribute to efficiency. besides, the searching process is also fast (100× faster than generating a) thanks to faiss. thus it is possible to performed ausds frequently at batch-level without harming computation. after acquiring adversarial samples s a using knn search, we mix s a with random samples s r drawn from unlabeled text corpus t i by the ratio of p : 1p, where p is a hyper-parameter determined on the development set. the motivation of appending random samples is to balance exploration and exploitation, thus avoiding the model continuously retrieve samples in a small neighborhood. we perform top-k ranking over the information entropy of the mixed samples to further retrieve samples with higher uncertainty. since the size of the mixed samples is comparable to the batch size, the computation cost is acceptable. the remaining samples are further sent to an oracle annotator o to obtain their labels. ## experiments we evaluate the ausds learning framework on sequence classification and sequence labeling tasks. for the oracle labeler o, we directly use the labels provided by the datasets. in all the experiments, we take average results of 5 runs with different random seeds to alleviate the influence of randomness. ## conclusion uncertainty sampling is an effective way of reducing the labeled data size in sentence learning. but uncertainty sampling of high latency may lead to an ineffective sampling problem. in this study, we propose adversarial uncertainty sampling in discrete space for active sentence learning to address the ineffective sampling problem. the proposed ausds is more efficient than traditional uncertainty sampling by leveraging adversarial attacks and projecting discrete sentences into pre-trained lm space. experimental results on five datasets show that the proposed approach outperforms strong baselines in most cases, and achieve better sampling effectiveness.