Textual Data Augmentation for Efficient Active Learning on Tiny Datasets (2020)

Abstract: In this paper we propose a novel data augmentation approach where guided outputs of a language generation model, e.g. GPT-2, when labeled, can improve the performance of text classifiers through an active learning process. We transform the data generation task into an optimization problem which maximizes the usefulness of the generated output, using Monte Carlo Tree Search (MCTS) as the optimization strategy and incorporating entropy as one of the optimization criteria. We test our approach against a Non-Guided Data Generation (NGDG) process that does not optimize for a reward function. Starting with a small set of data, our results show an increased performance with MCTS of 26% on the TREC-6 Questions dataset, and 10% on the Stanford Sentiment Treebank SST-2 dataset. Compared with NGDG, we are able to achieve increases of 3% and 5% on TREC-6 and SST-2.

## introduction active learning (al) is a well applied approach in areas where unlabeled data is abundantly available, but labels are either scarce or costly to obtain @xcite . in al, a classifier is improved upon through an iterative learning process; at each cycle, a subset of the original dataset with the most informative examples is selected to be labeled, typically by a human expert, before it is then added to the existing training data @xcite . previous active learning research on textual data to our knowledge has always assumed the availability of datasets containing large pools of unlabeled data. in cases where the available data is insufficient for active learning, the burden is transferred to the data collection process, where additional data must either be manually created, or collected from real world interactions. one strategy for creating data is to transform existing examples in certain ways in order to produce new data items and hence increase the size of the training dataset. this approach has been applied successfully in computer vision for example, by manipulating existing images while preserving the label to create additional data points @xcite . however, in nlp, augmenting data is a very difficult task due to the complex nature of language @xcite . in this paper we assume a real-life scenario where the data at hand is insufficient for running a typical active learning algorithm. we introduce a method that enables us to automatically generate artificial text examples that complement an existing dataset. our approach minimizes the human factor in data creation by automating the process through a guided searching procedure. once a set of examples is generated, it is required to be manually labeled before it is added to the original training set. the classifier is then retrained on the new, augmented training set. this procedure is repeated multiple times until the desired performance is either achieved or until performance no longer improves. we do this in active learning cycles, where only a subset of the generated data is used; this involves selecting examples in terms of how much information they would add to an existing classifier. in our experiments we use entropy as a measure of informativeness. as our aim is to find the most informative examples, we tackle this problem by applying a search approach. given that text examples are generated from an extremely large number of possible combinations, we apply the monte carlo tree search mcts algorithm @xcite to limit the search space. here mcts is expected to guide a language generation model to output informative examples. in this context, mcts assigns values to previously generated examples using a scoring function that incorporates the learning classifier of the previous active learning cycle, starting with a baseline classifier trained on the initial dataset for the first active learning run. in our experiments, we test mcts with two different scoring functions: one that only measures the uncertainty of the generated examples through entropy, and another that combines the measure of uncertainty with a measure of diversity by computing the cosine similarity of every newly generated example with the previous content. these scores determine the text premise that is passed to the language model when generating newer examples. we compare mcts to non-guided data generation (ngdg), an approach where the knowledge of the learning classifier is not involved in the data generation process. here, for each newly generated example, the text premise is always a token representing the beginning of a sentence, <bos>. the remainder of this paper is organized as follows; section 2 provides a background as well as an overview of related literature. section 3 describes the proposed approach. section 4 presents the experiments which were carried out. section 5 gives conclusions and plans for future work. ## proposed method in games, mcts can be applied to predict moves in order to counter an opponent's strategy so that a winnable state is reached. however, text generation is more similar to a single player scenario, where decisions are based on which token to select when moving from one state to the other. a language model calculates a probability distribution over a sequence of words. when passing over a stream of text, each vocabulary token is assigned a probability score for occurring next. hence, tokens with higher probability scores are more likely to appear next in the sequence. in our setting, we are interested in multiple token candidates for all remaining words in the sequence. to achieve this we use a top-k sampling scheme as used by @xcite . at each time step, each token in the vocabulary is assigned a probability score for coming next in the sequence. to get the top k candidates, vocabulary tokens are sorted by their probability scores and anything below the k'th token is then zeroed out. the probability mass is then redistributed among the k token candidates. this process can be modeled as a tree where each node represents a token linked to k child nodes representing the top k candidate tokens that are likely to appear next in the sequence. hence, this is similar to a board game where each board position is represented by a node: the root node corresponds to an empty board while a terminal node is where no further moves can be made. in our setting, we use a special token for both the root and terminal nodes. we represent the starting token @xmath0. as an example, a language model that is finetuned on a survey on pet adoption could be used to generate the tree of predictions in figure 1 . a full version of this tree would represent all the possible combinations of text that can be generated by the language model. in an ideal setting, we would search this tree for the paths that represent the most informative examples. however, given that the tree will grow exponentially as the number of next token candidates is increased, it would be computationally expensive to apply a brute force search algorithm where every path is examined. for this reason, we apply the monte carlo tree search (mcts) algorithm in the data generation process, as discussed next. figure 1 : mcts traverses down the tree as it creates paths spanning from the root @xmath1. tokens of the same path form a sentence when concatenated e.g. the path in red. ## conclusion in this paper we proposed a framework for improving a classifier's performance with synthetic data. we have shown in our experiments that even when starting with just a few examples, we are able to achieve noticeable improvements. we believe this approach is likely to work for any domain or language so long as the language model is able to generate meaningful output. in this work for instance, we did not need more than 20 examples to fine-tune gpt-2 for the sst-2 experiments, or 30 for the trec-6 experiments. we expect even better results when more examples are provided or with the application of an improved language model like gpt-3. in future work, we plan to extend our approach to further improve the reward and policy functions, and to reduce the human-labeling factor.