Generative Adversarial Networks for Annotated Data Augmentation in Data Sparse NLU (2020)

Abstract: Data sparsity is one of the key challenges associated with model development in Natural Language Understanding (NLU) for conversational agents. The challenge is made more complex by the demand for high quality annotated utterances commonly required for supervised learning, usually resulting in weeks of manual labor and high cost. In this paper, we present our results on boosting NLU model performance through training data augmentation using a sequential generative adversarial network (GAN). We explore data generation in the context of two tasks, the bootstrapping of a new language and the handling of low resource features. For both tasks we explore three sequential GAN architectures, one with a token-level reward function, another with our own implementation of a token-level Monte Carlo rollout reward, and a third with sentence-level reward. We evaluate the performance of these feedback models across several sampling methodologies and compare our results to upsampling the original data to the same scale. We further improve the GAN model performance through the transfer learning of the pre-trained embeddings. Our experiments reveal synthetic data generated using the sequential generative adversarial network provides significant performance boosts across multiple metrics and can be a major benefit to the NLU tasks.

## introduction over recent years, various task-oriented conversational agents, such as amazon alexa, apple's siri, google assistant, and microsoft's cortana, have become more popular in people's everyday life and are expected to be highly intelligent. for the nlu component, this means that we expect models to perform recognition of the actions and entities within a user's request with high accuracy. when first training an nlu model on a new language (a process referred to as bootstrapping a new language), there is a strong requirement for high quality annotated data that would support the most common user requests across a range of domains. as the modeling space expands to support new features and additional languages, nlu models are regularly re-trained on updated data sets to ensure support for these new functions. the major bottleneck in both of these processes is the labor and cost associated with collecting and annotating new training utterances for every new feature or language. recent advances in machine learning methods, including the use of techniques such as transfer learning @xcite and active learning @xcite , can lead to more efficient data usage by nlu models and therefore decrease the need for annotated training data. additionally, data augmentation models are being widely explored. the advantage of data augmentation is that once synthetic data is generated, it can be ingested into subsequent models without additional effort, allowing for faster experimentation. nlu models in dialog systems can perform a variety of tasks @xcite . in this study, we will focus on three of them: domain classification (dc) -identify the domain that the user request belongs to (music, reminders, alarm, etc.), intent classification (ic) -extract actions requested by users (play music, find a restaurant, set an alarm, etc.), and named entity recognition (ner) -identify and extract entities (names, values, dates, locations, etc.) from user requests. for each utterance we expect our nlu model to output a domain, intent, and set of extracted entities with corresponding tags. for example, if a user requests "play bohemian rhapsody by queen", we expect the nlu model to return {domain: music, intent: play song, named entities: [(bohemian rhapsody, song name), (queen, artist name)]}. we call this output annotation, and the utterance along with annotation is called an annotated utterance. named entities with corresponding labels are called slots. for our nlu model to perform well on real-time user requests, we need to train it on a large dataset of diverse annotated utterances. however, there could be some areas of functionality where large datasets for training are not available. to boost model performance in situations where training data is limited, we use synthetic data generated from a small set of unique utterances that cover the basic functionality of the user experience, called golden utterances. we leverage a sequence generative adversarial networks (seqgan) introduced by @xcite to generate new utterances from this "seed" set, and use these generated utterances to augment training data and evaluate the performance of the classification and recognition tasks. we also investigate how the metrics that we use to evaluate the quality of the generated synthetic data links to the performance boost in the underlying tasks. synthetic data have shown to be useful for ic model boosting. for example, @xcite explored a set of encoder-decoder models and proposed the use of conditional vaes (cvaes) to generate phrase templates, called carrier phrases. authors used cvaes to control the domain, intent, and slot types to generate desirable outputs that resulted in a higher f1 score on the intent classification task. @xcite focused on a few-shot ic problem where new categories with limited training data are introduced into an existing system with mature categories. they compared different techniques that were designed to augment training data, including upsampling, random perturbation, extrapolation, cvaes, and delta-encoders, and combined feature space augmentation with popular bert pre-training @xcite to provide better performance. the use of gans has been previously explored for text data augmentation in language modeling @xcite @xcite @xcite @xcite @xcite and sentiment classification @xcite . however, discrete text sequence generation brings about several challenges: first, one needs to generate a set of discrete tokens from a random sample of real-valued continuous data, and second, gans are designed to give feedback on entire sequences, whereas generators need guidance for each subsequent token. the seqgan model developed by @xcite attempts to resolve these issues by applying reinforcement algorithms for the gan objective with a policy gradient that evaluates current state-action value using monte carlo (mc) search. in this work, we adopt a seqgan model to boost dc, ic, and ner tasks in nlu models that suffer from sparse data limitations. ## data generation we explore the following different gan frameworks and expansions to generate synthetic annotated utterances: the model architecture consists of three parts: generator pre-training, discriminator pre-training, and adversarial training. in the pre-training parts, we first train the generator, followed by the discriminator, for 80 epochs each. for each adversarial epoch, we update the generator once and then for 35 steps we generate negative examples using the current state of the generator, combined with the same number of positive examples from the training data, and re-train the discriminator. the total number of adversarial training epochs is set to 600. all hyper-parameters were chosen based on observations of when the loss functions and synthetic data quality evaluations either stabilize or clearly degrade. ## conclusions in this paper, we evaluate the use of the seqgan model for synthetic annotated data generation to boost nlu model performance. we have shown that adding synthetic data to bolster our goldens can significantly improve dnn model performance in intent classification and named entity recognition tasks. we propose a token-level reward with monte carlo search rollout to guide the generator model, that showed better performance when compared with a regular token-level reward implementation, sentence-level reward implementations both with and without monte carlo tree search, and with a pure upsampling strategy. we also show that using seqgan together with embeddings pre-trained on high-resource domains to generate synthetic data can significantly improve the performance of low-resource domains. embeddings pre-trained on different tasks can carry over the information they have learned and that can be especially useful in low-resource model building scenarios.