Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generation (2020)

Abstract: Data augmentation is proven to be effective in many NLU tasks, especially for those suffering from data scarcity. In this paper, we present a powerful and easy to deploy text augmentation framework, Data Boost, which augments data through reinforcement learning guided conditional generation. We evaluate Data Boost on three diverse text classification tasks under five different classifier architectures. The result shows that Data Boost can boost the performance of classifiers especially in low-resource data scenarios. For instance, Data Boost improves F1 for the three tasks by 8.7% on average when given only 10% of the whole data for training. We also compare Data Boost with six prior text augmentation methods. Through human evaluations (N=178), we confirm that Data Boost augmentation has comparable quality as the original data with respect to readability and class consistency.

## introduction data augmentation is a widely-used technique in classification tasks. in the field of computer vision (cv), data is augmented by flipping, cropping, tilting, and altering rgb channels of the original images @xcite @xcite ; however, similar intuitive and simple strategies do not obtain equal success in nlp tasks. existing methods tend to produce augmentation with low readability or unsatisfying semantic consistency @xcite . original so cute! the baby is very lovely! naive aug. delete + swap so cute! is the baby very! word2vec aug. insert + replace so cute adorable! the baby is very fabulous! back translate aug. eng. → fr. → eng. cute! the baby is very cute! data boost look at this adorable baby! he is so cute! although back-translation is intuitive and valid, its generation skews towards high frequency words (e.g., cute, lovely are both back-translated to cute), which not only causes repetition but also leads to lexical shrinkage in the augmented data. in a nutshell, existing techniques are still far from perfect, partially due to the strong interdependency of syntactic and semantic features in text data. in recent years, we have witnessed extensive progress in language models (lm). large-scale lms such as bert @xcite , xl-net @xcite , and gpt-2 @xcite , are commonly trained on large amounts of text data (e.g., gpt-2 was trained on 8 million web pages that emphasized content diversity). one of the most interesting usages of these models is utilizing them as text generators @xcite @xcite . in this paper, we explore whether we can leverage the generation ability of the state-of-the-art lms, to generate augmented samples for a given target class. augmentation samples should exhibit features of the target class. off-the-shelf lms cannot be directly used to augment data; since they are not trained for specific contexts, their generation is undirected and random. conditional lms can generate text directed by certain condition (e.g., target class), but they require training a lm from scratch with data covering all the conditions. @xcite , for instance, trained a 1.6 billion-parameter lm conditioned to a variety of control codes. the training is rather costly; however, collecting sufficient data for the training is also tedious, especially in low-resource tasks @xcite . we thus present data boost: a reinforcement learning guided text data augmentation framework built on off-the-shelf lm (gpt-2). data boost requires neither collecting extra data nor training a task-specific lm from scratch. we convert gpt-2 into a conditional generator, and for a given task, we guide the generator towards specific class labels during its decoding stage through reinforcement learning. the generated samples can then serve as augmentation data which are similar to the original data in terms of semantics and readability. the advantages of data boost are three-fold: first, data boost is powerful. we achieve significant advances in three tasks on five different classifiers compared with six related works. second, data boost generates sentence-level augmentation. unlike prior methods that do word-level or phraselevel replacement @xcite , our augmented data is of much greater variety in terms of vocabulary and sentence structure. human evaluations also verify the high readability and label consistency of our augmentation. third, data boost is easy to deploy. it does not require external datasets or training separate systems (like machine translation model in the back-translation method). instead, we take the off-the-shelf gpt-2 language model and modify its decoding stage without changing its architecture. ## tasks & datasets we evaluated and compared data boost with several state-of-the-art text augmentation methods on the following three tasks: offense detection foot_2 icwsm 20' data challenge dataset (@xmath0, 603) for offensive language detection on tweets. the dataset consists of four classes: {normal, spam, abusive and hateful} with ratio {53.9%, 27.1%, 14.1%, 4.9%} respectively. sentiment analysis foot_3 semeval 2017 task 4a dataset (@xmath1, 631) for sentiment analysis in tweets. there are three classes in the dataset: {positive, neutral and negative} with ratio {34.7%, 49.8%, 15.5%}. irony classification foot_4 semeval 2018 task 3a dataset (@xmath2, 817) for irony detection in tweets. it has binary classes: {ironic, non-ironic}, with ratio {50.2%, 49.8%}. offense detection and irony classification are popular nlu tasks that are low-resource. sentiment analysis, though seemingly well-resolved according to some literature @xcite , is reported to have severe overfitting problems when given extremely limited training data @xcite . we choose challenging datasets varying in total data size (n ≈ 80k, 17k, 3k) and the number of class (# of @xmath3, 3, 2) for a realistic evaluation of our framework. we removed all punctuation, stop words, hashtags and url links in the samples for all datasets. samples whose length was above 30 tokens were filtered out (around 2% of the data on average) as 30 was also used as the max sequence length for data boost generation. we further split the data into training and test set by the ratio {80%, 20%}, and maintained the original class distributions. we made sure the distributions remained the same in all of our experiments. ## experiments 6 we conducted extensive experiments to answer the following three overarching questions about data boost: ## comparison with related work a few words about conditional generation: ctrl @xcite and bart @xcite are large-scale conditional lms trained ## limitations in this section we discuss the limitations of data boost. the performance gain achieved by using data boost could be marginal on certain tasks, especially those whose classes cannot be modeled well by lexical features. for example, we experimented with data boost for metaphor detection using the lcc dataset @xcite , sarcasm classification using the ghosh dataset @xcite , and formality detection using the gyafc formality style transfer dataset @xcite . we saw marginal improvements in the tasks, with an absolute increase in f1 scores of 1.3%, 0.9%, and 0.7% for the three tasks re-spectively (in the extreme data scarcity case, where we expect data boost to help the most; i.e., when boosting 1% of the original data to 80%). we found that it was difficult for our model to extract explicit lexical features for the metaphor, sarcastic, and formal classes. this could be because syntactic features play a role in these classes. it is challenging for data boost to compose meaningful augmentation in such cases, given that our guidance on the generation is token-by-token. ## conclusion we have proposed a powerful and easy to deploy approach to augment text data through conditional generation. by leveraging an off-the-shelf language model (gpt-2), we successfully guide the generation towards a specified direction (i.e, target class), with the help of reinforcement learning. we find that data boost improves the performance of classification tasks, is classifier-agnostic, and that it surpasses several prior augmentation methods in three diverse classification tasks. in the future, we plan to implement a more sophisticated guidance for the augmentation by adding syntactic and position features to the reward function, to enable augmentation of more diverse types of text data. the code will be made available upon request.