EP i DA : An Easy Plug-in Data Augmentation Framework for High Performance Text Classification (2022)

Abstract: Recent works have empirically shown the effectiveness of data augmentation (DA) in NLP tasks, especially for those suffering from data scarcity. Intuitively, given the size of generated data, their diversity and quality are crucial to the performance of targeted tasks. However, to the best of our knowledge, most existing methods consider only either the diversity or the quality of augmented data, thus cannot fully mine the potential of DA for NLP. In this paper, we present an easy and plug-in data augmentation framework EPiDA to support effective text classification. EPiDA employs two mechanisms: relative entropy maximization (REM) and conditional entropy minimization (CEM) to control data generation, where REM is designed to enhance the diversity of augmented data while CEM is exploited to ensure their semantic consistency. EPiDA can support efficient and continuous data generation for effective classifier training. Extensive experiments show that EPiDA outperforms existing SOTA methods in most cases, though not using any agent networks or pre-trained generation networks, and it works well with various DA algorithms and classification models.

## introduction data augmentation (da) is widely-used in classification tasks @xcite @xcite . in computer vision (cv), @xcite @xcite adopt strategies like flipping, cropping, tilting to perform da. in natural language processing (nlp), @xcite @xcite find that native augmentation skills such as spelling errors, synonym replacement, deleting and swapping, can bring considerable performance improvement. all these methods use generally, given the size of generated data, their diversity and quality are crucial to the performance of targeted tasks @xcite . recent works have begun to emphasize the diversity or quality of augmented data. for example, in cv, aa @xcite , fast-aa @xcite and lta @xcite employ agent networks to learn how to enhance diversity. in nlp, language models are widely used to control generation quality, including back-translation @xcite , seq2seq models @xcite @xcite , gpt-2 @xcite @xcite and @xcite . in addition, some works @xcite in nlp utilize adversarial augmentation to enrich the diversity of the samples. however, to the best of our knowledge, most existing works consider only either the quality or the diversity of augmented data, so cannot fully exploit the potential of data augmentation for nlp tasks. besides, recent existing da methods for nlp tasks usually resort to pre-trained language models, are extremely inefficient due to huge model complexity and tedious finetuning, which limits the scope of their applications. in this paper, we propose a new data augmentation framework for text classification. this framework is called epida (the abbreviation of easy plug-in data augmentation), which employs two mechanisms to control the diversity and quality da seas input sample candidate samples classifier selected samples high diversity samples (scored by rem) high quality samples (scored by cem) high diversity and quality samples (rem+cem) feedback candidate samples { } { } { } { } t c figure 1: the pipeline of epida. epida first evaluates the candidate samples from the perspectives of diversity and quality, then takes the samples of high diversity and quality as training samples. it can work with various da algorithms and classification models. ## related work in this section, we first review the related work of da for nlp, then expound the differences between our method and the major existing ones. according to the methodology of data generation, existing methods can be categorized into three types: rule-based, interpolation-based, and model-based, respectively. ## method as shown in fig. 1 , epida consists of three components: a da algorithm t , a classifier or classification model c, and a sample evaluation and selection (seas) module that is the core component of epida. generally, the da algorithm and the classifier can be any of existing da algorithms and classifiers. with the feedback of the classifier, seas evaluates candidate samples generated by the da algorithm in terms of diversity and quality via the relative entropy maximization (rem) mechanism and the conditional entropy minimization (cem) mechanism, and outputs the qualified samples to further train the classifier. so epida can serve as a plug-in component to boost existing da algorithms for training better target models. ## performance evaluation in this section, we conduct extensive experiments to evaluate epida, including performance comparison with sota methods, performance evaluation when working with different da algorithms and classification models, ablation study, and qualitative visualization of samples augmented by epida. ## conclusion in this paper, we present an easy plug-in data augmentation technique epida to control augmented data diversity and quality via two mechanisms: relative entropy maximization and conditional entropy minimization. through extensive experiments, we show that epida outperforms existing methods, and can work well with different da algorithms and classification models. epida is general, effective, efficient, and easy-to-deploy. in the future, more verification of our method is expected to be conducted on other classification tasks.