Reinforced Counterfactual Data Augmentation for Dual Sentiment Classification (2021)

Abstract: Data augmentation and adversarial perturbation approaches have recently achieved promising results in solving the over-fitting problem in many natural language processing (NLP) tasks including sentiment classification. However, existing studies aimed to improve the generalization ability by augmenting the training data with synonymous examples or adding random noises to word embeddings, which cannot address the spurious association problem. In this work, we propose an end-to-end reinforcement learning framework, which jointly performs counterfactual data generation and dual sentiment classification. Our approach has three characteristics:1) the generator automatically generates massive and diverse antonymous sentences; 2) the discriminator contains a original-side sentiment predictor and an antonymous-side sentiment predictor, which jointly evaluate the quality of the generated sample and help the generator iteratively generate higher-quality antonymous samples; 3) the discriminator is directly used as the final sentiment classifier without the need to build an extra one. Extensive experiments show that our approach outperforms strong data augmentation baselines on several benchmark sentiment classification datasets. Further analysis confirms our approach’s advantages in generating more diverse training samples and solving the spurious association problem in sentiment classification.

## introduction deep learning techniques (e.g., cnn, rnn, pretrained language models) have achieved great success in many natural language processing (nlp) tasks including sentiment classification. despite their promising results, recent studies reported that due to the over-fitting problem these models may easily fail in attacking examples with even little modification on real examples @xcite @xcite . researchers have attempted to address this issue from two main perspectives: data augmentation and adversarial perturbation. the former tries to augment the training data by generating synonymous sentences @xcite @xcite ; the latter aims to improve the generalization ability by applying perturbations to the word embeddings @xcite . although these methods have achieved sound performance, they still suffer from the spurious association problem. machine learning systems are trained to exploit the associations between the input features and the output labels to make accurate predictions. for example, if a neutral word (e.g.,"book") occurs more frequently in the positive class than in the negative class of the training data, "book" will have a spurious association with the positive class. recently, counterfactual data augmentation has shown to be an effective way to address the spurious association problem in sentiment classification @xcite @xcite @xcite . the core idea behind this line of work is to construct training and test samples by generating antonymous sentences and reversing its sentiment label. in the previous example, by generating an antonymous sample for each training sample, the frequency of "book" in the negative class will also increase, and thus the spurious association between "book" and the positive class will be alleviated. however, these methods still have three shortcomings: 1) they either relied on human efforts or resorted to rules for antonymous sample construction which is labor-intensive and timecosting. the diversity of generated samples is also limited; 2) they regarded antonymous sample generation and sentiment classification as two separate tasks, and pipeline them; 3) they mostly merged the generated antonymous samples into the original training set, and ignored the one-toone correspondence between the antonymous and original samples. in this paper, we propose an end-to-end reinforcement learning framework named reinforced counterfactual data augmentation (rcda) for joint counterfactual data augmentation and dual sentiment classification. the counterfactual sentence generation and the dual sentiment classification modules are regarded as a generator and a discriminator, and integrated in a reinforcement learning framework. first, the generator uses one-tomany antonym and synonym lists obtained from wordnet to generate massive antonymous candidates based on multi-label learning, and automatically select the best antonymous sentence based on reinforcement learning. second, the discriminator contains an original-side sentiment predictor and an antonymous-side sentiment predictor, which regards the original and antonymous samples as pairs to perform dual sentiment classification. the action reward in reinforcement learning is also computed based on both original and antonymous sides. finally, the discriminator can be directly used as the final sentiment classifier for the testing examples. extensive experiments on four benchmark datasets indicate that our approach significantly outperform strong data augmentation baselines. further analysis demonstrates that our method is more effective in generating diverse training samples and alleviating the spurious association problem in sentiment classification. the contributions of this paper can be summarized as follows: ## approach figure 1 illustrates the overall architecture of our framework, which contains two main modules: 1) antonymous sentence generator. given an original sentence, the generator replaces each word in the original sentence with one of its antonyms or synonyms from wordnet to generate a number of antonymous sentences as candidates; 2) dual discriminator. it contains an original-side sentiment predictor and an antonymous-side sentiment predictor, which regards the original and antonymous samples as pairs to perform dual sentiment prediction. ## conclusion in this paper, we propose an end-to-end reinforcement learning framework named reinforced counterfactual data augmentation (rcda) for joint counterfactual data augmentation and dual sentiment classification, to address the over-fitting problem and improve the generalization ability of sentiment classification models. rcda contains an antonymous sentence generator to automatically generate massive diverse antonymous sentences and a dual discriminator with an originalside sentiment predictor and an antonymous-side sentiment predictor, which are jointly optimized based on our reinforcement learning framework. experiments on four benchmark datasets show that our approach consistently outperforms strong data augmentation baselines. in-depth analysis demonstrates the advantage of our approach in generating diverse training data and alleviating the spurious association problem.