# Data Augmentation Methods for Enhancing Robustness in Text Classification Tasks
###### Abstract  
Text classification is widely studied in natural language processing (NLP). Deep learning models, including large pre-trained models like BERT and DistilBERT, have achieved impressive results in text classification tasks. However, these models' robustness against adversarial attacks remains an area of concern. To address this concern, we propose three data augmentation methods to improve the robustness of such pre-trained models. We evaluated our methods on four text classification datasets by fine-tuning DistilBERT on the augmented datasets and exposing the resulting models to adversarial attacks to evaluate their robustness. In addition to enhancing the robustness, our proposed methods can improve the accuracy and F1-score on three datasets. We also conducted comparison experiments with two existing data augmentation methods. We found that one of our proposed methods demonstrates a similar improvement in terms of performance, but all demonstrate a superior robustness improvement. artificial intelligence; natural language processing; text classification; data augmentation; robustness improvement +
Footnote †: journal:  
0000-0001-659X/0000-0000-0000-0000-0000-0000-0000-0000-0000-00000-00000-00000-00000-00000-00000-00000-000000-000000-0000000000000000000000000000000  
antonyms to provide a semantic variation. Antipode-based methods, on the other hand, combine antonyms and words with opposite sentiments to create both semantic and lexical diversity, leading to an increased robustness. Overall, these approaches enhance the robustness of the generated text by incorporating a greater range of semantic and lexical variation. Pure Antonym-based data augmentation methods have been previously used for sentiment analysis, but the two other data augmentation methods we propose are entirely novel. These methods combine synonyms and similar sentiment words (Cognate-based) or antonyms and sentiment words (Antipode-based). This study defines robustness as accuracy under attack and the attack success rate. Accuracy under attack is the accuracy of a target model under an adversarial attack, measured as the number of samples that resist the attack out of the total number of samples. The attack success rate is the proportion of adversarial attacks that successfully change the prediction of a target model from correct to incorrect. These data augmentation processes and victim models are independent. Our proposed methods can potentially be utilized not only for the mentioned text classification tasks but also for tasks such as topic segmentation [18] and authorship attribution [19] that place a high value on robustness due to their semantic lexical diversity and potential for counterfeiting. We evaluated our proposed data augmentation methods on the AG-News [20], TREC [21; 22], SUBJ [23], and SMS-Spam [24] datasets. We fine-tuned DistilBERT [5] on the augmented datasets and then exposed the resulting models to adversarial attacks to evaluate their robustness. In addition to enhancing the robustness, our proposed methods improved the accuracy and F1-score on three datasets. We also conducted comparison experiments with two existing data augmentation methods: EDA [10] and CheckList [7]. We found that one of our proposed methods performs a similar improvement in terms of the performance, but all demonstrate a superior robustness improvement. Our contributions can be summarized as follows:  
1. To the best of our knowledge, we are the first to apply data augmentation techniques to improve the robustness. 2. We propose two novel data augmentation methods for text classification: Cognate-based methods, which combine synonyms and similar sentiment words, and Antipode-based methods, which combine antonyms and sentiment words. 3. We conducted experiments on the AG-News [20], TREC [21; 22], SUBJ [23], and SMS-Spam [24] datasets and demonstrated the effectiveness of our methods. 4.