Weight Perturbation as Defense against Adversarial Word Substitutions (2022)

Abstract: The existence and pervasiveness of textual adversarial examples have raised serious concerns to security-critical applications. Many methods have been developed to defend against adversarial attacks for neural natural language processing (NLP) models.Adversarial training is one of the most successful defense methods by adding some random or intentional perturbations to the original input texts and making the models robust to the perturbed examples.In this study, we explore the feasibility of improving the adversarial robustness of NLP models by performing perturbations in the parameter space rather than the input feature space.The weight perturbation helps to find a better solution (i.e., the values of weights) that minimizes the adversarial loss among other feasible solutions.We found that the weight perturbation can significantly improve the robustness of NLP models when it is combined with the perturbation in the input embedding space, yielding the highest accuracy on both clean and adversarial examples across different datasets.

## introduction deep neural networks (dnns) have achieved impressive results in a wide range of domains, but they were found to be vulnerable to adversarial examples maliciously crafted by adding a small perturbation to original examples @xcite . many studies have demonstrated the vulnerability of dnns on various natural language processing (nlp) tasks, including machine translation @xcite , dialogue systems @xcite ) and text classification @xcite @xcite @xcite . these methods attack an nlp model by replacing, scrambling, and erasing characters or words under certain semantic and syntactic constraints. the existence and pervasiveness of textual adversarial examples have raised serious concerns, espe-cially when nlp models are deployed to securitysensitive applications. many methods have been proposed to defend against adversarial attacks for neural nlp models, including adversarial data augmentation @xcite , adversarial training @xcite and certified defense @xcite @xcite . most of them improve the adversarial robustness of nlp models by applying some perturbations on input data and making the models robust to these perturbations in the input space. for example, one of the most effective methods is adversarial training that applies a min-max optimization into the training process by adding (usually gradient-guided) perturbations to the input embeddings @xcite @xcite . by augmenting these perturbed examples with the original training data, the models are robust to such perturbations. however, it is infeasible to enumerate and explore all possible inputs that would be fed to models by adversaries. in this study, we want to explore the feasibility of enhancing the robustness of neural nlp models by performing weight perturbations in the parameter space. the weight perturbation is useful to find a better solution in the parameter space (i.e., the weights) that minimizes the adversarial loss among other feasible solutions. adversarial weight perturbation @xcite has been investigated in the image domain, but our preliminary experiments show that their methods to implement weight perturbation cannot be trivially applied to nlp models. the existing weight perturbation results in inferior robustness and requires a long training time due to the discrete nature of texts. we found that the weight perturbation works better for nlp models when it is combined with the perturbation in the input feature space. based on this finding, we propose a mixed adversarial training method with accumulated weight perturbation, named mawp. the mixed adversarial training was designed to boost the model's robustness by combining the weight perturbation with the traditional adversarial training (i.e., perturbation in the input embedding space like freelb @xcite ). in this way, the resulting models can benefit more from the weight perturbation by exposing them to the input perturbation during the training process. the accumulated weight perturbation was mainly introduced to accelerate the training process while the model's robustness can be further improved. the accumulated perturbation takes the smoothed form of a weighted sum of gradient descents calculated in the previously-performed weight perturbations, which carries the global gradient information and gives a clear signal in which direction the parameters should move to aggressively if the successive gradients point in a similar direction. through extensive experiments, we demonstrate that our method can boost the robustness of nlp models to a great extent while suffering no or little performance drop on the clean data across three different datasets. ## preliminary in the following, we first introduce the traditional adversarial training that performs perturbation in the input feature space, and then give a brief review of the adversarial weight perturbation. before diving into the details, we need to set up some notations. a training dataset @xmath0, y i )} n @xmath1, ..., c} pairs, where d is the size of feature vectors and c is the number of classes. given a neural text classifier with a set of trainable weights w and a loss function of l(w, x, y), the regular training aims to find the values of weights w that minimizes the empirical risk of e (x,y)∼d [l(w, x, y)]. in adversarial training, we denote the adversarial perturbation to input feature vectors x as δ, the weight perturbation to model's weights w as ϵ w , and the number of ascent steps as @xmath2, ..., k}. ## method in the following, we first introduce our accumulated weight perturbation method that is designed to accelerate the training process when fine-tuning a pre-trained language model. after then, we discuss how to combine it with adversarial training to further improve the robustness of nlp models. ## experiments we conducted three sets of experiments. the first one is to evaluate our mawp in both clean accuracy and adversarial robustness on several datasets under three representative attack algorithms, compared to seven baseline methods. the goal of the second one is to investigate the impact of the number of ascent steps and training epochs on the performance. in the third experiment, we would like to better understand the interpretability of adversarial training via visualizations. three widely-used text classification datasets were used for evaluation: stanford sentiment treebank (sst-2) @xcite , ag-news corpus (agnews) @xcite and internet movie database (imdb) @xcite . sst-2 has about 67, 000 sentences for binary categories, and imdb consists of about 50, 000 movie reviews for positive and negative sentiment classification, and agnews is a text classification dataset pertaining to four categories containing about 30, 000 news articles. for fair comparison, we used a bert-based model @xcite as the base model for all the defense methods. ## conclusion this study is among the first ones to explore the feasibility of improving the adversarial robustness of neural nlp models by performing the perturbations in the parameter space (i.e., weights) rather than the input feature space (i.e., word embeddings). we experimentally demonstrate that the weight perturbation can be used to find a better solution in the parameter space by minimizing the adversarial loss with a multi-step, gradient-guided optimization method. we also show that the proposed method is complementary to existing adversarial training methods, and the combination of our method and freelb achieved state-of-the-art accuracy on both clean and adversarial examples on multiple benchmark datasets.