Defense against Synonym Substitution-based Adversarial Attacks via D irichlet Neighborhood Ensemble (2021)

Abstract: Although deep neural networks have achieved prominent performance on many NLP tasks, they are vulnerable to adversarial examples. We propose Dirichlet Neighborhood Ensemble (DNE), a randomized method for training a robust model to defense synonym substitution-based attacks. During training, DNE forms virtual sentences by sampling embedding vectors for each word in an input sentence from a convex hull spanned by the word and its synonyms, and it augments them with the training data. In such a way, the model is robust to adversarial attacks while maintaining the performance on the original clean data. DNE is agnostic to the network architectures and scales to large models (e.g., BERT) for NLP applications. Through extensive experimentation, we demonstrate that our method consistently outperforms recently proposed defense methods by a significant margin across different network architectures and multiple data sets.

## introduction deep neural networks are powerful but vulnerable to adversarial examples that are intentionally crafted to fool the networks. recent studies have shown the vulnerability of deep neural networks in many nlp tasks, including reading comprehension @xcite ), text classification @xcite @xcite , machine translation @xcite @xcite , dialogue systems @xcite , and dependency parsing @xcite . these methods attack an nlp model by replacing, scrambling, and erasing characters or words under certain semantic and syntactic constraints. in particular, most of them craft adversarial examples by substituting words with their synonyms in an input text to maximally increase the prediction error while maintaining the adversarial examples' fluency and naturalness. in this paper, we focus on these word substitution-based threat models and discuss the strategy to defend against such attacks. the goal of adversarial defenses is to learn a model capable of achieving high test accuracy on both clean and adversarial examples. adversarial training is one of the most successful defense methods for nlp models @xcite @xcite . during the training time, they replace a word with one of its synonyms that maximizes the prediction loss. by augmenting these adversarial examples with the original training data, the model is robust to such perturbations. however, it is infeasible to explore all possible combinations where each word in a sentence can be replaced with any of its synonyms. also, when updating word embeddings during training, the distance between a word and its synonyms in the embedding space change dynamically. therefore, the point-wise guarantee becomes insufficient, and the resulting models have shown to be vulnerable to strong attacks @xcite . on the other hand, several certified defense methods have recently been proposed to ensure that the model predictions are unchanged when input word embeddings are perturbed within the convex hull formed by the embeddings of a word and its synonyms @xcite . however, due to the difficulty of propagating convex hull through deep neural networks, they compute a loose outer bound using interval bound propagation (ibp). as a result, the convex hull may contain irrelevant words and lead to a significant performance drop on the clean data. in this paper, we propose dirichlet neighborhood ensemble (dne) to create virtual sentences by mixing the embedding of the original word in the input sentence with its synonyms. by training on these virtual sentences, the model can enhance the robustness against word substitution-based per-turbations. specifically, our method samples an embedding vector in the convex hull formed by a word and its synonyms to ensure the robustness within such a region. in contrast to ibp, our approach better represents the synonyms' subspace by creating virtual sentences. to deal with complex error surface (e.g., surfaces containing multiple hills and valleys), a gradient-guided optimizer is applied to search for the most vulnerable points within the convex hull. by minimizing the error with these vulnerable points, we can guarantee with high probability that the resulting model is robust at any point within the convex hull (i.e., a set of synonyms). the framework can be extended to higher-order neighbors (synonyms) to boost the robustness further. in the inference time, the same dirichlet sampling technique is used, and the prediction scores on the virtual sentences are ensembled to get a robust output. through extensive experiments with various model architectures on multiple data sets, we show that dne consistently achieves better performance on clean and adversarial samples than existing defense methods. by conducting a detailed analysis, we found that dne enables the embeddings of a set of similar words to be updated together in a coordinated way. in contrast, prior approaches either fix the word vectors during training (e.g., in the certified defenses) or update individual word vectors independently (e.g., in the adversarial training). we believe it is the crucial property why dne leads to a more robust nlp model. furthermore, unlike most certified defenses, the proposed method is easy to implement and can be integrated into any existing neural network including those with large architecture such as bert @xcite . ## related work in the text domain, adversarial training is one of the most successful defenses @xcite @xcite . a family of fast-gradient sign methods (fgsm) was introduced by @xcite to generate adversarial examples in the image domain. they showed that the robustness and generalization of machine learning models could be improved by including high-quality adversarial examples in the training data. @xcite proposed an fgsmlike adversarial training method to the text domain by applying perturbations to the word embeddings rather than to the original input itself. @xcite extended the work of @xcite to improve the interpretability by constraining the directions of perturbations toward the existing words in the word embedding space. @xcite applied several types of noises to perturb the input word embeddings, such as gaussian, bernoulli, and adversarial noises, to mitigate the overfitting problem of nlp models. @xcite proposed a novel adversarial training algorithm, called freelb (free large-batch), which adds adversarial perturbations to word embeddings and minimizes the resultant adversarial loss inside different regions around input samples. they add norm-bounded adversarial perturbations to the input sentences' embeddings using a gradient-based method and enlarge the batch size with diversified adversarial samples under such norm constraints. however, they focus on the effects on generalization rather than the robustness against adversarial attacks. recently a set of certified defenses has been introduced, which guarantees robustness to some specific types of attacks. for example, @xcite and @xcite use a bounding technique, interval bound propagation (ibp) to formally verify a model's robustness against word substitutionbased perturbations. @xcite and @xcite proposed the robustness verification and training method for transformers based on linear relaxation-based perturbation analysis. however, these defenses often lead to loose upper bounds for arbitrary networks and result in a higher cost of clean accuracy. furthermore, due to the difficulty of verification, certified defense methods are usually not scalable and remain hard to scale to complex prediction pipelines. to achieve certified robustness on large architectures, @xcite proposed a certified robust method called safer which is structure-free. however, the base classifier of safer is trained by the adversarial data augmentation. as shown in our experiments, randomly perturbing a word to its synonyms performs poorly in practice. in the image domain, randomization has been shown to overcome many of these obstacles in the ibp-based defense. empirically, @xcite showed that random resizing and padding in the input domain could improve the robustness. @xcite proposed to add gaussian noise in both the input layer and intermediate layers of cnn in both training and inference time to improve the robustness. @xcite provided a certified guarantee of this method, and later on, the bound is significantly improved in @xcite . the resulting algorithm, called randomized smoothing, has become widely used in certifying `2 robustness for image classifiers. these random smoothing methods are very much under-explored in nlp models. the main reason is that the adversarial examples in texts are usually generated by word substitution-based perturbations instead of small `p norm. in this paper, we show that random smoothing can be integrating with adversarial training to boost the empirical robust accuracy. ## method we here consider a word substitution-based threat model, where every word in an input sentence can be replaced with one of its synonyms. given a sentence and synonym sets, we would like to ensure that the prediction of a model trained with our method cannot be altered by any word substitutionbased perturbation to the sentence. however, the number of possible perturbations scales exponentially with sentence length, so data augmentation cannot cover all perturbations of an input sentence. we use a convex hull formed by a word and its synonyms to capture word substitutions, which allows us to search for the worse-case over the convex hull. by minimizing the error with the worst-case, we can guarantee with high probability that the model is robust at any point within the convex hull (i.e., a set of synonyms). the proposed method can be viewed as a kind of randomized defense on nlp models, where our main contribution is to show that it is essential to ensure the model works well in a region within the convex hull formed by the embeddings of a word and its synonyms instead of only ensuring model is good under discrete perturbation. although dne does not provide certified lower bounds like ibp, it achieves much better accuracy on both clean and adversarial data on different models, datasets, and attacks compared with ibp. dne also can be easily integrated into any neural networks, including large architecture such as bert. let f be a base classifier which maps an input sentence x 2 x to a class label y 2 y. we consider the setting where for each word x i in the sentence x, we are given a set of its synonyms s(x i ) including x i itself, where we know replacing x i with any of s(x i ) is unlikely to change the semantic meaning of the sentence 1 . we relax the set of discrete points (a word and its synonyms) to a convex hull spanned by the word embeddings of all these points, denoted by c(x i ). we assume any perturbation within this convex hull will keep the semantic meaning unchanged, and define a smoothed classifier g(x) based on random sampling within the convex hull as follows. note that it is impossible to precisely calculate the probabilities with which f classifies x as each class, so we use a monte carlo algorithm for evaluating g(x). as shown in fig. 1 (a ), for a sentence x, we draw k samples of x by running k noisecorrupted copies of x through the base classifier f (x), where x is generated by replacing the embedding of every word x j in the sentence x with a point randomly sampled from c(x j ) (the pentagon with yellow dashed borders). if the class y appeared with maximal weight in the categorical distribution x, the smoothed classifier g(x) returns y. the decision regions of the base classifier are drawn in different colors if we evaluate the smoothed classifier at an input x j , where the regions with different colors represent different classes. assuming that the word x i is replaced with x j by an adversary, we need to sample the points from the convex hull c(x j ) in the inference time. however, some of x j 's synonyms (indicated by yellow circles) are outside the region formed by x i and its synonyms (indicated by blue circles). we thus should expand this region to the polygon with green dashed borders to make sure that the model makes the same prediction for any point sampled from the expanded region. we ensure that the smoothed 1 follow @xcite , we base our sets of word substitutions s(xi) on the method of @xcite . figure 1: consider a word (sentence of length one) x i and its convex hull c(x i ) (projected to 2d for illustration) spanned by the set of its synonyms (blue circles). we assume that an adversary replaces x i with one of its synonyms x j . (a) evaluating the smoothed classifier at the input x j . the decision regions of the base classifier f are drawn in blue, green, and pink colors, representing different classes. if we expand c(x i ) to the polygon with green dashed borders when training the base classifier f , the size of the intersection of this polygon and c(x j ) is large enough to ensure that the smoothed classifier g labels x j as f (x i ). here, the region where g labels x j as f (x i ) is "blue." (b) an example convex hull used to train the base classifier. since the size of the intersection of c(x i ) and c(x j ) is small, we expand c(x i ) to the convex hull spanned by x i 's neighbors and "neighbors of neighbors" in their embedding space when training the base classifier f . starting from three points v 1 , v 2 and v 3 sampled from the expanded convex hull (the largest polygon with green dashed borders), q 1 , q 2 and q 3 are the local "worst-case" points found by searching over the entire convex hull with the gradient-guided optimization method. ## experiments we conducted experiments on multiple data sets for text classification and natural language inference tasks. various model architectures (bag-of-words, cnn, lstm, and attention-based) were used to evaluate our dne and other defense methods under two recently proposed attacks. ren et al. (2019) described a greedy algorithm, called probability weighted word saliency (pwws), for adversarial text attacks based on word substitutions with synonyms. the word replacement order is determined by taking both word saliency and prediction probability into account. alzantot et al. (2018) developed a generic algorithm-based attack, denoted by ga, to generate semantically and syntactically similar adversarial examples. they use a language model (lm) @xcite to rule out candidate substitute words that do not fit within the context. however, unlike pwws, ruling out some candidates by the lm will significantly reduce the number of candidate substitute words (65% off on average). for a fair comparison, we report the robust accuracy under ga attack both with and without using the lm. we measure adversarial accuracy on perturbations found by the two attacks (pwws and ga) on 1, 000 randomly selected test examples for each data set. we primarily compare with recently proposed defense methods, including adversarial training (adv) @xcite and the interval bound propagation (ibp) based methods @xcite . the former can improve the model's robustness without suffering many drops on the clean input data by adding adversarial examples in the training stage. the latter was shown to be more robust to word substitution-based perturbations than ones trained with data augmentation. to demonstrate that mixing the embedding of the original word with its synonyms performs better than naively replacing the word with its synonyms, we designed a new baseline, called ran. the models trained by ran will take the corrupted copies of each input sentence as inputs, in which every word of the sentence is randomly replaced with one of its synonyms. the same random replacement is used in the inference time, and the prediction scores are ensembled to get an output. ran can be viewed as a variant of safer @xcite , where during the training safer's perturbation set is replaced with the synonym set used by the adversaries and the number of ensembles is reduced to 16 (instead of 5, 000) at the inference time, which make it feasible to be evaluated empirically under the attacks. ## conclusion in this study, we develop a novel defense algorithm for nlp models to substantially improve the robust accuracy without sacrificing their performance too much on clean data. this method is broadly applicable, generic, scalable, and can be incorporated with little effort in any neural network, and scales to large architectures. a novel adversarial training algorithm is also proposed, enabling nlp models to defend against the strong attacks that search for the worst-case over all combinations of word substitutions. we demonstrated through extensive experimentation that our adversarially trained smooth classifiers consistently outperform all existing empirical and certified defenses by a significant margin on three datasets across different network architectures, establishing state-of-the-art for defenses against adversarial text attacks. we choose to focus on synonym swapping because it is one of the most influential and widelyused attack methods. there is still no effective method to defend against existing attack algorithms from this kind, such as hotflip @xcite , pwws (2019), ga (2018), textfooler @xcite etc. a general method to defend more different attacks is worth exploring, but we choose to leave this as future work.