Generating Textual Adversaries with Minimal Perturbation (2022)

Abstract: Many word-level adversarial attack approaches for textual data have been proposed in recent studies. However, due to the massive search space consisting of combinations of candidate words, the existing approaches face the problem of preserving the semantics of texts when crafting adversarial counterparts. In this paper, we develop a novel attack strategy to find adversarial texts with high similarity to the original texts while introducing minimal perturbation. The rationale is that we expect the adversarial texts with small perturbation can better preserve the semantic meaning of original texts. Experiments show that, compared with state-of-the-art attack approaches, our approach achieves higher success rates and lower perturbation rates in four benchmark datasets.

## introduction recent studies have demonstrated that deep learning based natural language processing (nlp) models are vulnerable to adversarial attacks @xcite @xcite . various textual adversarial attack strategies have been proposed including character-level, word-level, and sentence-level attacks @xcite . in this paper, we focus on wordlevel attacks on text classifiers, especially the word substitution-based attacks, which are shown to be both effective and efficient @xcite @xcite @xcite . a successful adversarial attack should satisfy three requirements: 1) can fool the neural network models to make the wrong prediction; 2) the modification is slight and imperceptible to human judgments; and 3) can find the adversarial example in a reasonable run time. however, existing approaches suffer from various limitations. for example, the original a good film with a solid pedigree both in front of and, more specifically, behind the camera. pwws (11.76%) a commodity film with a solid pedigree both in front of and, more specifically, bum the camera. textfooler (29.4%) a better film with a stable pedigree both in newsweek of and, more concretely, behind the camcorder. bert-attack (29.4%) a fair positive with a solid pedigree both in after of and, more roughly, behind the canvas. tampers (5.88%) a harmless film with a solid pedigree both in front of and, more specifically, behind the camera. in this work, we propose textual adversarial attack with minimal perturbation in a reduced search space (tampers) to craft high quality adversarial texts. the major motivation is that, by substituting a vulnerable word with its synonyms or sememes, we expect that the adversarial text with small perturbation can have large semantic preservation. we model the word-level attacks as a combinatorial optimization problem @xcite . we then develop a two-step heuristic for solving this problem. in the first step search space reduction, we first build candidate lists for all the content words by combining synonyms from word-net @xcite and sememes from hownet @xcite similar to other word level attack strategies @xcite . to avoid searching in a huge search space, instead of searching through all content words to generate adversarial texts, we reduce the search space by finding a small set of vulnerable words that are sufficient to fool the neural network using a greedy algorithm. then, in the second step iterative search, we further minimize the perturbation by iteratively restoring the substituted words back to the original words while keeping the classifier making the wrong prediction. each time we restore a word, we adopt the genetic algorithm (ga) to search adversarial texts over the remaining substituted words. in this way, we expect to find an adversarial example with the least perturbation on the original text. we have conducted experiments on four benchmark datasets. the results show that our approach can successfully fool the fine-tuned bert model with high success attack rates and low perturbation rates. ## methodology given any text @xmath0, . . . , w n , . . . , w n } with n words and the true label y , consider a pretrained classifier that maps x to y , i.e., f : @xmath1. we assume the soft-label black-box setting where adversaries can query the classifier for classification probabilities on the given sample, but have no access to the model structures, parameters as well as training data. our goal is to craft an adversarial sample x * such that f (x * ) ̸ = y with minimal word substitutions. we propose a novel framework called textual adversarial attack with minimal perturbation in a reduced search space (tampers), which consists of two steps. the search space reduction step is to find vulnerable words with the goal of reducing the search space, while the iterative search step is to further minimize the perturbation so that the semantics of the original text can be well-preserved. the overview of tampers is shown in figure 1 . below we describe each step in detail. ## conclusions in this work, we developed a novel word-level attack framework, called tampers, that adopts a two-step approach with the goal of minimizing the perturbation. the first step reduces the search space and finds an initial adversarial text with the reduced perturbation. the second step then further minimizes the perturbation by iteratively restoring the substituted words back to the original words while keeping the classifier getting fooled. experimental results show that tampers generally achieves minimal perturbation in crafting adversarial examples while keeping a high success attack rate.