Gradient-based Adversarial Attacks against Text Transformers (2021)

Abstract: We propose the first general-purpose gradient-based adversarial attack against transformer models. Instead of searching for a single adversarial example, we search for a distribution of adversarial examples parameterized by a continuous-valued matrix, hence enabling gradient-based optimization. We empirically demonstrate that our white-box attack attains state-of-the-art attack performance on a variety of natural language tasks, outperforming prior work in terms of adversarial success rate with matching imperceptibility as per automated and human evaluation. Furthermore, we show that a powerful black-box transfer attack, enabled by sampling from the adversarial distribution, matches or exceeds existing methods, while only requiring hard-label outputs.

## introduction deep neural networks are sensitive to small, often imperceptible changes in the input, as evidenced by the existence of so-called adversarial examples @xcite . the dominant method for constructing adversarial examples defines an adversarial loss, which encourages prediction error, and then minimizes the adversarial loss over the input space with established optimization techniques. to ensure that the perturbation is hard to detect by humans, existing methods also introduce a perceptibility constraint into the optimization problem. variants of this general strategy have been successfully applied to image and speech data @xcite @xcite . however, optimization-based search strategies for obtaining adversarial examples are much more challenging with text data. attacks against continuous data types such as image and speech utilize gradient descent for superior efficiency, but the discrete nature of natural languages prohibits such first-order techniques. in addition, perceptibility for continuous data can be approximated with l 2and l ∞ -norms, but such metrics are not readily applicable to text data. to circumvent this issue, some existing attack approaches have opted for heuristic word replacement strategies and optimizing by greedy or beam search using black-box queries @xcite @xcite . such heuristic strategies typically introduce unnatural changes that are grammatically or semantically incorrect @xcite . in this paper, we propose a general-purpose framework for gradient-based adversarial attacks, and apply it against transformer models on text data. our framework, gbda (gradient-based distributional attack), consists of two key components that circumvent the difficulties of gradient descent for discrete data under perceptibility constraints. first, instead of constructing a single adversarial example, we search for an adversarial distribution. we instantiate examples with the gumbelsoftmax distribution @xcite , parameterized by a continuous-valued matrix of coefficients that we optimize with a vanilla gradientbased method. second, we enforce perceptibility and fluency using bertscore @xcite and language model perplexity, respectively, both of which are differentiable and can be added to the objective function as soft constraints. the combination of these two components enables powerful, efficient, gradient-based text adversarial attacks. we empirically demonstrate the efficacy of gbda against several transformer models. in addition, we also evaluate under the transfer-based black-box threat model by sampling from the optimized adversarial distribution and querying against a different, potentially unknown target model. on a variety of tasks including news/article categorization, sentiment analysis, and natural language inference, our method achieves state-of-the-art attack success rate, while preserving fluency, grammatical correctness, and a high level of semantic similarity to the original input. in summary, the main contributions of our paper are as follows: 1. we define a parameterized adversarial distribution and optimize it using gradient-based methods. in contrast, most prior work construct a single adversarial example using black-box search. 2. by incorporating differentiable fluency and semantic similarity constraints into the adversarial loss, our white-box attack produces more natural adversarial texts while achieving a new state-ofthe-art success rate. 3. the adversarial distribution can be sampled efficiently to query different target models in a blackbox setting. this enables a powerful transfer attack that matches or exceeds the performance of existing attacks. compared to prior work that operate on continuous-valued outputs from the target model, this transfer attack only requires hard labels. ## background adversarial examples constitute a class of robustness attacks against neural networks. let h : @xmath0, y are the input and output domains, respectively. suppose that @xmath1. an (untargeted) adversarial example is a sample @xmath2. the notion of perceptibility is introduced so that x preserves the semantic meaning of x for a human observer. at a high level, x constitutes an attack on the model's robustness if a typical human would not misclassify x but the model h does. for image data, since the input domain x is a subset of the euclidean space r d , a common surrogate for perceptibility is a distance metric such as the euclidean distance or the chebyshev distance. in general, one can define a perceptibility metric ρ : x × @xmath3, x ) ≤ . ## gbda: gradient-based distributional attack in this section, we detail gbda-our generalpurpose framework for gradient-based text attacks against transformers. our framework leverages two important insights: (1) we define a parameterized adversarial distribution that enables gradientbased search using the gumbel-softmax @xcite ; and (2) we promote fluency and semantic faithfulness of the perturbed text using soft constraints on both perplexity and semantic similarity. ## experiments in this section, we empirically validate our attack framework on a benchmark suite of natural language tasks. code to reproduce our results is open sourced on github foot_0 . ## conclusion and future work we presented gbda, a framework for gradientbased white-box attack against text transformers. our approach overcomes many ad-hoc constraints and limitations from the existing text attack literature by leveraging a novel adversarial distribution formulation, allowing end-to-end optimization of the adversarial loss and fluency constraints with gradient descent. this makes our method generic and potentially applicable to any model for token sequence prediction. limitations. one clear limitation of gbda is its restriction to only token replacements. indeed, our adversarial distribution formulation using the gumbel-softmax does not trivially extend to token insertions and deletions. this limitation may adversely affect the naturalness of the generated adversarial examples. we hope to extend our framework to incorporate a broader set of token-level changes in the future. in addition, the adversarial distribution p θ is highly over-parameterized. despite most adversarial examples requiring only a few token changes, the distribution parameter θ is of size n×v , which is especially excessive for longer sentences. future work may be able to reduce the number of parameters without affecting attack performance.