Step by Step Loss Goes Very Far: Multi-Step Quantization for Adversarial Text Attacks (2023)

Abstract: We propose a novel gradient-based attack against transformer-based language models that searches for an adversarial example in a continuous space of tokens probabilities. Our algorithm mitigates the gap between adversarial loss for continuous and discrete text representations by performing multi-step quantization in a quantization-compensation loop. Experiments show that our method significantly outperforms other approaches on various natural language processing (NLP) tasks.

## introduction deep neural networks achieve impressive results, but their vulnerability to adversarial attacks causes major security threats and is a concern when interpreting or explaining model predictions. in computer vision, the most successful attack methods use gradient-based optimization techniques @xcite . they minimize adversarial loss function that encourages the prediction error and imperceptibility of a generated example. development of optimization-based attacks in nlp is much more challenging due to the discrete nature of text. recent methods @xcite overcome this limitation by performing a gradient descent in the continuous space of token representations and eventually quantizing them into discrete text. a quantization of a token can significantly change its embedding and cause an undesired change of the loss value, degrading the adversarial example. to our knowledge, all existing optimization-based nlp attacks quantize all tokens in a text at once, which creates a considerable gap between adversarial loss for continuous and discrete text representations. in this paper, we propose mango 1 (multistep quantization gradient-based adversarial op1 code available at github.com/gmum/mango. timizer): a novel optimization-based attack against transformer @xcite language models that mitigates the aforementioned gap by performing multi-step quantization in a quantizationcompensation loop. mango quantizes continuous token representations one by one and reoptimizes the adversarial example after each quantization to compensate undesired degradation of adversarial loss value. the construction of mango introduces interesting problems that are addressed in section 3. mango achieves superior performance in various nlp tasks, outperforming recent whitebox (optimization-based) and black-box attacks. ## related work adversarial attacks can be roughly divided into two categories: white-box attacks that have access to the internal model's states (e.g. gradient) and more common black-box attacks that only know outputs of the model. in our paper, we focus on a white-box version of our mango attack. in appendix d, we develop a version of mango that can be used in the loosened black-box setting. black-box methods most black-box nlp attacks define a space of character or word replacements and heuristically search it for an adversarial example @xcite . the search space is limited with semantic ad hoc constraints (e.g. limiting edit distance or restricting possible replacements to synonyms) to preserve the attack's imperceptibility. such constraints disallow some specific perturbations (e.g. replacing a word with its antagonist even if the semantics is preserved in the context of other perturbations) and tend to generate semantically incorrect examples @xcite . white-box methods many white-box methods use gradients to guide a heuristic search in a space of text perturbations @xcite @xcite . recent methods take a step further and perform gradient descent optimization. they aim to find an example that minimizes the adversarial loss function, which encourages the prediction error and the imperceptibility of the attack. because the similarity and fluency of an example are controlled by a powerful external model used in the loss, optimization-based methods do not require hand-crafted semantic constraints, making them more flexible than black-box ones. adapting gradient descent in nlp attacks is a challenging problem due to the discrete nature of the optimized text. @xcite overcome this issue by performing optimization in the continuous space of token embeddings and replacing each token with a possibly new token, which embedding is the closest to the optimized one. an alternative approach is the gbda method @xcite that optimizes a continuous distribution of stochastic one-hot vectors and repeatedly samples adversarial examples from the optimized distribution until it fools the attacked model. quantization both methods mentioned above quantize all continuous representations of tokens to a text at once. quantization of a single token may significantly change its embedding and cause an undesirable change of adversarial loss value. when quantizing all tokens at once, the changes accumulate to a considerable gap between adversarial loss for continuous and discrete text representations (see section 6). our mango mitigates this gap. ## mango this section describes our mango method. unlike other optimization-based methods that quantize all token representations at once, mango constitutes an entirely new algorithm that quantizes a token and compensates for the resulting change in an adversarial loss value in a step-by-step manner. the construction of mango introduces interesting problems that are addressed in the optimization, vector selection and candidates selection paragraphs and are further evaluated in section 5. ## experiments in this section, we evaluate mango on various nlp tasks and compare it to recent nlp attacks. baselines we compare our method with the latest white-box gbda attack @xcite , as well as recent black-box attacks implemented in tex-tattack @xcite : bert-attack @xcite , bae @xcite and textfooler @xcite . to emphasize the importance of multi-step quantization, we evaluate the naive version of mango that performs quantization in one step. mango, naive and gbda attacks use identical loss. all hyperparameters are listed in appendix a. tasks we attack bert models from textattack fine-tuned on three text classification tasks: ag news @xcite , yelp reviews @xcite , imdb @xcite , and mnli task for natural language inference, @xcite . in mnli p., an attack is allowed to modify only the premise, and in mnli h., only the hypothesis. for each task, we randomly select 1000 attack targets from the training set. we use a training set as it provides more challenging targets and is more relevant to adversarial training @xcite . ## ablation study in this section, we evaluate three solutions from section 3 that improve the core idea of multi-step quantization: figure 1 compares different mango settings. we may observe that selection of probability vector for quantization by maximal entropy ("max entropy") is better than selection by minimal entropy ("min entropy"). resetting the optimizer after every quantization enhances the performance for both "max entropy" and "min entropy" settings. finally, we see that mango benefits from using both token's probability and gradient direction to score token candidates. ## visualization of quantization gap to visualize the quantization gap between adversarial loss for continuous and discrete text representations, we compared adversarial losses of mango, gbda and a naive version of mango that does not use multi-step quantization. the comparison can be found in figure 2 . we observe that the naive method converges to the lowest value loss in the optimization phase, but the value explodes after quantization. the gbda method, which samples probability vectors that resemble discrete one-hot vectors using gumbel-softmax @xcite , reaches a higher minimum, but its quantization gap is much smaller than that of naive method. finally, in the case of mango, we observe sudden peaks and slow declines of loss values that correspond to the quantization-compensation loop, in which the quantization of single tokens is followed by the compensation of the quantization gap. after optimization, mango continues to quantize tokens step by step further decreasing the loss. mango obtains a significantly lower final adversarial loss than gbda and naive, avoiding the quantization gap. ## conclusion we developed mango, a novel optimizationbased attack against transformer models that mitigates the gap between adversarial loss for continuous and discrete text representations using a quantization-compensation loop. mango achieves superior results on various nlp tasks, outperforming recent black-box and optimizationbased attacks.